mirror of
https://github.com/laaielichter/Reverseport.git
synced 2026-11-07 10:16:04 -10:00
feat(port): unpack Unity asset bundles and .assets files in re_archive (#89)
This commit is contained in:
Roel authored and GitHub committed
1 parent
6e5e0fb98a
commit
de173ec0ca
18 files changed
+1796
-39
No files matched your search
@@ -134,8 +134,9 @@ explicit path or endpoint and do not need it.
|
||||
- ASAR or extracted JavaScript/Electron tree:
|
||||
`analyze_javascript_application`.
|
||||
- Archive or pack members: `re_archive` lists or extracts zip/apk/jar/asar,
|
||||
Godot .pck, Ren'Py .rpa, GameMaker data.win, FMOD/Wwise banks, Unreal .pak and
|
||||
Flash .swf. For IPA/MSIX/AppX/DMG or a PyInstaller executable, or when you need
|
||||
Godot .pck, Unity asset bundles and .assets (textures as PNG, meshes as OBJ,
|
||||
audio, objects as JSON), Ren'Py .rpa, GameMaker data.win, FMOD/Wwise banks,
|
||||
Unreal .pak and Flash .swf. For IPA/MSIX/AppX/DMG or a PyInstaller executable, or when you need
|
||||
Evidence IDs and integrity findings: `open_binary` with the local path, then
|
||||
`inspect_artifact`, starting with `detail: "summary"` for large inventories. A
|
||||
PyInstaller CArchive is the application layer; the bootloader is only the
|
||||
|
||||
@@ -39,7 +39,8 @@ unit, verified. No files at all? Give it a name or a store page.
|
||||
ReversePort gives agents one consistent way to do all of that:
|
||||
|
||||
- 🎮 **Game formats built in** — Godot `.pck` (also inside the `.exe`) and `.gdc`/`.scn`/`.res` back to an editable
|
||||
project, Unity IL2CPP `global-metadata.dat`, `.unitypackage`, GameMaker, Ren'Py, RPG Maker, Unreal `.pak`, FMOD and
|
||||
project, Unity asset bundles and `.assets` (textures as PNG, meshes as OBJ, audio, scripts in use), Unity IL2CPP
|
||||
`global-metadata.dat`, `.unitypackage`, GameMaker, Ren'Py, RPG Maker, Unreal `.pak`, FMOD and
|
||||
Wwise banks; it also recognises Quake PAK, Doom WAD, XNB and VPK. No engine install needed.
|
||||
- 🔍 **Understand anything** — engine and version of any project or exported game, dependency graphs, scenes, symbols
|
||||
in 30+ languages, executables, packs and unknown data. JS/TS monorepos (pnpm, Turborepo, Nx, Lerna, Rush, tsconfig
|
||||
@@ -196,6 +197,7 @@ the version you have and `reverseport skills get <name>` prints the matching gui
|
||||
| _What is this game made with, and how is it built?_ | `re_scan` → engine + version, languages, entry points, how it starts |
|
||||
| _I only have the exported Godot game, not the project._ | `re_archive` extracts the .pck, `re_decode` turns .gdc/.scn/.res/project.binary back into an editable project |
|
||||
| _A Unity IL2CPP game without its C# source._ | `re_il2cpp`: classes, methods and literals from global-metadata.dat, plus a C# skeleton |
|
||||
| _Get the art, models and audio out of my Unity build._ | `re_archive` on data.unity3d, .assets or a bundle: PNG, OBJ, FSB5, text, JSON, plus which C# scripts each object uses |
|
||||
| _What changed between build 43 and 44?_ | `re_compare` folder-to-folder, pack-to-pack and .NET types and methods, biggest changes first |
|
||||
| _Keep this investigation honest — I need to show why._ | `re_case`: evidence → claims that cite it → unknowns → decision → tests → report |
|
||||
| _Map this codebase and write ARCHITECTURE.md._ | `re_graph`, `re_symbols`, `re_report` |
|
||||
@@ -236,6 +238,7 @@ Agent: re_scan → Godot 4.3, 6 scripts, 5 scenes, main scene main.tscn, autolo
|
||||
| Source projects and exported games | Engine + version (Godot, Unity, Unreal, GameMaker, RPG Maker, Ren'Py, LÖVE, Defold, Bevy, MonoGame, web, Electron, Flutter, .NET, Go, Rust…), dependencies, cycles, scenes, symbols | Nothing |
|
||||
| Packs and archives | Godot `.pck` (also embedded in the `.exe`), zip/apk/jar, `.unitypackage`, tar.gz, asar — list and extract; Quake PAK, Doom WAD, XNB, VPK and more recognised | Nothing |
|
||||
| Unity IL2CPP | Classes, methods and literals from `global-metadata.dat`, plus a C# skeleton | Nothing |
|
||||
| Unity builds and asset bundles | Every object by class and name, scenes, C# scripts in use; textures → PNG, meshes → OBJ, audio, text, JSON (UnityFS LZ4/LZMA) | Nothing |
|
||||
| Native binaries | Mach-O, ELF, PE, wasm: functions, pseudocode, strings, xrefs, call paths, layout | Ghidra, Hopper or IDA for decompilation ([setup](docs/installation.md#hopper)) |
|
||||
| .NET assemblies | Metadata, CIL, native boundaries, type- and method-level build comparisons | Nothing |
|
||||
| Android | Manifest, classes, decompiled methods, references, resources, devices | JADX / Apktool / adb ([guide](docs/android-analysis.md)) |
|
||||
|
||||
@@ -83,6 +83,9 @@ Alles staat in de [changelog](https://github.com/laaielichter/reverseport/blob/m
|
||||
4, `re_save` leest savebestanden, `port_shader` zet shaders om, `port_convert {physics}` houdt springen en vallen
|
||||
gelijk, `port_trace` vindt de eerste tick waarop twee versies verschillen, en `re_il2cpp` en `port_rpgmaker` helpen
|
||||
bij Unity IL2CPP en RPG Maker.
|
||||
- **Unity-builds en asset bundles** — `re_archive` op `data.unity3d`, `sharedassets0.assets` of een bundle (LZ4/LZMA)
|
||||
toont elk object met klasse en naam, de scènes en welke C#-scripts de objecten gebruiken; `action:"extract"` schrijft
|
||||
textures als PNG, meshes als OBJ, audio (FSB5), TextAssets en objecten als JSON. Zonder Unity, zonder extra software.
|
||||
- **Monorepo's en websites** — `re_scan` en `re_graph` begrijpen JS/TS-monorepo's (pnpm, Turborepo, Nx, tsconfig
|
||||
paths, `#imports`, Deno-import-maps) en Rust-workspaces. `capture_web_screenshot` met een `selector` geeft van één element op een site een
|
||||
bijgesneden afbeelding, de HTML en de CSS.
|
||||
|
||||
@@ -14,7 +14,7 @@ Any client config: `reverseport connect <client>`.
|
||||
## Route by target
|
||||
|
||||
- Project folder or exported game build: `re_scan` → `re_graph` → `re_symbols` / `re_scene`; `re_report` for ARCHITECTURE.md.
|
||||
- Exported Godot game without the project: `re_archive` extract, then `re_decode` (.gdc/.scn/.res/project.binary → .gd/.tscn/.tres/project.godot). Unity IL2CPP game without source: `re_il2cpp` on global-metadata.dat. Executable, pack or unknown file, quick look: `re_binary`, `re_archive` (Godot .pck also inside .exe, zip/apk/jar, asar, Ren'Py .rpa, GameMaker data.win, FMOD .bank, Wwise .bnk, Unreal .pak, Flash .swf, Unity WebGL .data, Emscripten .data), `re_probe`, `re_extract`.
|
||||
- Exported Godot game without the project: `re_archive` extract, then `re_decode` (.gdc/.scn/.res/project.binary → .gd/.tscn/.tres/project.godot). Unity IL2CPP game without source: `re_il2cpp` on global-metadata.dat. Unity build or asset bundle: `re_archive` on data.unity3d, *.assets or the bundle (objects, scenes, scripts in use; extract → PNG, OBJ, audio, text, JSON). Executable, pack or unknown file, quick look: `re_binary`, `re_archive` (Godot .pck also inside .exe, Unity bundles and .assets, zip/apk/jar, asar, Ren'Py .rpa, GameMaker data.win, FMOD .bank, Wwise .bnk, Unreal .pak, Flash .swf, Unity WebGL .data, Emscripten .data), `re_probe`, `re_extract`.
|
||||
- Two builds: `re_compare`. Unity project with pink materials or missing scripts: `re_refs` (GUID audit). Translatable text out (.pot/.csv) and back in: `re_strings`. Does this GDScript/C# use real Godot 4.x/3 APIs (no Godot 3 names, no invented members)? `re_apicheck`. Godot 3 project to Godot 4: `port_upgrade` (converted copy + manual list), then `re_apicheck`. Save files (layers, fields, checksums, diff two saves, schema + data classes): `re_save`. Shaders between engines (ShaderLab/Shadertoy/GLSL → Godot, Godot → three.js/Unity) as marked first drafts: `port_shader`. Physics differences and converted numbers (gravity, step, damping, friction rules, layers): `port_convert {from, to, physics:{…}}`. RPG Maker MV/MZ → Godot: `port_rpgmaker`. Behaviour parity: `port_verify` flags timing/randomness/order statically, `port_trace` logs both engines per tick and compares.
|
||||
- Compiled code in depth: `open_binary` → `search_procedures` / `search_strings` → `procedure_pseudo_code`, `xrefs`, `procedure_callers` (Ghidra, Hopper or IDA).
|
||||
- .NET: `inspect_managed_artifact`. Android: `inspect_android_package`. Electron/web bundles: `analyze_javascript_application`. Live pages: `list_browser_targets`, `observe_web_session`.
|
||||
|
||||
+26
-21
@@ -1,6 +1,7 @@
|
||||
// Archive and pack listing/extraction: Godot PCK (standalone or embedded in the game executable), ZIP
|
||||
// (apk, jar, ipa, nupkg, docx…), tar / tar.gz (.unitypackage), Electron asar. Unity bundles and Unreal paks
|
||||
// are identified (header only). Encrypted content is reported, never decrypted. Extraction writes only into
|
||||
// (apk, jar, ipa, nupkg, docx…), tar / tar.gz (.unitypackage), Electron asar, Unity asset bundles and serialized
|
||||
// .assets files (unity.mjs), Unreal paks and more game formats (gamearchives.mjs). Encrypted content is reported,
|
||||
// never decrypted. Extraction writes only into
|
||||
// the ReversePort workspace (or the chosen output folder) and refuses paths that would escape it.
|
||||
import fs from 'node:fs/promises';
|
||||
import { createReadStream, createWriteStream } from 'node:fs';
|
||||
@@ -13,7 +14,8 @@ import { RPError, readAt, requireFile, compileMatcher, sha256, MiB } from './uti
|
||||
import { identify } from './formats.mjs';
|
||||
import { parsePE, parseELF } from './binary.mjs';
|
||||
import { safeJoin, workspaceDir, extractionName } from './workspace.mjs';
|
||||
import { readRpa, readGameMaker, readFmodBank, readWwiseBank, readUnrealPak, readSwf, readUnityWebData, readEmscriptenData } from './gamearchives.mjs';
|
||||
import { readRpa, readGameMaker, readFmodBank, readFsb5, readWwiseBank, readUnrealPak, readSwf, readUnityWebData, readEmscriptenData, swfLzmaBody } from './gamearchives.mjs';
|
||||
import { readUnityBundle, readUnityAssets, looksLikeSerializedFile } from './unity.mjs';
|
||||
|
||||
const MAX_ENTRIES = 2_000_000;
|
||||
|
||||
@@ -223,17 +225,7 @@ export async function readAsar(abs) {
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- header-only formats
|
||||
async function describeOnly(abs, size, id) {
|
||||
const head = await readAt(abs, 0, 512);
|
||||
if (id === 'unityfs') {
|
||||
let p = 8;
|
||||
const fmt = head.readUInt32BE(p); p += 4;
|
||||
const cs = () => { const e = head.indexOf(0, p); const s = head.toString('latin1', p, e); p = e + 1; return s; };
|
||||
const unityVersion = cs(), revision = cs();
|
||||
p += 8 + 8; // total size, compressed+uncompressed block info sizes
|
||||
const flags = head.readUInt32BE(p);
|
||||
return { format: 'unityfs', bundleFormat: fmt, unityVersion: unityVersion === '5.x.x' ? `${revision} (player)` : unityVersion, unityRevision: revision, compression: ['none', 'LZMA', 'LZ4', 'LZ4HC'][flags & 0x3f] ?? flags & 0x3f, entries: [], note: 'Asset bundles are identified but not unpacked in this version. Port from the Unity project source when you can.' };
|
||||
}
|
||||
async function describeOnly(abs, size) {
|
||||
const tail = await readAt(abs, Math.max(0, size - 512), Math.min(512, size));
|
||||
const m = tail.lastIndexOf(Buffer.from([0xe1, 0x12, 0x6f, 0x5a]));
|
||||
if (m >= 0) return { format: 'unreal-pak', pakVersion: tail.readUInt32LE(m + 4), entries: [], note: 'Unreal paks are identified but not unpacked. Port from the Unreal project source (Content/, Source/).' };
|
||||
@@ -255,15 +247,18 @@ export async function openArchive(file) {
|
||||
}
|
||||
if (kind === 'renpy-rpa') return { abs, size, ...(await readRpa(abs, size)) };
|
||||
if (kind === 'gamemaker-data') return { abs, size, ...(await readGameMaker(abs, size)) };
|
||||
if (head.toString('latin1', 0, 4) === 'FSB5') { const fsb = await readFsb5(abs, 0, size); if (fsb) return { abs, size, format: 'fsb5', fsb: { version: fsb.fsbVersion, codec: fsb.codec, samples: fsb.samples }, entries: fsb.entries, note: fsb.note }; }
|
||||
if (kind === 'fmod-bank') return { abs, size, ...(await readFmodBank(abs, size)) };
|
||||
if (kind === 'wwise-bank') return { abs, size, ...(await readWwiseBank(abs, size)) };
|
||||
if (kind === 'swf') return { abs, size, ...(await readSwf(abs, size)) };
|
||||
if (head.subarray(0, 16).toString('latin1').startsWith('UnityWebData') || /\.(data\.(gz|br)|unityweb)$/i.test(abs) || (/\.data$/i.test(abs) && /Build[\\/]/.test(abs))) { try { return { abs, size, ...(await readUnityWebData(abs)) }; } catch (e) { if (e.code !== 'BAD_FORMAT') throw e; } }
|
||||
if (/\.data$/i.test(abs)) { try { return { abs, size, ...(await readEmscriptenData(abs)) }; } catch (e) { if (e.code !== 'BAD_FORMAT') throw e; } }
|
||||
if (/\.pak$/i.test(abs) && kind !== 'zip') { try { return { abs, size, ...(await readUnrealPak(abs, size)) }; } catch (e) { if (e.code !== 'BAD_FORMAT') throw e; } }
|
||||
if (kind === 'unityfs' || /\.pak$/i.test(abs)) { const d = await describeOnly(abs, size, kind); if (d) return { abs, size, ...d }; }
|
||||
if (kind === 'unityfs' || /^Unity(Web|Raw)\0/.test(head.toString('latin1', 0, 9))) return { abs, size, ...(await readUnityBundle(abs, size)) };
|
||||
if (/\.pak$/i.test(abs)) { const d = await describeOnly(abs, size); if (d) return { abs, size, ...d }; }
|
||||
if (looksLikeSerializedFile(head, size)) return { abs, size, ...(await readUnityAssets(abs, size)) };
|
||||
try { return { abs, size, ...(await readPck(abs, size)) }; } catch (e) { if (e.code !== 'NOT_A_PCK') throw e; }
|
||||
throw new RPError('BAD_FORMAT', `Not a supported archive (${identify(head).name}). Supported: Godot .pck (also embedded in a game .exe/binary), zip/apk/jar/ipa, tar, tar.gz/.unitypackage, Electron .asar, Ren'Py .rpa, GameMaker data.win, FMOD .bank, Wwise .bnk, Unreal .pak, Flash .swf.`);
|
||||
throw new RPError('BAD_FORMAT', `Not a supported archive (${identify(head).name}). Supported: Godot .pck (also embedded in a game .exe/binary), zip/apk/jar/ipa, tar, tar.gz/.unitypackage, Electron .asar, Unity asset bundles and .assets/level files, Ren'Py .rpa, GameMaker data.win, FMOD .bank and .fsb, Wwise .bnk, Unreal .pak, Flash .swf.`);
|
||||
}
|
||||
|
||||
class Md5Pass extends Transform {
|
||||
@@ -276,16 +271,16 @@ export async function archiveTool({ file, action = 'list', match, limit = 500, o
|
||||
const want = compileMatcher(match);
|
||||
const selected = arc.entries.filter(e => want(e.path));
|
||||
const summary = { format: arc.format, file: arc.abs, size: arc.size };
|
||||
for (const k of ['packVersion', 'godotVersion', 'embedded', 'embeddedVia', 'start', 'encryptedDirectory', 'unityVersion', 'unityRevision', 'compression', 'pakVersion', 'bundleFormat', 'note', 'rpaVersion', 'keyed', 'renpy', 'chunks', 'game', 'counts', 'names', 'fsb', 'events', 'buses', 'vcas', 'bankVersion', 'bankId', 'objects', 'objectCount', 'namedIds', 'mountPoint', 'encryptedIndex', 'encryptionKeyGuid', 'compressionMethods', 'unreal', 'swfVersion', 'stage', 'tags', 'actionScript', 'documentClass', 'symbolClasses', 'frameLabels', 'classes', 'identifierStrings', 'loader', 'packageSize', 'engine', 'unity']) if (arc[k] !== undefined) summary[k] = arc[k];
|
||||
for (const k of ['packVersion', 'godotVersion', 'embedded', 'embeddedVia', 'start', 'encryptedDirectory', 'unityVersion', 'unityRevision', 'compression', 'pakVersion', 'bundleFormat', 'note', 'rpaVersion', 'keyed', 'renpy', 'chunks', 'game', 'counts', 'names', 'fsb', 'events', 'buses', 'vcas', 'bankVersion', 'bankId', 'objects', 'objectCount', 'namedIds', 'mountPoint', 'encryptedIndex', 'encryptionKeyGuid', 'compressionMethods', 'unreal', 'swfVersion', 'stage', 'tags', 'actionScript', 'documentClass', 'symbolClasses', 'frameLabels', 'classes', 'identifierStrings', 'loader', 'packageSize', 'engine', 'unity', 'blockCompression', 'blocks', 'encrypted', 'signature', 'assets']) if (arc[k] !== undefined) summary[k] = arc[k];
|
||||
const byExt = {};
|
||||
for (const e of arc.entries) { const ext = path.extname(e.path).toLowerCase() || '(none)'; byExt[ext] = byExt[ext] ?? { files: 0, bytes: 0 }; byExt[ext].files++; byExt[ext].bytes += e.size ?? 0; }
|
||||
const topTypes = Object.entries(byExt).sort((a, b) => b[1].bytes - a[1].bytes).slice(0, 25).map(([ext, v]) => ({ ext, ...v }));
|
||||
const max = Math.max(1, Math.min(20000, limit));
|
||||
if (action === 'list') {
|
||||
return { ...summary, totalEntries: arc.entries.length, matching: selected.length, totalBytes: arc.entries.reduce((s, e) => s + (e.size ?? 0), 0), types: topTypes, entries: selected.slice(0, max).map(({ localHeader, method, ...e }) => e), truncated: selected.length > max };
|
||||
return { ...summary, totalEntries: arc.entries.length, matching: selected.length, totalBytes: arc.entries.reduce((s, e) => s + (e.size ?? 0), 0), types: topTypes, entries: selected.slice(0, max).map(({ localHeader, method, produce, prefix, ...e }) => e), truncated: selected.length > max };
|
||||
}
|
||||
if (action !== 'extract') throw new RPError('INVALID_ARGUMENT', 'action must be "list" or "extract".');
|
||||
if (arc.format === 'unityfs' || (arc.format === 'unreal-pak' && !arc.entries.length)) throw new RPError('UNSUPPORTED_ARCHIVE', summary.note ?? 'nothing to extract');
|
||||
if (!arc.entries.length && ['unityfs', 'unity-bundle', 'unreal-pak'].includes(arc.format)) throw new RPError('UNSUPPORTED_ARCHIVE', summary.note ?? 'nothing to extract');
|
||||
const budget = Math.min(maxBytes ?? 4096 * MiB, 64 * 1024 * MiB);
|
||||
const root = out ? path.resolve(out) : await workspaceDir('extracted', extractionName(arc.abs, sha256(`${arc.abs}:${arc.size}`)));
|
||||
await fs.mkdir(root, { recursive: true });
|
||||
@@ -303,9 +298,9 @@ export async function archiveTool({ file, action = 'list', match, limit = 500, o
|
||||
if (written + (e.size ?? 0) > budget) { skipped.push({ path: e.path, reason: `over the ${Math.round(budget / MiB)} MiB extraction budget (raise maxBytes)` }); return false; }
|
||||
return true;
|
||||
};
|
||||
if (arc.format === 'swf' && arc.compression === 'zlib') {
|
||||
if (arc.format === 'swf' && arc.compression !== 'none') {
|
||||
const whole = await fs.readFile(arc.abs);
|
||||
const body = zlib.inflateSync(whole.subarray(8));
|
||||
const body = arc.compression === 'LZMA' ? swfLzmaBody(whole, whole.readUInt32LE(4)) : zlib.inflateSync(whole.subarray(8));
|
||||
const header = Buffer.from(whole.subarray(0, 8)); header.write('FWS', 0, 'latin1');
|
||||
const dest = path.join(root, 'swf-inflated.swf');
|
||||
await fs.writeFile(dest, Buffer.concat([header, body]));
|
||||
@@ -326,6 +321,16 @@ export async function archiveTool({ file, action = 'list', match, limit = 500, o
|
||||
if (!canWrite(e)) continue;
|
||||
const dest = safeJoin(root, e.path);
|
||||
await fs.mkdir(path.dirname(dest), { recursive: true });
|
||||
if (e.produce) {
|
||||
let made;
|
||||
try { made = await e.produce(); } catch (err) { if (!(err instanceof RPError)) throw err; skipped.push({ path: e.path, reason: err.message }); continue; }
|
||||
if (made.chunks) { const ws = createWriteStream(dest); await pipeline(Readable.from(made.chunks), ws); }
|
||||
else await fs.writeFile(dest, made.data);
|
||||
const bytes = made.data?.length ?? e.size ?? 0;
|
||||
written += bytes;
|
||||
files.push({ path: e.path, file: dest, size: bytes, ...(made.note ? { note: made.note } : {}) });
|
||||
continue;
|
||||
}
|
||||
if (arc.format === 'zip') {
|
||||
if (![0, 8].includes(e.method)) { skipped.push({ path: e.path, reason: `compression method ${e.method} not supported` }); continue; }
|
||||
const start = await zipDataStart(arc.abs, e);
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
// Decoders Node does not ship: LZ4 block format and raw LZMA (LZMA1, as used by Unity asset bundles, ZWS Flash
|
||||
// files and .lzma "alone" files). Hand-written to keep ReversePort free of runtime dependencies. Both decode into
|
||||
// a buffer of a known size and fail with BAD_FORMAT on corrupt input instead of reading or writing out of bounds.
|
||||
import { RPError } from './util.mjs';
|
||||
|
||||
const corrupt = what => new RPError('BAD_FORMAT', `Corrupt ${what} data.`);
|
||||
|
||||
/** Decode one LZ4 block (no frame header) into exactly `outSize` bytes. */
|
||||
export function lz4Block(src, outSize) {
|
||||
const out = Buffer.alloc(outSize);
|
||||
let s = 0, o = 0;
|
||||
while (s < src.length) {
|
||||
const token = src[s++];
|
||||
let lit = token >> 4;
|
||||
if (lit === 15) { let b; do { if (s >= src.length) throw corrupt('LZ4'); b = src[s++]; lit += b; } while (b === 255); }
|
||||
if (s + lit > src.length || o + lit > outSize) throw corrupt('LZ4');
|
||||
src.copy(out, o, s, s + lit); s += lit; o += lit;
|
||||
if (s >= src.length) break; // the last sequence has literals only
|
||||
if (s + 2 > src.length) throw corrupt('LZ4');
|
||||
const off = src[s] | (src[s + 1] << 8); s += 2;
|
||||
if (off === 0 || off > o) throw corrupt('LZ4');
|
||||
let len = token & 15;
|
||||
if (len === 15) { let b; do { if (s >= src.length) throw corrupt('LZ4'); b = src[s++]; len += b; } while (b === 255); }
|
||||
len += 4;
|
||||
if (o + len > outSize) throw corrupt('LZ4');
|
||||
if (off >= len) out.copy(out, o, o - off, o - off + len);
|
||||
else for (let i = 0; i < len; i++) out[o + i] = out[o - off + i];
|
||||
o += len;
|
||||
}
|
||||
if (o !== outSize) throw corrupt('LZ4');
|
||||
return out;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- LZMA
|
||||
const PROB_INIT = 1024, STATES = 12, POS_STATES_MAX = 16, END_POS_MODEL = 14, FULL_DISTANCES = 128, ALIGN_BITS = 4;
|
||||
|
||||
class RangeDecoder {
|
||||
constructor(buf, pos) {
|
||||
this.buf = buf; this.pos = pos;
|
||||
if (buf[this.pos++] !== 0) throw corrupt('LZMA');
|
||||
this.range = 0xffffffff; this.code = 0;
|
||||
for (let i = 0; i < 4; i++) this.code = ((this.code << 8) | this.byte()) >>> 0;
|
||||
if (this.code === this.range) throw corrupt('LZMA');
|
||||
}
|
||||
byte() { if (this.pos >= this.buf.length) throw corrupt('LZMA'); return this.buf[this.pos++]; }
|
||||
normalize() { if (this.range < 0x1000000) { this.range = (this.range << 8) >>> 0; this.code = ((this.code << 8) | this.byte()) >>> 0; } }
|
||||
bit(probs, i) {
|
||||
const p = probs[i], bound = (this.range >>> 11) * p;
|
||||
let b;
|
||||
if (this.code < bound) { probs[i] = p + ((2048 - p) >> 5); this.range = bound; b = 0; }
|
||||
else { probs[i] = p - (p >> 5); this.code -= bound; this.range -= bound; b = 1; }
|
||||
this.normalize();
|
||||
return b;
|
||||
}
|
||||
direct(n) {
|
||||
let res = 0;
|
||||
for (let i = 0; i < n; i++) {
|
||||
this.range >>>= 1;
|
||||
let t = 0;
|
||||
if (this.code >= this.range) { this.code -= this.range; t = 1; }
|
||||
res = ((res << 1) | t) >>> 0;
|
||||
this.normalize();
|
||||
}
|
||||
return res;
|
||||
}
|
||||
tree(probs, base, bits) { let m = 1; for (let i = 0; i < bits; i++) m = (m << 1) + this.bit(probs, base + m); return m - (1 << bits); }
|
||||
reverse(probs, base, bits) { let m = 1, sym = 0; for (let i = 0; i < bits; i++) { const b = this.bit(probs, base + m); m = (m << 1) + b; sym |= b << i; } return sym; }
|
||||
}
|
||||
|
||||
const probs = n => new Uint16Array(n).fill(PROB_INIT);
|
||||
|
||||
class LenDecoder {
|
||||
constructor() { this.choice = probs(2); this.low = probs(POS_STATES_MAX << 3); this.mid = probs(POS_STATES_MAX << 3); this.high = probs(256); }
|
||||
decode(rc, posState) {
|
||||
if (rc.bit(this.choice, 0) === 0) return rc.tree(this.low, posState << 3, 3);
|
||||
if (rc.bit(this.choice, 1) === 0) return 8 + rc.tree(this.mid, posState << 3, 3);
|
||||
return 16 + rc.tree(this.high, 0, 8);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode raw LZMA: `props` are the 5 property bytes (lc/lp/pb and dictionary size), `data` the range-coded stream,
|
||||
* `outSize` the uncompressed size (the stream may or may not end with an end marker).
|
||||
*/
|
||||
export function lzmaDecode(props, data, outSize, { endMarker = false } = {}) {
|
||||
let d = props[0];
|
||||
if (d >= 9 * 5 * 5) throw corrupt('LZMA');
|
||||
const lc = d % 9; d = Math.floor(d / 9);
|
||||
const lp = d % 5, pb = Math.floor(d / 5);
|
||||
const out = Buffer.alloc(outSize);
|
||||
const rc = new RangeDecoder(data, 0);
|
||||
const literal = probs(0x300 << (lc + lp));
|
||||
const isMatch = probs(STATES << 4), isRep = probs(STATES), isRepG0 = probs(STATES), isRepG1 = probs(STATES), isRepG2 = probs(STATES), isRep0Long = probs(STATES << 4);
|
||||
const posSlot = probs(4 << 6), posDecoders = probs(1 + FULL_DISTANCES - END_POS_MODEL), align = probs(1 << ALIGN_BITS);
|
||||
const lenDec = new LenDecoder(), repLenDec = new LenDecoder();
|
||||
const pbMask = (1 << pb) - 1, lpMask = (1 << lp) - 1;
|
||||
let state = 0, rep0 = 0, rep1 = 0, rep2 = 0, rep3 = 0, o = 0;
|
||||
const distance = len => {
|
||||
const slot = rc.tree(posSlot, Math.min(len, 3) << 6, 6);
|
||||
if (slot < 4) return slot;
|
||||
const direct = (slot >>> 1) - 1;
|
||||
let dist = ((2 | (slot & 1)) << direct) >>> 0;
|
||||
if (slot < END_POS_MODEL) return (dist + rc.reverse(posDecoders, dist - slot, direct)) >>> 0;
|
||||
dist = (dist + ((rc.direct(direct - ALIGN_BITS) << ALIGN_BITS) >>> 0)) >>> 0;
|
||||
return (dist + rc.reverse(align, 0, ALIGN_BITS)) >>> 0;
|
||||
};
|
||||
while (o < outSize) {
|
||||
const posState = o & pbMask;
|
||||
if (rc.bit(isMatch, (state << 4) + posState) === 0) {
|
||||
const prev = o > 0 ? out[o - 1] : 0;
|
||||
const base = 0x300 * (((o & lpMask) << lc) + (prev >> (8 - lc)));
|
||||
let sym = 1;
|
||||
if (state >= 7) {
|
||||
if (rep0 >= o) throw corrupt('LZMA');
|
||||
let matchByte = out[o - rep0 - 1];
|
||||
do {
|
||||
const matchBit = (matchByte >> 7) & 1; matchByte <<= 1;
|
||||
const b = rc.bit(literal, base + ((1 + matchBit) << 8) + sym);
|
||||
sym = (sym << 1) | b;
|
||||
if (matchBit !== b) break;
|
||||
} while (sym < 0x100);
|
||||
}
|
||||
while (sym < 0x100) sym = (sym << 1) | rc.bit(literal, base + sym);
|
||||
out[o++] = sym - 0x100;
|
||||
state = state < 4 ? 0 : state < 10 ? state - 3 : state - 6;
|
||||
continue;
|
||||
}
|
||||
let len;
|
||||
if (rc.bit(isRep, state) !== 0) {
|
||||
if (o === 0) throw corrupt('LZMA');
|
||||
if (rc.bit(isRepG0, state) === 0) {
|
||||
if (rc.bit(isRep0Long, (state << 4) + posState) === 0) {
|
||||
state = state < 7 ? 9 : 11;
|
||||
out[o] = out[o - rep0 - 1]; o++;
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
let dist;
|
||||
if (rc.bit(isRepG1, state) === 0) dist = rep1;
|
||||
else { if (rc.bit(isRepG2, state) === 0) dist = rep2; else { dist = rep3; rep3 = rep2; } rep2 = rep1; }
|
||||
rep1 = rep0; rep0 = dist;
|
||||
}
|
||||
len = repLenDec.decode(rc, posState);
|
||||
state = state < 7 ? 8 : 11;
|
||||
} else {
|
||||
rep3 = rep2; rep2 = rep1; rep1 = rep0;
|
||||
len = lenDec.decode(rc, posState);
|
||||
state = state < 7 ? 7 : 10;
|
||||
rep0 = distance(len);
|
||||
if (rep0 === 0xffffffff) break; // end marker
|
||||
if (rep0 >= o) throw corrupt('LZMA');
|
||||
}
|
||||
len += 2;
|
||||
if (o + len > outSize) throw corrupt('LZMA');
|
||||
for (let i = 0; i < len; i++, o++) out[o] = out[o - rep0 - 1];
|
||||
}
|
||||
if (o !== outSize) { if (endMarker) return out.subarray(0, o); throw corrupt('LZMA'); }
|
||||
return out;
|
||||
}
|
||||
|
||||
/** A .lzma ("LZMA alone") file: 5 property bytes, a 64-bit uncompressed size (all ones: unknown), then the stream. */
|
||||
export function lzmaAlone(buf, maxSize) {
|
||||
if (buf.length < 13) throw corrupt('LZMA');
|
||||
const size = buf.readBigUInt64LE(5);
|
||||
if (size === 0xffffffffffffffffn) return lzmaDecode(buf.subarray(0, 5), buf.subarray(13), maxSize, { endMarker: true }); // size unknown: ends with a marker
|
||||
if (size > BigInt(maxSize)) throw new RPError('TOO_LARGE', `The LZMA stream unpacks to ${size} bytes, over the ${maxSize}-byte limit.`);
|
||||
return lzmaDecode(buf.subarray(0, 5), buf.subarray(13), Number(size));
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import fs from 'node:fs/promises';
|
||||
import zlib from 'node:zlib';
|
||||
import path from 'node:path';
|
||||
import { RPError, readAt } from './util.mjs';
|
||||
import { lzmaDecode } from './decompress.mjs';
|
||||
|
||||
// ------------------------------------------------------------------------------------------------ Ren'Py RPA
|
||||
/** Minimal Python pickle reader for the shapes Ren'Py's index uses (dict of str → list of tuples of int/bytes). */
|
||||
@@ -215,8 +216,23 @@ export async function readFsb5(abs, base, maxLen) {
|
||||
for (let i = 0; i < samples.length; i++) samples[i].size = (i + 1 < samples.length ? samples[i + 1].dataOffset : dataSize) - samples[i].dataOffset;
|
||||
if (names) { let q = 0; for (let i = 0; i < samples.length && q + 4 <= names.length; i++) { const off = names.readUInt32LE(4 * i); const e = names.indexOf(0, off); samples[i].name = names.toString('utf8', off, e < 0 ? names.length : e); q += 4; } }
|
||||
const codec = FSB_CODECS[mode] ?? `codec ${mode}`;
|
||||
const entries = samples.map(s => ({ path: `samples/${s.name ?? `sample_${s.index}`}.${codec === 'VORBIS' ? 'fsbvorbis' : codec === 'MPEG' ? 'mp3' : codec.startsWith('PCM') ? 'pcm' : codec.toLowerCase()}`, offset: dataBase + s.dataOffset, size: s.size, frequency: s.frequency, channels: s.channels, samples: s.samples, codec }));
|
||||
return { fsbVersion: version, codec, samples: samples.length, entries, totalDataSize: dataSize, note: codec === 'VORBIS' ? 'FSB5 Vorbis packets have no Ogg headers (FMOD keeps the setup tables keyed by CRC); extracted .fsbvorbis needs a tool with FMOD\'s header table (e.g. vgmstream) to become .ogg.' : codec.startsWith('PCM') ? 'raw PCM: wrap in a WAV header with the listed frequency/channels.' : `${codec} data as stored.` };
|
||||
const bits = { PCM8: 8, PCM16: 16, PCM24: 24, PCM32: 32, PCMFLOAT: 32 }[codec];
|
||||
const entries = samples.map(s => {
|
||||
const e = { path: `samples/${s.name ?? `sample_${s.index}`}.${codec === 'VORBIS' ? 'fsbvorbis' : codec === 'MPEG' ? 'mp3' : bits ? 'wav' : codec.toLowerCase()}`, offset: dataBase + s.dataOffset, size: s.size, frequency: s.frequency, channels: s.channels, samples: s.samples, codec };
|
||||
if (bits && s.size > 0) { e.prefix = wavHeader(s.size, s.channels, s.frequency, bits, codec === 'PCMFLOAT'); e.size += e.prefix.length; }
|
||||
return e;
|
||||
});
|
||||
return { fsbVersion: version, codec, samples: samples.length, entries, totalDataSize: dataSize, note: codec === 'VORBIS' ? 'FSB5 Vorbis packets have no Ogg headers (FMOD keeps the setup tables keyed by CRC); extracted .fsbvorbis needs a tool with FMOD\'s header table (e.g. vgmstream) to become .ogg.' : bits ? 'PCM samples extract as playable .wav files.' : `${codec} data as stored.` };
|
||||
}
|
||||
|
||||
/** A 44-byte RIFF/WAVE header for `dataBytes` of interleaved PCM. */
|
||||
export function wavHeader(dataBytes, channels, rate, bits, float = false) {
|
||||
const h = Buffer.alloc(44);
|
||||
h.write('RIFF', 0, 'latin1'); h.writeUInt32LE(36 + dataBytes, 4); h.write('WAVEfmt ', 8, 'latin1');
|
||||
h.writeUInt32LE(16, 16); h.writeUInt16LE(float ? 3 : 1, 20); h.writeUInt16LE(channels, 22); h.writeUInt32LE(rate, 24);
|
||||
h.writeUInt32LE(rate * channels * bits / 8, 28); h.writeUInt16LE(channels * bits / 8, 32); h.writeUInt16LE(bits, 34);
|
||||
h.write('data', 36, 'latin1'); h.writeUInt32LE(dataBytes, 40);
|
||||
return h;
|
||||
}
|
||||
|
||||
export async function readFmodBank(abs, size) {
|
||||
@@ -389,6 +405,12 @@ function abcStrings(abc) {
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The body of a ZWS (LZMA) SWF: "ZWS", version, uncompressed length, compressed length, 5 LZMA property bytes, data. */
|
||||
export function swfLzmaBody(whole, fileLength) {
|
||||
if (whole.length < 17 || fileLength < 8 || fileLength > 512 * 1024 * 1024) throw new RPError('BAD_FORMAT', 'Bad ZWS (LZMA) SWF header.');
|
||||
return lzmaDecode(whole.subarray(12, 17), whole.subarray(17), fileLength - 8, { endMarker: true });
|
||||
}
|
||||
|
||||
export async function readSwf(abs, size) {
|
||||
const head = await readAt(abs, 0, 8);
|
||||
const sig = head.toString('latin1', 0, 3);
|
||||
@@ -397,7 +419,7 @@ export async function readSwf(abs, size) {
|
||||
let body;
|
||||
if (sig === 'FWS') body = await readAt(abs, 8, Math.min(size - 8, 256 * 1024 * 1024));
|
||||
else if (sig === 'CWS') { try { body = zlib.inflateSync(await readAt(abs, 8, size - 8)); } catch { throw new RPError('BAD_FORMAT', 'SWF zlib body did not inflate.'); } }
|
||||
else return { format: 'swf', swfVersion: version, compression: 'LZMA', fileLength, entries: [], note: 'ZWS (LZMA-compressed) SWF: Node has no LZMA decoder; decompress with 7-Zip/xz into FWS first (header: FWS + version + length, then the raw body), then run re_archive again.' };
|
||||
else body = swfLzmaBody(await readAt(abs, 0, size), fileLength);
|
||||
// Header: RECT (nbits), frame rate (fixed 8.8), frame count.
|
||||
let p = 0;
|
||||
const nbits = body[0] >> 3;
|
||||
@@ -428,9 +450,9 @@ export async function readSwf(abs, size) {
|
||||
const symbolClasses = symbols.map(s => s.name);
|
||||
const documentClass = symbols.find(s => s.id === 0)?.name ?? null;
|
||||
return {
|
||||
format: 'swf', swfVersion: version, compression: sig === 'CWS' ? 'zlib' : 'none', fileLength, stage: { width: (twips[1] - twips[0]) / 20, height: (twips[3] - twips[2]) / 20, frameRate, frameCount }, tags, entries,
|
||||
format: 'swf', swfVersion: version, compression: sig === 'CWS' ? 'zlib' : sig === 'ZWS' ? 'LZMA' : 'none', fileLength, stage: { width: (twips[1] - twips[0]) / 20, height: (twips[3] - twips[2]) / 20, frameRate, frameCount }, tags, entries,
|
||||
actionScript: as3 ? 'AS3 (DoABC)' : as2 ? 'AS2 (DoAction)' : 'none', documentClass, symbolClasses: symbolClasses.slice(0, 300), frameLabels: labels.slice(0, 300), classes: [...classes].slice(0, 500), identifierStrings: strings.size,
|
||||
note: sig === 'CWS' ? 'Compressed SWF: entries are listed from the inflated body; to extract, save the inflated body as FWS (re_archive action:"extract" writes it) and extract from that.' : 'Images, sounds and ABC blocks extract as-is; shapes, sprites and timelines are vector/timeline data (rebuild from the symbol and class names with re_blueprint; Ruffle can run the original).',
|
||||
note: sig !== 'FWS' ? 'Compressed SWF: entries are listed from the inflated body; to extract, save the inflated body as FWS (re_archive action:"extract" writes it) and extract from that.' : 'Images, sounds and ABC blocks extract as-is; shapes, sprites and timelines are vector/timeline data (rebuild from the symbol and class names with re_blueprint; Ruffle can run the original).',
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
// Unity Mesh (as read through its type tree) → Wavefront OBJ: positions, first UV set, normals and one group per
|
||||
// sub-mesh. Reads the vertex streams of Unity 5 through 6 (three vertex-format tables), external vertex data
|
||||
// (m_StreamData) and compressed meshes (PackedBitVector). X is negated and faces are reversed, because Unity is
|
||||
// left-handed and OBJ is right-handed (the convention AssetStudio and UnityPy use too).
|
||||
|
||||
const half = h => {
|
||||
const s = h & 0x8000 ? -1 : 1, e = (h >> 10) & 0x1f, f = h & 0x3ff;
|
||||
if (e === 0) return s * f * 2 ** -24;
|
||||
if (e === 31) return f ? NaN : s * Infinity;
|
||||
return s * (1 + f / 1024) * 2 ** (e - 15);
|
||||
};
|
||||
|
||||
// [size, read(buf, offset)] per format, for the three tables Unity used over time.
|
||||
const R = {
|
||||
f32: [4, (b, o) => b.readFloatLE(o)], f16: [2, (b, o) => half(b.readUInt16LE(o))],
|
||||
un8: [1, (b, o) => b[o] / 255], sn8: [1, (b, o) => Math.max(b.readInt8(o) / 127, -1)],
|
||||
un16: [2, (b, o) => b.readUInt16LE(o) / 65535], sn16: [2, (b, o) => Math.max(b.readInt16LE(o) / 32767, -1)],
|
||||
u8: [1, (b, o) => b[o]], s8: [1, (b, o) => b.readInt8(o)], u16: [2, (b, o) => b.readUInt16LE(o)], s16: [2, (b, o) => b.readInt16LE(o)],
|
||||
u32: [4, (b, o) => b.readUInt32LE(o)], s32: [4, (b, o) => b.readInt32LE(o)],
|
||||
};
|
||||
const FORMATS_5 = [R.f32, R.f16, R.un8, R.u8, R.u32];
|
||||
const FORMATS_2017 = [R.f32, R.f16, R.un8, R.un8, R.sn8, R.un16, R.sn16, R.u8, R.s8, R.u16, R.s16, R.u32, R.s32];
|
||||
const FORMATS_2019 = [R.f32, R.f16, R.un8, R.sn8, R.un16, R.sn16, R.u8, R.s8, R.u16, R.s16, R.u32, R.s32];
|
||||
|
||||
function unpackInts(pv, start = 0, count = pv.m_NumItems) {
|
||||
const bitSize = pv.m_BitSize, data = pv.m_Data, out = new Array(count);
|
||||
let bitPos = bitSize * start, index = Math.floor(bitPos / 8);
|
||||
bitPos %= 8;
|
||||
for (let i = 0; i < count; i++) {
|
||||
let bits = 0, value = 0;
|
||||
while (bits < bitSize) {
|
||||
if (index >= data.length) throw new RangeError('packed data too short');
|
||||
value += ((data[index] >> bitPos) % 2 ** (bitSize - bits)) * 2 ** bits;
|
||||
const n = Math.min(bitSize - bits, 8 - bitPos);
|
||||
bitPos += n; bits += n;
|
||||
if (bitPos === 8) { index++; bitPos = 0; }
|
||||
}
|
||||
out[i] = value;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
function unpackFloats(pv, start = 0, count = pv.m_NumItems) {
|
||||
if (!pv.m_BitSize) return new Array(count).fill(pv.m_Start);
|
||||
const scale = pv.m_Range / (2 ** pv.m_BitSize - 1);
|
||||
return unpackInts(pv, start, count).map(x => x * scale + pv.m_Start);
|
||||
}
|
||||
const groups = (flat, n) => { const out = []; for (let i = 0; i + n <= flat.length; i += n) out.push(flat.slice(i, i + n)); return out; };
|
||||
|
||||
/**
|
||||
* Vertex attributes and index list of a Mesh value. `external(path, offset, size)` returns the bytes of
|
||||
* m_StreamData (vertex data kept in a .resS). Returns { positions, uvs, normals, indices, use16, subMeshes } or null.
|
||||
*/
|
||||
export async function meshGeometry(m, unity, external) {
|
||||
const major = unity[0];
|
||||
const out = { positions: null, uvs: null, normals: null, indices: null, subMeshes: m.m_SubMeshes ?? [] };
|
||||
// Index buffer width.
|
||||
out.use16 = m.m_Use16BitIndices != null ? !!m.m_Use16BitIndices : m.m_IndexFormat != null ? m.m_IndexFormat === 0 : true;
|
||||
const ib = Buffer.isBuffer(m.m_IndexBuffer) ? m.m_IndexBuffer : Buffer.alloc(0);
|
||||
if (ib.length) { const w = out.use16 ? 2 : 4; out.indices = []; for (let o = 0; o + w <= ib.length; o += w) out.indices.push(w === 2 ? ib.readUInt16LE(o) : ib.readUInt32LE(o)); }
|
||||
// Vertex streams.
|
||||
const vd = m.m_VertexData;
|
||||
let data = Buffer.isBuffer(vd?.m_DataSize) ? vd.m_DataSize : Buffer.alloc(0);
|
||||
if (vd?.m_VertexCount && !data.length && m.m_StreamData?.size && m.m_StreamData?.path) data = await external(m.m_StreamData.path, Number(m.m_StreamData.offset), Number(m.m_StreamData.size));
|
||||
if (vd?.m_VertexCount && data.length && Array.isArray(vd.m_Channels)) {
|
||||
const count = vd.m_VertexCount, channels = vd.m_Channels;
|
||||
const table = major < 2017 ? FORMATS_5 : major < 2019 ? FORMATS_2017 : FORMATS_2019;
|
||||
const fmt = c => table[c.format] ?? null;
|
||||
const streams = [];
|
||||
let offset = 0;
|
||||
const streamCount = 1 + Math.max(...channels.map(c => c.stream));
|
||||
for (let s = 0; s < streamCount; s++) {
|
||||
let stride = 0;
|
||||
for (const c of channels) if (c.stream === s && c.dimension > 0) stride += (c.dimension & 0xf) * (fmt(c)?.[0] ?? 0);
|
||||
streams.push({ offset, stride });
|
||||
offset = Math.ceil((offset + count * stride) / 16) * 16;
|
||||
}
|
||||
const read = (chn, dims) => {
|
||||
const c = channels[chn];
|
||||
if (!c || !(c.dimension & 0xf) || !fmt(c)) return null;
|
||||
const [size, get] = fmt(c), s = streams[c.stream], dim = c.dimension & 0xf, rows = [];
|
||||
for (let v = 0; v < count; v++) {
|
||||
const base = s.offset + v * s.stride + c.offset;
|
||||
if (base + dim * size > data.length) return null;
|
||||
const row = [];
|
||||
for (let k = 0; k < Math.min(dim, dims); k++) row.push(get(data, base + k * size));
|
||||
rows.push(row);
|
||||
}
|
||||
return rows;
|
||||
};
|
||||
out.positions = read(0, 3);
|
||||
out.normals = read(1, 3);
|
||||
out.uvs = read(major >= 2018 ? 4 : 3, 2);
|
||||
}
|
||||
// Compressed mesh (mesh compression on in the import settings).
|
||||
const cm = m.m_CompressedMesh;
|
||||
if (!out.positions && cm?.m_Vertices?.m_NumItems > 0) {
|
||||
const n = Math.floor(cm.m_Vertices.m_NumItems / 3);
|
||||
out.positions = groups(unpackFloats(cm.m_Vertices), 3);
|
||||
if (cm.m_UV?.m_NumItems >= n * 2) {
|
||||
const info = cm.m_UVInfo ?? 0;
|
||||
let start = 0, dim = 2;
|
||||
if (info) { const bits = info & 0xf; if (bits & 4) dim = 1 + (bits & 3); else start = -1; }
|
||||
if (start >= 0) out.uvs = groups(unpackFloats(cm.m_UV, 0, n * dim), dim).map(r => r.slice(0, 2));
|
||||
}
|
||||
if (cm.m_Normals?.m_NumItems > 0) {
|
||||
const xy = groups(unpackFloats(cm.m_Normals), 2), signs = unpackInts(cm.m_NormalSigns);
|
||||
out.normals = xy.map(([x, y], i) => {
|
||||
const zz = 1 - x * x - y * y;
|
||||
let z = zz >= 0 ? Math.sqrt(zz) : 0, nx = x, ny = y;
|
||||
if (zz < 0) { const l = Math.hypot(x, y) || 1; nx /= l; ny /= l; }
|
||||
if (signs[i] === 0) z = -z;
|
||||
return [nx, ny, z];
|
||||
});
|
||||
}
|
||||
if (!out.indices && cm.m_Triangles?.m_NumItems > 0) out.indices = unpackInts(cm.m_Triangles);
|
||||
}
|
||||
return out.positions?.length ? out : null;
|
||||
}
|
||||
|
||||
const num = v => (Number.isFinite(v) ? String(Number(v.toPrecision(9))) : '0');
|
||||
|
||||
/** Mesh value → OBJ text, or null when the mesh has no readable vertices. */
|
||||
export async function meshToObj(m, unity, external) {
|
||||
const g = await meshGeometry(m, unity, external);
|
||||
if (!g) return null;
|
||||
const name = String(m.m_Name ?? 'mesh').replace(/\s+/g, '_') || 'mesh';
|
||||
const lines = [`g ${name}`];
|
||||
for (const [x, y, z = 0] of g.positions) lines.push(`v ${num(-x)} ${num(y)} ${num(z)}`);
|
||||
const vt = g.uvs?.length === g.positions.length, vn = g.normals?.length === g.positions.length;
|
||||
if (vt) for (const [u, v = 0] of g.uvs) lines.push(`vt ${num(u)} ${num(v)}`);
|
||||
if (vn) for (const [x, y, z] of g.normals) lines.push(`vn ${num(-x)} ${num(y)} ${num(z)}`);
|
||||
const ref = i => { const n = i + 1; return vt && vn ? `${n}/${n}/${n}` : vt ? `${n}/${n}` : vn ? `${n}//${n}` : `${n}`; };
|
||||
const idx = g.indices ?? [];
|
||||
const valid = i => i >= 0 && i < g.positions.length;
|
||||
let faces = 0;
|
||||
g.subMeshes.forEach((s, k) => {
|
||||
lines.push(`g ${name}_${k}`);
|
||||
const first = Math.floor(s.firstByte / (g.use16 ? 2 : 4)), count = s.indexCount, base = s.baseVertex ?? 0;
|
||||
const at = i => idx[i] + base;
|
||||
if (s.topology === 0) {
|
||||
for (let i = first; i + 2 < first + count && i + 2 < idx.length; i += 3) {
|
||||
const a = at(i), b = at(i + 1), c = at(i + 2);
|
||||
if ([a, b, c].every(valid)) { lines.push(`f ${ref(c)} ${ref(b)} ${ref(a)}`); faces++; }
|
||||
}
|
||||
} else if (s.topology === 2) {
|
||||
for (let i = first; i + 3 < first + count && i + 3 < idx.length; i += 4) {
|
||||
const [a, b, c, d] = [at(i), at(i + 1), at(i + 2), at(i + 3)];
|
||||
if ([a, b, c, d].every(valid)) { lines.push(`f ${ref(c)} ${ref(b)} ${ref(a)}`, `f ${ref(d)} ${ref(c)} ${ref(a)}`); faces += 2; }
|
||||
}
|
||||
}
|
||||
});
|
||||
return { text: `${lines.join('\n')}\n`, vertices: g.positions.length, faces };
|
||||
}
|
||||
+13
-2
@@ -105,7 +105,11 @@ export async function detectEngines(root, files) {
|
||||
if (ggm[0]) version = (await readAt(ggm[0].abs, 0, 256)).toString('latin1').match(UNITY_VERSION)?.[1] ?? null;
|
||||
const il2cpp = rels.some(r => /GameAssembly\.(dll|so|dylib)$/i.test(r) || /il2cpp_data\//.test(r));
|
||||
const managed = find(/_Data\/Managed\/Assembly-CSharp\.dll$/);
|
||||
add({ id: 'unity-build', name: `Unity game build${version ? ` (${version})` : ''}`, kind: 'build', version, evidence: [...ggm.map(f => f.rel), ...rels.filter(r => /UnityPlayer\./.test(r))].slice(0, 5), details: { scriptingBackend: il2cpp ? 'IL2CPP' : managed.length ? 'Mono' : 'unknown', gameAssemblies: managed.map(f => f.rel) } });
|
||||
// Serialized assets (scenes, shared assets, the compressed data.unity3d) and asset bundles, for re_archive.
|
||||
const assetFiles = find(/_Data\/(data\.unity3d|level\d+|sharedassets\d+\.assets|resources\.assets|globalgamemanagers(\.assets)?)$/).map(f => f.rel)
|
||||
.sort((a, b) => Number(!/data\.unity3d$/.test(a)) - Number(!/data\.unity3d$/.test(b)) || a.localeCompare(b, 'en', { numeric: true }));
|
||||
const bundles = find(/StreamingAssets\/.+\.(bundle|unity3d|ab|assetbundle)$/i).map(f => f.rel);
|
||||
add({ id: 'unity-build', name: `Unity game build${version ? ` (${version})` : ''}`, kind: 'build', version, evidence: [...ggm.map(f => f.rel), ...rels.filter(r => /UnityPlayer\./.test(r))].slice(0, 5), details: { scriptingBackend: il2cpp ? 'IL2CPP' : managed.length ? 'Mono' : 'unknown', gameAssemblies: managed.map(f => f.rel), ...(assetFiles.length ? { assetFiles: assetFiles.slice(0, 12), ...(assetFiles.length > 12 ? { assetFileCount: assetFiles.length } : {}) } : {}), ...(bundles.length ? { assetBundles: bundles.slice(0, 8), assetBundleCount: bundles.length } : {}) } });
|
||||
}
|
||||
// Unreal
|
||||
for (const f of find(/\.uproject$/i)) {
|
||||
@@ -242,7 +246,14 @@ function nextSteps(engines, binaries, root) {
|
||||
const j = v => JSON.stringify(v);
|
||||
if (engines.some(e => e.kind === 'project')) steps.push(`re_graph ${j({ path: root })} for the dependency graph and porting order`, `re_symbols ${j({ path: root, match: '<name>' })} for classes and functions`, `port_plan ${j({ source: root, to: '<unity|godot|godot-csharp|unreal|threejs|language>' })} to start a port`);
|
||||
if (engines.some(e => e.id === 'godot-build')) { const pck = binaries.find(b => /\.pck$/i.test(b.file ?? b.rel ?? b)); steps.push(`re_archive ${j({ file: pck ? `${root}/${pck.file ?? pck.rel ?? pck}` : `${root}/<game>.pck` })} to list the packed project`); }
|
||||
if (engines.some(e => e.id === 'unity-build')) steps.push(`re_binary ${j({ file: `${root}/<Game>_Data/Managed/Assembly-CSharp.dll` })} lists the game classes (Mono builds)`);
|
||||
const unityBuild = engines.find(e => e.id === 'unity-build');
|
||||
if (unityBuild) {
|
||||
const assets = unityBuild.details.assetFiles ?? [];
|
||||
const pick = assets.find(a => /data\.unity3d$/.test(a)) ?? assets.find(a => /globalgamemanagers$/.test(a)) ?? assets[0];
|
||||
if (pick) steps.push(`re_archive ${j({ file: `${root}/${pick}` })} lists the build's objects, scenes and the C# scripts they use; action:"extract" writes textures as PNG, audio, TextAssets and objects`);
|
||||
if (unityBuild.details.assetBundles?.length) steps.push(`re_archive ${j({ file: `${root}/${unityBuild.details.assetBundles[0]}` })} unpacks an asset bundle the same way`);
|
||||
if (unityBuild.details.scriptingBackend !== 'IL2CPP') steps.push(`re_binary ${j({ file: `${root}/<Game>_Data/Managed/Assembly-CSharp.dll` })} lists the game classes (Mono builds)`);
|
||||
}
|
||||
if (binaries.length) { const b = binaries[0]; steps.push(`re_binary ${j({ file: `${root}/${b.file ?? b.rel ?? b}` })} for executables and libraries`); }
|
||||
return steps;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
// Unity texture formats → RGBA pixels → PNG, hand-written (no dependencies). Decodes the formats games ship most:
|
||||
// uncompressed 8-bit layouts, BC1/BC3/BC4/BC5 (DXT, desktop) and ETC1/ETC2/ETC2+EAC alpha (mobile). Formats it
|
||||
// cannot decode (ASTC, BC6H/BC7, crunched, PVRTC) are written as a container another tool opens (.astc, .dds) or as
|
||||
// raw bytes, and the result says which.
|
||||
import zlib from 'node:zlib';
|
||||
|
||||
/** Unity TextureFormat ids → name. */
|
||||
export const TEXTURE_FORMATS = {
|
||||
1: 'Alpha8', 2: 'ARGB4444', 3: 'RGB24', 4: 'RGBA32', 5: 'ARGB32', 6: 'ARGBFloat', 7: 'RGB565', 8: 'BGR24', 9: 'R16', 10: 'DXT1', 11: 'DXT3',
|
||||
12: 'DXT5', 13: 'RGBA4444', 14: 'BGRA32', 15: 'RHalf', 16: 'RGHalf', 17: 'RGBAHalf', 18: 'RFloat', 19: 'RGFloat', 20: 'RGBAFloat', 21: 'YUY2',
|
||||
22: 'RGB9e5Float', 24: 'BC6H', 25: 'BC7', 26: 'BC4', 27: 'BC5', 28: 'DXT1Crunched', 29: 'DXT5Crunched', 30: 'PVRTC_RGB2', 31: 'PVRTC_RGBA2',
|
||||
32: 'PVRTC_RGB4', 33: 'PVRTC_RGBA4', 34: 'ETC_RGB4', 35: 'ATC_RGB4', 36: 'ATC_RGBA8', 41: 'EAC_R', 42: 'EAC_R_SIGNED', 43: 'EAC_RG',
|
||||
44: 'EAC_RG_SIGNED', 45: 'ETC2_RGB', 46: 'ETC2_RGBA1', 47: 'ETC2_RGBA8', 48: 'ASTC_RGB_4x4', 49: 'ASTC_RGB_5x5', 50: 'ASTC_RGB_6x6',
|
||||
51: 'ASTC_RGB_8x8', 52: 'ASTC_RGB_10x10', 53: 'ASTC_RGB_12x12', 54: 'ASTC_RGBA_4x4', 55: 'ASTC_RGBA_5x5', 56: 'ASTC_RGBA_6x6',
|
||||
57: 'ASTC_RGBA_8x8', 58: 'ASTC_RGBA_10x10', 59: 'ASTC_RGBA_12x12', 62: 'RG16', 63: 'R8', 64: 'ETC_RGB4Crunched', 65: 'ETC2_RGBA8Crunched',
|
||||
66: 'ASTC_HDR_4x4', 67: 'ASTC_HDR_5x5', 68: 'ASTC_HDR_6x6', 69: 'ASTC_HDR_8x8', 70: 'ASTC_HDR_10x10', 71: 'ASTC_HDR_12x12', 72: 'RG32',
|
||||
73: 'RGB48', 74: 'RGBA64',
|
||||
};
|
||||
|
||||
const RAW_BPP = { 1: 1, 2: 2, 3: 3, 4: 4, 5: 4, 6: 16, 7: 2, 8: 3, 9: 2, 13: 2, 14: 4, 15: 2, 16: 4, 17: 8, 18: 4, 19: 8, 20: 16, 62: 2, 63: 1, 72: 4, 73: 6, 74: 8 };
|
||||
const BLOCK8 = new Set([10, 26, 34, 41, 42, 45, 46]);
|
||||
const BLOCK16 = new Set([11, 12, 24, 25, 27, 43, 44, 47]);
|
||||
const ASTC_DIM = id => {
|
||||
const n = id >= 66 ? id - 66 : id >= 54 ? id - 54 : id - 48;
|
||||
const d = [4, 5, 6, 8, 10, 12][n];
|
||||
return d;
|
||||
};
|
||||
const isAstc = id => (id >= 48 && id <= 59) || (id >= 66 && id <= 71);
|
||||
|
||||
/** Bytes of one mip level, or null when the format's size rule is unknown here. */
|
||||
export function levelBytes(format, w, h) {
|
||||
if (RAW_BPP[format]) return w * h * RAW_BPP[format];
|
||||
if (BLOCK8.has(format)) return Math.ceil(w / 4) * Math.ceil(h / 4) * 8;
|
||||
if (BLOCK16.has(format)) return Math.ceil(w / 4) * Math.ceil(h / 4) * 16;
|
||||
if (isAstc(format)) { const d = ASTC_DIM(format); return Math.ceil(w / d) * Math.ceil(h / d) * 16; }
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Bytes of a whole mip chain of `mips` levels. */
|
||||
export function chainBytes(format, w, h, mips) {
|
||||
let total = 0;
|
||||
for (let i = 0; i < Math.max(1, mips); i++) {
|
||||
const b = levelBytes(format, Math.max(1, w >> i), Math.max(1, h >> i));
|
||||
if (b == null) return null;
|
||||
total += b;
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- block decoders (write RGBA into a 4x4 tile)
|
||||
const rgb565 = d => [(d >> 8 & 0xf8) | (d >> 13), (d >> 3 & 0xfc) | (d >> 9 & 3), (d << 3 & 0xf8) | (d >> 2 & 7)];
|
||||
|
||||
function bc1(src, o, tile, alpha) {
|
||||
const c0 = src.readUInt16LE(o), c1 = src.readUInt16LE(o + 2);
|
||||
const a = rgb565(c0), b = rgb565(c1);
|
||||
const pal = [[...a, 255], [...b, 255]];
|
||||
if (c0 > c1 || !alpha) {
|
||||
pal.push([0, 1, 2].map(i => Math.floor((a[i] * 2 + b[i]) / 3)).concat(255), [0, 1, 2].map(i => Math.floor((a[i] + b[i] * 2) / 3)).concat(255));
|
||||
} else {
|
||||
pal.push([0, 1, 2].map(i => Math.floor((a[i] + b[i]) / 2)).concat(255), [0, 0, 0, 0]);
|
||||
}
|
||||
const idx = src.readUInt32LE(o + 4);
|
||||
for (let i = 0; i < 16; i++) { const c = pal[(idx >>> (2 * i)) & 3]; tile.set(c, i * 4); }
|
||||
}
|
||||
|
||||
function alphaBlock(src, o) {
|
||||
const a0 = src[o], a1 = src[o + 1];
|
||||
const pal = [a0, a1];
|
||||
if (a0 > a1) for (let i = 1; i < 7; i++) pal.push(Math.floor((a0 * (7 - i) + a1 * i) / 7));
|
||||
else { for (let i = 1; i < 5; i++) pal.push(Math.floor((a0 * (5 - i) + a1 * i) / 5)); pal.push(0, 255); }
|
||||
let bits = 0n;
|
||||
for (let i = 7; i >= 2; i--) bits = (bits << 8n) | BigInt(src[o + i]);
|
||||
const out = new Uint8Array(16);
|
||||
for (let i = 0; i < 16; i++) out[i] = pal[Number((bits >> BigInt(3 * i)) & 7n)];
|
||||
return out;
|
||||
}
|
||||
|
||||
const clamp = v => (v < 0 ? 0 : v > 255 ? 255 : v);
|
||||
const ETC1_MOD = [[2, 8], [5, 17], [9, 29], [13, 42], [18, 60], [24, 80], [33, 106], [47, 183]];
|
||||
const ETC_SUB = [[0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1], [0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1]];
|
||||
const ETC2_DIST = [3, 6, 11, 16, 23, 32, 41, 64];
|
||||
const ORDER = [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15]; // column-major pixel i → row-major tile slot
|
||||
const ORDER_REV = [15, 11, 7, 3, 14, 10, 6, 2, 13, 9, 5, 1, 12, 8, 4, 0];
|
||||
const EAC_MOD = [[-3, -6, -9, -15, 2, 5, 8, 14], [-3, -7, -10, -13, 2, 6, 9, 12], [-2, -5, -8, -13, 1, 4, 7, 12], [-2, -4, -6, -13, 1, 3, 5, 12],
|
||||
[-3, -6, -8, -12, 2, 5, 7, 11], [-3, -7, -9, -11, 2, 6, 8, 10], [-4, -7, -8, -11, 3, 6, 7, 10], [-3, -5, -8, -11, 2, 4, 7, 10],
|
||||
[-2, -6, -8, -10, 1, 5, 7, 9], [-2, -5, -8, -10, 1, 4, 7, 9], [-2, -4, -8, -10, 1, 3, 7, 9], [-2, -5, -7, -10, 1, 4, 6, 9],
|
||||
[-3, -4, -7, -10, 2, 3, 6, 9], [-1, -2, -3, -10, 0, 1, 2, 9], [-4, -6, -8, -9, 3, 5, 7, 8], [-3, -5, -7, -9, 2, 4, 6, 8]];
|
||||
|
||||
/** ETC1 / ETC2 RGB block (8 bytes) into `tile` (16 RGBA pixels, row-major), alpha 255. */
|
||||
function etc2(d, o, tile) {
|
||||
const b = i => d[o + i];
|
||||
let j = (b(6) << 8) | b(7), k = (b(4) << 8) | b(5);
|
||||
const put = (slot, r, g, bl) => { tile[slot * 4] = r; tile[slot * 4 + 1] = g; tile[slot * 4 + 2] = bl; tile[slot * 4 + 3] = 255; };
|
||||
const app = (c, m) => [clamp(c[0] + m), clamp(c[1] + m), clamp(c[2] + m)];
|
||||
const sub = (c, code, flip) => {
|
||||
const table = ETC_SUB[flip];
|
||||
for (let i = 0; i < 16; i++, j >>= 1, k >>= 1) { const s = table[i]; const m = ETC1_MOD[code[s]][j & 1]; put(ORDER[i], ...app(c[s], k & 1 ? -m : m)); }
|
||||
};
|
||||
const code = [b(3) >> 5, (b(3) >> 2) & 7], flip = b(3) & 1;
|
||||
if (!(b(3) & 2)) { // individual
|
||||
const c = [[(b(0) & 0xf0) | (b(0) >> 4), (b(1) & 0xf0) | (b(1) >> 4), (b(2) & 0xf0) | (b(2) >> 4)],
|
||||
[(b(0) & 0x0f) | ((b(0) << 4) & 0xf0), (b(1) & 0x0f) | ((b(1) << 4) & 0xf0), (b(2) & 0x0f) | ((b(2) << 4) & 0xf0)]];
|
||||
return sub(c, code, flip);
|
||||
}
|
||||
const r = b(0) & 0xf8, g = b(1) & 0xf8, bl = b(2) & 0xf8;
|
||||
const delta = x => ((x << 3) & 0x18) - ((x << 3) & 0x20);
|
||||
const dr = delta(b(0)), dg = delta(b(1)), db = delta(b(2));
|
||||
const set4 = colors => { k <<= 1; for (let i = 0; i < 16; i++, j >>= 1, k >>= 1) put(ORDER[i], ...colors[(k & 2) | (j & 1)]); };
|
||||
if (r + dr < 0 || r + dr > 255) { // T mode
|
||||
const c0 = [((b(0) << 3) & 0xc0) | ((b(0) << 4) & 0x30) | ((b(0) >> 1) & 0xc) | (b(0) & 3), (b(1) & 0xf0) | (b(1) >> 4), (b(1) & 0x0f) | ((b(1) << 4) & 0xf0)];
|
||||
const c1 = [(b(2) & 0xf0) | (b(2) >> 4), (b(2) & 0x0f) | ((b(2) << 4) & 0xf0), (b(3) & 0xf0) | (b(3) >> 4)];
|
||||
const dd = ETC2_DIST[((b(3) >> 1) & 6) | (b(3) & 1)];
|
||||
return set4([c0, app(c1, dd), c1, app(c1, -dd)]);
|
||||
}
|
||||
if (g + dg < 0 || g + dg > 255) { // H mode
|
||||
const c0 = [((b(0) << 1) & 0xf0) | ((b(0) >> 3) & 0xf), 0, 0], c1 = [((b(2) << 1) & 0xf0) | ((b(2) >> 3) & 0xf), 0, 0];
|
||||
c0[1] = ((b(0) << 5) & 0xe0) | (b(1) & 0x10); c0[1] |= c0[1] >> 4;
|
||||
c0[2] = (b(1) & 8) | ((b(1) << 1) & 6) | (b(2) >> 7); c0[2] |= (c0[2] << 4) & 0xf0;
|
||||
c1[1] = ((b(2) << 5) & 0xe0) | ((b(3) >> 3) & 0x10); c1[1] |= c1[1] >> 4;
|
||||
c1[2] = ((b(3) << 1) & 0xf0) | ((b(3) >> 3) & 0xf);
|
||||
let di = (b(3) & 4) | ((b(3) << 1) & 2);
|
||||
if (c0[0] > c1[0] || (c0[0] === c1[0] && (c0[1] > c1[1] || (c0[1] === c1[1] && c0[2] >= c1[2])))) di++;
|
||||
const dd = ETC2_DIST[di];
|
||||
return set4([app(c0, dd), app(c0, -dd), app(c1, dd), app(c1, -dd)]);
|
||||
}
|
||||
if (bl + db < 0 || bl + db > 255) { // planar
|
||||
const o0 = [((b(0) << 1) & 0xfc) | ((b(0) >> 5) & 3), ((b(0) << 7) & 0x80) | (b(1) & 0x7e) | (b(0) & 1), ((b(1) << 7) & 0x80) | ((b(2) << 2) & 0x60) | ((b(2) << 3) & 0x18) | ((b(3) >> 5) & 4)];
|
||||
o0[2] |= o0[2] >> 6;
|
||||
const h0 = [((b(3) << 1) & 0xf8) | ((b(3) << 2) & 4) | ((b(3) >> 5) & 3), (b(4) & 0xfe) | (b(4) >> 7), ((b(4) << 7) & 0x80) | ((b(5) >> 1) & 0x7c)];
|
||||
h0[2] |= h0[2] >> 6;
|
||||
const v0 = [((b(5) << 5) & 0xe0) | ((b(6) >> 3) & 0x1c) | ((b(5) >> 1) & 3), ((b(6) << 3) & 0xf8) | ((b(7) >> 5) & 0x6) | ((b(6) >> 4) & 1), ((b(7) << 2) & 0xfc) | ((b(7) >> 4) & 3)];
|
||||
for (let y = 0, i = 0; y < 4; y++) for (let x = 0; x < 4; x++, i++) {
|
||||
const ch = n => clamp((x * (h0[n] - o0[n]) + y * (v0[n] - o0[n]) + 4 * o0[n] + 2) >> 2);
|
||||
put(i, ch(0), ch(1), ch(2));
|
||||
}
|
||||
return;
|
||||
}
|
||||
const c = [[r | (r >> 5), g | (g >> 5), bl | (bl >> 5)], [r + dr, g + dg, bl + db]];
|
||||
c[1] = c[1].map(v => v | (v >> 5));
|
||||
sub(c, code, flip);
|
||||
}
|
||||
|
||||
/** EAC alpha block (8 bytes) into the alpha channel of `tile`. */
|
||||
function eacAlpha(d, o, tile) {
|
||||
const base = d[o], mult = d[o + 1] >> 4;
|
||||
if (!mult) { for (let i = 0; i < 16; i++) tile[i * 4 + 3] = base; return; }
|
||||
const table = EAC_MOD[d[o + 1] & 0xf];
|
||||
let l = d.readBigUInt64BE(o);
|
||||
for (let i = 0; i < 16; i++, l >>= 3n) tile[ORDER_REV[i] * 4 + 3] = clamp(base + mult * table[Number(l & 7n)]);
|
||||
}
|
||||
|
||||
function blocks(data, w, h, bytes, decode) {
|
||||
const out = Buffer.alloc(w * h * 4);
|
||||
const tile = new Uint8Array(64);
|
||||
const bw = Math.ceil(w / 4), bh = Math.ceil(h / 4);
|
||||
for (let by = 0; by < bh; by++) for (let bx = 0; bx < bw; bx++) {
|
||||
decode((by * bw + bx) * bytes, tile);
|
||||
for (let y = 0; y < 4; y++) {
|
||||
const py = by * 4 + y; if (py >= h) break;
|
||||
for (let x = 0; x < 4; x++) {
|
||||
const px = bx * 4 + x; if (px >= w) break;
|
||||
const t = (y * 4 + x) * 4, p = (py * w + px) * 4;
|
||||
out[p] = tile[t]; out[p + 1] = tile[t + 1]; out[p + 2] = tile[t + 2]; out[p + 3] = tile[t + 3];
|
||||
}
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function raw(data, w, h, pick, bpp) {
|
||||
const out = Buffer.alloc(w * h * 4);
|
||||
for (let i = 0; i < w * h; i++) out.set(pick(data, i * bpp), i * 4);
|
||||
return out;
|
||||
}
|
||||
|
||||
const nib = v => (v & 15) * 17;
|
||||
const DECODERS = {
|
||||
1: (d, w, h) => raw(d, w, h, (s, o) => [0, 0, 0, s[o]], 1),
|
||||
2: (d, w, h) => raw(d, w, h, (s, o) => { const v = s.readUInt16LE(o); return [nib(v >> 8), nib(v >> 4), nib(v), nib(v >> 12)]; }, 2),
|
||||
3: (d, w, h) => raw(d, w, h, (s, o) => [s[o], s[o + 1], s[o + 2], 255], 3),
|
||||
4: (d, w, h) => raw(d, w, h, (s, o) => [s[o], s[o + 1], s[o + 2], s[o + 3]], 4),
|
||||
5: (d, w, h) => raw(d, w, h, (s, o) => [s[o + 1], s[o + 2], s[o + 3], s[o]], 4),
|
||||
7: (d, w, h) => raw(d, w, h, (s, o) => [...rgb565(s.readUInt16LE(o)), 255], 2),
|
||||
8: (d, w, h) => raw(d, w, h, (s, o) => [s[o + 2], s[o + 1], s[o], 255], 3),
|
||||
9: (d, w, h) => raw(d, w, h, (s, o) => [s[o + 1], 0, 0, 255], 2),
|
||||
13: (d, w, h) => raw(d, w, h, (s, o) => { const v = s.readUInt16LE(o); return [nib(v >> 12), nib(v >> 8), nib(v >> 4), nib(v)]; }, 2),
|
||||
14: (d, w, h) => raw(d, w, h, (s, o) => [s[o + 2], s[o + 1], s[o], s[o + 3]], 4),
|
||||
62: (d, w, h) => raw(d, w, h, (s, o) => [s[o], s[o + 1], 0, 255], 2),
|
||||
63: (d, w, h) => raw(d, w, h, (s, o) => [s[o], 0, 0, 255], 1),
|
||||
10: (d, w, h) => blocks(d, w, h, 8, (o, t) => bc1(d, o, t, true)),
|
||||
12: (d, w, h) => blocks(d, w, h, 16, (o, t) => { bc1(d, o + 8, t, false); const a = alphaBlock(d, o); for (let i = 0; i < 16; i++) t[i * 4 + 3] = a[i]; }),
|
||||
26: (d, w, h) => blocks(d, w, h, 8, (o, t) => { const v = alphaBlock(d, o); for (let i = 0; i < 16; i++) t.set([v[i], v[i], v[i], 255], i * 4); }),
|
||||
27: (d, w, h) => blocks(d, w, h, 16, (o, t) => { const r = alphaBlock(d, o), g = alphaBlock(d, o + 8); for (let i = 0; i < 16; i++) t.set([r[i], g[i], 0, 255], i * 4); }),
|
||||
34: (d, w, h) => blocks(d, w, h, 8, (o, t) => etc2(d, o, t)),
|
||||
45: (d, w, h) => blocks(d, w, h, 8, (o, t) => etc2(d, o, t)),
|
||||
47: (d, w, h) => blocks(d, w, h, 16, (o, t) => { etc2(d, o + 8, t); eacAlpha(d, o, t); }),
|
||||
};
|
||||
|
||||
export const DECODABLE = new Set(Object.keys(DECODERS).map(Number));
|
||||
|
||||
/** Decode level 0 of `data` into top-down RGBA rows (Unity stores rows bottom-up). Returns null when unsupported. */
|
||||
export function decodeTexture(data, format, w, h) {
|
||||
const dec = DECODERS[format];
|
||||
if (!dec) return null;
|
||||
const need = levelBytes(format, w, h);
|
||||
if (need == null || data.length < need) return null;
|
||||
const rgba = dec(data, w, h);
|
||||
const row = w * 4, flipped = Buffer.alloc(rgba.length);
|
||||
for (let y = 0; y < h; y++) rgba.copy(flipped, (h - 1 - y) * row, y * row, (y + 1) * row);
|
||||
return flipped;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- PNG
|
||||
const CRC = new Int32Array(256).map((_, n) => { let c = n; for (let k = 0; k < 8; k++) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1; return c; });
|
||||
function crc32(buf) { let c = -1; for (let i = 0; i < buf.length; i++) c = CRC[(c ^ buf[i]) & 0xff] ^ (c >>> 8); return (c ^ -1) >>> 0; }
|
||||
function chunk(type, data) {
|
||||
const len = Buffer.alloc(4); len.writeUInt32BE(data.length);
|
||||
const td = Buffer.concat([Buffer.from(type, 'latin1'), data]);
|
||||
const crc = Buffer.alloc(4); crc.writeUInt32BE(crc32(td));
|
||||
return Buffer.concat([len, td, crc]);
|
||||
}
|
||||
|
||||
/** RGBA rows → PNG file bytes. */
|
||||
export function encodePng(rgba, w, h) {
|
||||
const ihdr = Buffer.alloc(13);
|
||||
ihdr.writeUInt32BE(w, 0); ihdr.writeUInt32BE(h, 4); ihdr[8] = 8; ihdr[9] = 6;
|
||||
const raw = Buffer.alloc((w * 4 + 1) * h);
|
||||
for (let y = 0; y < h; y++) rgba.copy(raw, y * (w * 4 + 1) + 1, y * w * 4, (y + 1) * w * 4);
|
||||
return Buffer.concat([Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]), chunk('IHDR', ihdr), chunk('IDAT', zlib.deflateSync(raw)), chunk('IEND', Buffer.alloc(0))]);
|
||||
}
|
||||
|
||||
/**
|
||||
* A texture as a file: PNG when decodable; otherwise a container another tool reads (.astc for ASTC, .dds for
|
||||
* BC6H/BC7) or the raw level-0 bytes. Returns { ext, data, decoded, note? }.
|
||||
*/
|
||||
export function textureFile(data, format, w, h) {
|
||||
const rgba = decodeTexture(data, format, w, h);
|
||||
if (rgba) return { ext: '.png', data: encodePng(rgba, w, h), decoded: true };
|
||||
const name = TEXTURE_FORMATS[format] ?? `format ${format}`;
|
||||
if (isAstc(format)) {
|
||||
const d = ASTC_DIM(format), head = Buffer.alloc(16);
|
||||
head.writeUInt32LE(0x5ca1ab13, 0); head[4] = d; head[5] = d; head[6] = 1;
|
||||
head.writeUIntLE(w, 7, 3); head.writeUIntLE(h, 10, 3); head.writeUIntLE(1, 13, 3);
|
||||
const size = levelBytes(format, w, h);
|
||||
return { ext: '.astc', data: Buffer.concat([head, data.subarray(0, size)]), decoded: false, note: `${name}: written as .astc (decode with astcenc -d, or any ASTC-aware viewer); the image is stored upside down.` };
|
||||
}
|
||||
if (format === 24 || format === 25) {
|
||||
const head = Buffer.alloc(148);
|
||||
head.write('DDS ', 0, 'latin1');
|
||||
head.writeUInt32LE(124, 4); head.writeUInt32LE(0x1 | 0x2 | 0x4 | 0x1000 | 0x80000, 8); head.writeUInt32LE(h, 12); head.writeUInt32LE(w, 16);
|
||||
head.writeUInt32LE(levelBytes(format, w, h), 20); head.writeUInt32LE(1, 28);
|
||||
head.writeUInt32LE(32, 76); head.writeUInt32LE(0x4, 80); head.write('DX10', 84, 'latin1'); head.writeUInt32LE(0x1000, 108);
|
||||
head.writeUInt32LE(format === 25 ? 98 : 95, 128); head.writeUInt32LE(3, 132); head.writeUInt32LE(1, 140);
|
||||
return { ext: '.dds', data: Buffer.concat([head, data.subarray(0, levelBytes(format, w, h))]), decoded: false, note: `${name}: written as .dds (DX10 header); the image is stored upside down.` };
|
||||
}
|
||||
return { ext: '.tex', data, decoded: false, note: `${name}: not decoded here; the raw texture bytes are written (${/Crunched/.test(name) ? 'Unity crunch data needs a crunch decoder' : 'decode with a tool that reads this format'}).` };
|
||||
}
|
||||
+2
-2
@@ -56,8 +56,8 @@ export const tools = [
|
||||
{ name: 're_binary', brief: 'Inspect an executable or bytecode file: format, sections, imports, .NET types, strings, hints.', title: 'Inspect an executable or bytecode',
|
||||
description: 'Inspects PE/EXE/DLL (incl. .NET assemblies: runtime, types and methods), ELF, Mach-O (incl. universal), WebAssembly, Java .class, Android .dex, Lua/LuaJIT and Python bytecode: architecture, sections with entropy, imports/exports, PDB path, signing, overlay data (e.g. an embedded game pack), toolchain/engine hints and strings. Read-only: it does not unpack, decrypt or patch.',
|
||||
inputSchema: obj({ file: str('Binary file.'), strings: obj({ min: int('Minimum string length (default 6).', { minimum: 3, maximum: 64 }), limit: int('Strings to return (default 120).', { minimum: 1, maximum: 5000 }), match: str('Filter strings: substring, glob (*.dll) or /regex/i.'), maxMB: int('Scan this many MiB for strings (default 64).', { minimum: 1, maximum: 256 }), raw: bool('Include strings that look like byte noise (default false).') }, [], 'String options: min, limit, match, maxMB, raw.'), names: obj({ match: str('Filter import/export names: substring, glob or /regex/i.'), limit: int('Import and export names to return each (default 100).', { minimum: 1, maximum: 5000 }) }, [], 'Import/export name options: match, limit.'), maxTypes: int('.NET types to list (default 300).', { minimum: 1, maximum: 5000 }) }, ['file']) },
|
||||
{ name: 're_archive', brief: 'List or extract a pack/archive: Godot .pck (also inside .exe), zip/apk/jar, tar.gz, asar, Ren\'Py .rpa, GameMaker data.win, FMOD/Wwise banks, Unreal .pak, Flash .swf.', title: 'List or extract a pack/archive',
|
||||
description: 'Lists or extracts Godot .pck packs (also packs embedded in an exported .exe/.x86_64), ZIP family (apk, jar, ipa, nupkg, docx), tar and tar.gz (.unitypackage), Electron .asar, Ren\'Py .rpa (2.0/3.0, index XOR key, .rpyc noted as compiled AST), GameMaker data.win (chunks, game name/version, sprite/object/room/script names, textures as PNG/QOI, audio as WAV/OGG), FMOD Studio .bank (event/bus paths, embedded FSB5 samples with codec/rate), Wwise .bnk (DIDX media as .wem, HIRC object counts, names from SoundbanksInfo.xml), Unreal .pak v1–v11 (mount point, entries with compression; stored and Zlib entries extract; Oodle/encrypted listed only) and Flash .swf (stage, tags, AS2/AS3, document and symbol classes, frame labels; JPEG/sound/ABC/binary assets extract; CWS inflates to an FWS copy), Unity WebGL Build/*.data (also .br/.gz/.unityweb; flags IL2CPP metadata) and Emscripten .data packages (file list read from the loader .js next to it). Identifies Unity bundles (header only). Godot entries are MD5-checked on extraction. Encrypted packs/entries are reported, never decrypted. Extraction goes to the ReversePort workspace unless out is given.',
|
||||
{ name: 're_archive', brief: 'List or extract a pack/archive: Godot .pck (also inside .exe), Unity bundles and .assets (textures→PNG, meshes→OBJ, audio, text, scripts in use), zip/apk/jar, tar.gz, asar, Ren\'Py .rpa, GameMaker data.win, FMOD/Wwise banks, Unreal .pak, Flash .swf.', title: 'List or extract a pack/archive',
|
||||
description: 'Lists or extracts game packs and archives. Godot .pck (also embedded in an exported .exe/.x86_64; MD5-checked). Unity asset bundles (UnityFS, LZ4/LZMA) and serialized files (data.unity3d, level0, sharedassets0.assets, globalgamemanagers): every object by class and name, bundle container paths, the C# scripts MonoBehaviours use (with counts), the build\'s scene list; extract writes Texture2D as PNG (DXT/BC, ETC/ETC2, raw formats; ASTC as .astc, BC7 as .dds), Mesh as OBJ, AudioClip as its FSB5 bank, TextAsset as text and other objects as JSON (bundles) or raw bytes (player builds strip type trees). ZIP family (apk, jar, ipa), tar/tar.gz (.unitypackage), Electron .asar, Ren\'Py .rpa, GameMaker data.win (PNG/QOI, WAV/OGG), FMOD .bank and .fsb (PCM as WAV), Wwise .bnk, Unreal .pak v1–v11 (stored/Zlib extract), Flash .swf (CWS and ZWS inflate), Unity WebGL and Emscripten .data. Encrypted packs/entries are reported, never decrypted. Extraction goes to the ReversePort workspace unless out is given.',
|
||||
inputSchema: obj({ file: str('Archive/pack/executable.'), action: str('list (default) or extract.', { enum: ['list', 'extract'] }), match: str('Only entries matching: substring, glob (*.gd, scenes/*.tscn) or /regex/.'), limit: int('Max entries listed/extracted (default 500).', { minimum: 1, maximum: 20000 }), out: str('Extraction folder (default: workspace/extracted/<file>).'), maxBytes: int('Extraction budget in bytes (default 4 GiB).', { minimum: 1 }) }, ['file']) },
|
||||
{ name: 're_probe', brief: 'Identify unknown data: hex, entropy map, embedded files and record strides.', title: 'Probe unknown data',
|
||||
description: 'For unknown or custom formats: identifies the format at an offset, hex view, entropy map (text/code/compressed regions), embedded files (PNG, OGG, WAV, ZIP, GDPC, glTF, DDS, ELF/PE…) with offsets and sizes, and likely record strides for tables. Works on multi-GB files.',
|
||||
|
||||
@@ -0,0 +1,676 @@
|
||||
// Unity asset bundles (UnityFS, LZ4/LZMA blocks) and serialized asset files (.assets, level*, globalgamemanagers,
|
||||
// CAB-* inside bundles), read without Unity and without dependencies. Lists every object by class and name, the
|
||||
// bundle's container paths, the C# scripts the objects use and the scenes in the build; extracts TextAssets,
|
||||
// AudioClip banks (FSB5), textures as PNG, and any object as JSON when the file carries type trees (bundles do),
|
||||
// or as raw bytes when it does not (most player builds). Nothing is decrypted: encrypted bundles are reported.
|
||||
import path from 'node:path';
|
||||
import { RPError, readAt, statOrNull, MiB } from './util.mjs';
|
||||
import { lz4Block, lzmaDecode } from './decompress.mjs';
|
||||
import { TEXTURE_FORMATS, DECODABLE, chainBytes, textureFile } from './texture.mjs';
|
||||
import { meshToObj } from './mesh.mjs';
|
||||
|
||||
/** Unity class ids → class names (the ids Unity writes in YAML as !u!<id> and in binary object tables). */
|
||||
export const CLASS_NAMES = {
|
||||
1: 'GameObject', 2: 'Component', 3: 'LevelGameManager', 4: 'Transform', 5: 'TimeManager', 6: 'GlobalGameManager', 8: 'Behaviour',
|
||||
9: 'GameManager', 11: 'AudioManager', 13: 'InputManager', 18: 'EditorExtension', 19: 'Physics2DSettings', 20: 'Camera', 21: 'Material',
|
||||
23: 'MeshRenderer', 25: 'Renderer', 27: 'Texture', 28: 'Texture2D', 29: 'OcclusionCullingSettings', 30: 'GraphicsSettings', 33: 'MeshFilter',
|
||||
41: 'OcclusionPortal', 43: 'Mesh', 45: 'Skybox', 47: 'QualitySettings', 48: 'Shader', 49: 'TextAsset', 50: 'Rigidbody2D',
|
||||
53: 'Collider2D', 54: 'Rigidbody', 55: 'PhysicsManager', 56: 'Collider', 57: 'Joint', 58: 'CircleCollider2D', 59: 'HingeJoint',
|
||||
60: 'PolygonCollider2D', 61: 'BoxCollider2D', 62: 'PhysicsMaterial2D', 64: 'MeshCollider', 65: 'BoxCollider', 66: 'CompositeCollider2D',
|
||||
68: 'EdgeCollider2D', 70: 'CapsuleCollider2D', 72: 'ComputeShader', 74: 'AnimationClip', 78: 'TagManager', 81: 'AudioListener',
|
||||
82: 'AudioSource', 83: 'AudioClip', 84: 'RenderTexture', 86: 'CustomRenderTexture', 89: 'Cubemap', 90: 'Avatar', 91: 'AnimatorController',
|
||||
93: 'RuntimeAnimatorController', 94: 'ScriptMapper', 95: 'Animator', 96: 'TrailRenderer', 98: 'DelayedCallManager', 102: 'TextMesh',
|
||||
104: 'RenderSettings', 108: 'Light', 109: 'CGProgram', 111: 'Animation', 114: 'MonoBehaviour', 115: 'MonoScript', 116: 'MonoManager',
|
||||
117: 'Texture3D', 119: 'Projector', 120: 'LineRenderer', 121: 'Flare', 123: 'LensFlare', 124: 'FlareLayer', 128: 'Font',
|
||||
129: 'PlayerSettings', 130: 'NamedObject', 134: 'PhysicMaterial', 135: 'SphereCollider', 136: 'CapsuleCollider',
|
||||
137: 'SkinnedMeshRenderer', 138: 'FixedJoint', 141: 'BuildSettings', 142: 'AssetBundle', 143: 'CharacterController',
|
||||
144: 'CharacterJoint', 145: 'SpringJoint', 146: 'WheelCollider', 147: 'ResourceManager', 150: 'PreloadData', 153: 'ConfigurableJoint',
|
||||
154: 'TerrainCollider', 156: 'TerrainData', 157: 'LightmapSettings', 158: 'WebCamTexture', 159: 'EditorSettings', 162: 'EditorUserSettings',
|
||||
164: 'AudioReverbFilter', 165: 'AudioHighPassFilter', 166: 'AudioChorusFilter', 167: 'AudioReverbZone', 168: 'AudioEchoFilter',
|
||||
169: 'AudioLowPassFilter', 170: 'AudioDistortionFilter', 171: 'SparseTexture', 180: 'AudioBehaviour', 181: 'AudioFilter',
|
||||
182: 'WindZone', 183: 'Cloth', 184: 'SubstanceArchive', 185: 'ProceduralMaterial', 186: 'ProceduralTexture', 187: 'Texture2DArray',
|
||||
188: 'CubemapArray', 191: 'OffMeshLink', 192: 'OcclusionArea', 193: 'Tree', 195: 'NavMeshAgent', 196: 'NavMeshSettings',
|
||||
198: 'ParticleSystem', 199: 'ParticleSystemRenderer', 200: 'ShaderVariantCollection', 205: 'LODGroup', 206: 'BlendTree',
|
||||
207: 'Motion', 208: 'NavMeshObstacle', 210: 'SortingGroup', 212: 'SpriteRenderer', 213: 'Sprite', 214: 'CachedSpriteAtlas',
|
||||
215: 'ReflectionProbe', 218: 'Terrain', 220: 'LightProbeGroup', 221: 'AnimatorOverrideController', 222: 'CanvasRenderer',
|
||||
223: 'Canvas', 224: 'RectTransform', 225: 'CanvasGroup', 226: 'BillboardAsset', 227: 'BillboardRenderer', 228: 'SpeedTreeWindAsset',
|
||||
229: 'AnchoredJoint2D', 230: 'Joint2D', 231: 'SpringJoint2D', 232: 'DistanceJoint2D', 233: 'HingeJoint2D', 234: 'SliderJoint2D',
|
||||
235: 'WheelJoint2D', 238: 'NavMeshData', 240: 'AudioMixer', 241: 'AudioMixerController', 243: 'AudioMixerGroupController',
|
||||
244: 'AudioMixerEffectController', 245: 'AudioMixerSnapshotController', 246: 'PhysicsUpdateBehaviour2D', 247: 'ConstantForce2D',
|
||||
248: 'Effector2D', 249: 'AreaEffector2D', 250: 'PointEffector2D', 251: 'PlatformEffector2D', 252: 'SurfaceEffector2D',
|
||||
258: 'LightProbes', 271: 'SampleClip', 272: 'AudioMixerSnapshot', 273: 'AudioMixerGroup', 290: 'AssetBundleManifest',
|
||||
300: 'RuntimeInitializeOnLoadManager', 319: 'AvatarMask', 320: 'PlayableDirector', 328: 'VideoPlayer', 329: 'VideoClip',
|
||||
331: 'SpriteMask', 363: 'OcclusionCullingData', 1001: 'PrefabInstance', 1101: 'AnimatorStateTransition', 1102: 'AnimatorState',
|
||||
1105: 'HumanTemplate', 1107: 'AnimatorStateMachine', 1108: 'PreviewAnimationClip', 1109: 'AnimatorTransition', 1111: 'AnimatorTransitionBase',
|
||||
1113: 'LightmapParameters', 1120: 'LightingDataAsset', 19719996: 'TilemapCollider2D', 156049354: 'Grid', 483693784: 'TilemapRenderer',
|
||||
687078895: 'SpriteAtlas', 1660057539: 'SceneRoots', 1839735485: 'Tilemap',
|
||||
};
|
||||
export const className = id => CLASS_NAMES[id] ?? `Class${id}`;
|
||||
|
||||
// Unity's shared type-tree string table: a node name or type with the top offset bit set is an index into this
|
||||
// null-separated list (the list as Unity writes it; UnityPy and AssetStudio carry the same table).
|
||||
const COMMON = (`AABB AnimationClip AnimationCurve AnimationState Array Base BitField bitset bool char ColorRGBA Component data deque double
|
||||
dynamic_array FastPropertyName first float Font GameObject ${'Generic Mono'} GradientNEW GUID GUIStyle int list ${'long long'} map Matrix4x4f MdFour
|
||||
MonoBehaviour MonoScript m_ByteSize m_Curve m_EditorClassIdentifier m_EditorHideFlags m_Enabled m_ExtensionPtr m_GameObject m_Index m_IsArray
|
||||
m_IsStatic m_MetaFlag m_Name m_ObjectHideFlags m_PrefabInternal m_PrefabParentObject m_Script m_StaticEditorFlags m_Type m_Version Object pair
|
||||
PPtr<Component> PPtr<GameObject> PPtr<Material> PPtr<MonoBehaviour> PPtr<MonoScript> PPtr<Object> PPtr<Prefab> PPtr<Sprite> PPtr<TextAsset>
|
||||
PPtr<Texture> PPtr<Texture2D> PPtr<Transform> Prefab Quaternionf Rectf RectInt RectOffset second set short size SInt16 SInt32 SInt64 SInt8
|
||||
staticvector string TextAsset TextMesh Texture Texture2D Transform TypelessData UInt16 UInt32 UInt64 UInt8 ${'unsigned int'} ${'unsigned long long'}
|
||||
${'unsigned short'} vector Vector2f Vector3f Vector4f m_ScriptingClassIdentifier Gradient Type* int2_storage int3_storage BoundsInt
|
||||
m_CorrespondingSourceObject m_PrefabInstance m_PrefabAsset FileSize Hash128 RenderingLayerMask fixed_array EntityId LoadableObjectId LoadableSceneId`);
|
||||
const COMMON_STRINGS = (() => {
|
||||
// Multi-word entries are protected above by template slots; split on whitespace, then restore them.
|
||||
const multi = ['Generic Mono', 'long long', 'unsigned int', 'unsigned long long', 'unsigned short'];
|
||||
let text = COMMON;
|
||||
multi.forEach((m, i) => { text = text.replace(m, `\u0001${i}`); });
|
||||
const map = new Map();
|
||||
let off = 0;
|
||||
for (const raw of text.split(/\s+/).filter(Boolean)) {
|
||||
const s = raw.startsWith('\u0001') ? multi[Number(raw.slice(1))] : raw;
|
||||
map.set(off, s); off += Buffer.byteLength(s) + 1;
|
||||
}
|
||||
return map;
|
||||
})();
|
||||
|
||||
const PLATFORMS = { 2: 'macOS', 5: 'Windows (32-bit)', 9: 'iOS', 13: 'Android', 19: 'Windows (64-bit)', 20: 'WebGL', 24: 'Linux (64-bit)', 31: 'PS4', 33: 'Xbox One', 38: 'Nintendo Switch' };
|
||||
|
||||
// ---------------------------------------------------------------- binary reader
|
||||
class Bin {
|
||||
constructor(buf, le = true, base = 0) { this.b = buf; this.le = le; this.p = 0; this.base = base; }
|
||||
need(n) { if (n < 0 || this.p + n > this.b.length) throw new RangeError('read past the end'); }
|
||||
get left() { return this.b.length - this.p; }
|
||||
u8() { this.need(1); return this.b[this.p++]; }
|
||||
i8() { this.need(1); return this.b.readInt8(this.p++); }
|
||||
u16() { this.need(2); const v = this.le ? this.b.readUInt16LE(this.p) : this.b.readUInt16BE(this.p); this.p += 2; return v; }
|
||||
i16() { this.need(2); const v = this.le ? this.b.readInt16LE(this.p) : this.b.readInt16BE(this.p); this.p += 2; return v; }
|
||||
u32() { this.need(4); const v = this.le ? this.b.readUInt32LE(this.p) : this.b.readUInt32BE(this.p); this.p += 4; return v; }
|
||||
i32() { this.need(4); const v = this.le ? this.b.readInt32LE(this.p) : this.b.readInt32BE(this.p); this.p += 4; return v; }
|
||||
i64() { this.need(8); const v = this.le ? this.b.readBigInt64LE(this.p) : this.b.readBigInt64BE(this.p); this.p += 8; return v; }
|
||||
u64() { this.need(8); const v = this.le ? this.b.readBigUInt64LE(this.p) : this.b.readBigUInt64BE(this.p); this.p += 8; return v; }
|
||||
f32() { this.need(4); const v = this.le ? this.b.readFloatLE(this.p) : this.b.readFloatBE(this.p); this.p += 4; return v; }
|
||||
f64() { this.need(8); const v = this.le ? this.b.readDoubleLE(this.p) : this.b.readDoubleBE(this.p); this.p += 8; return v; }
|
||||
bytes(n) { this.need(n); const v = this.b.subarray(this.p, this.p + n); this.p += n; return v; }
|
||||
align(n = 4) { const abs = this.base + this.p; this.p = Math.min(this.b.length, this.p + ((n - (abs % n)) % n)); }
|
||||
cstr() { const e = this.b.indexOf(0, this.p); if (e < 0) throw new RangeError('unterminated string'); const s = this.b.toString('utf8', this.p, e); this.p = e + 1; return s; }
|
||||
/** Aligned length-prefixed string as raw bytes (Unity's read_aligned_string). */
|
||||
strBytes() { const n = this.i32(); if (n <= 0) return Buffer.alloc(0); const v = this.bytes(n); this.align(); return v; }
|
||||
str() { return this.strBytes().toString('utf8'); }
|
||||
}
|
||||
|
||||
const utf8 = new TextDecoder('utf-8', { fatal: true });
|
||||
const textOf = buf => { try { const s = utf8.decode(buf); return /\0/.test(s) ? null : s; } catch { return null; } };
|
||||
const looksName = s => s != null && s.length <= 512 && !/[\u0000-\u0008\u000e-\u001f]/.test(s);
|
||||
|
||||
// ---------------------------------------------------------------- byte sources
|
||||
/** A file read through a small cache of 1 MiB chunks (object tables are read in file order). */
|
||||
class FileSource {
|
||||
constructor(abs, size) { this.abs = abs; this.size = size; this.cache = new Map(); }
|
||||
async read(off, len) {
|
||||
if (off < 0 || len < 0 || off + len > this.size) throw new RPError('BAD_FORMAT', `Unity data points past the end of ${path.basename(this.abs)}.`);
|
||||
if (len > MiB) return readAt(this.abs, off, len);
|
||||
const first = Math.floor(off / MiB), last = Math.floor((off + len - 1) / MiB);
|
||||
const parts = [];
|
||||
for (let c = first; c <= last; c++) {
|
||||
let chunk = this.cache.get(c);
|
||||
if (!chunk) {
|
||||
chunk = await readAt(this.abs, c * MiB, MiB);
|
||||
this.cache.set(c, chunk);
|
||||
if (this.cache.size > 16) this.cache.delete(this.cache.keys().next().value);
|
||||
}
|
||||
parts.push(chunk);
|
||||
}
|
||||
const all = parts.length === 1 ? parts[0] : Buffer.concat(parts);
|
||||
const start = off - first * MiB;
|
||||
return all.subarray(start, start + len);
|
||||
}
|
||||
}
|
||||
|
||||
class SliceSource {
|
||||
constructor(parent, base, size) { this.parent = parent; this.base = base; this.size = size; }
|
||||
read(off, len) {
|
||||
if (off < 0 || len < 0 || off + len > this.size) throw new RPError('BAD_FORMAT', 'Unity data points past the end of a bundle entry.');
|
||||
return this.parent.read(this.base + off, len);
|
||||
}
|
||||
}
|
||||
|
||||
const BLOCK_CACHE_BYTES = 256 * MiB, MAX_BLOCK = 1024 * MiB;
|
||||
|
||||
/** The uncompressed data of a UnityFS bundle, decompressed block by block on demand. */
|
||||
class BlockStream {
|
||||
constructor(abs, blocks, dataStart) {
|
||||
this.abs = abs; this.blocks = []; this.cache = new Map(); this.cached = 0;
|
||||
let u = 0, c = dataStart;
|
||||
for (const b of blocks) { this.blocks.push({ ...b, uOff: u, cOff: c }); u += b.uSize; c += b.cSize; }
|
||||
this.size = u;
|
||||
}
|
||||
async block(i) {
|
||||
const hit = this.cache.get(i);
|
||||
if (hit) { this.cache.delete(i); this.cache.set(i, hit); return hit; }
|
||||
const b = this.blocks[i];
|
||||
if (b.uSize > MAX_BLOCK) throw new RPError('TOO_LARGE', `A bundle block unpacks to ${Math.round(b.uSize / MiB)} MiB, over the ${MAX_BLOCK / MiB} MiB limit.`);
|
||||
const raw = await readAt(this.abs, b.cOff, b.cSize);
|
||||
if (raw.length < b.cSize) throw new RPError('BAD_FORMAT', 'The bundle is truncated.');
|
||||
const out = decompress(raw, b.flags & 0x3f, b.uSize, 'bundle block');
|
||||
this.cache.set(i, out); this.cached += out.length;
|
||||
while (this.cached > BLOCK_CACHE_BYTES && this.cache.size > 1) { const [k, v] = this.cache.entries().next().value; this.cache.delete(k); this.cached -= v.length; }
|
||||
return out;
|
||||
}
|
||||
async read(off, len) {
|
||||
if (off < 0 || len < 0 || off + len > this.size) throw new RPError('BAD_FORMAT', 'A bundle entry points past the end of the bundle data.');
|
||||
if (!len) return Buffer.alloc(0);
|
||||
let lo = 0, hi = this.blocks.length - 1;
|
||||
while (lo < hi) { const mid = (lo + hi + 1) >> 1; if (this.blocks[mid].uOff <= off) lo = mid; else hi = mid - 1; }
|
||||
const parts = [];
|
||||
let need = len, pos = off;
|
||||
for (let i = lo; need > 0 && i < this.blocks.length; i++) {
|
||||
const b = this.blocks[i], data = await this.block(i);
|
||||
const s = pos - b.uOff, take = Math.min(need, b.uSize - s);
|
||||
parts.push(data.subarray(s, s + take)); need -= take; pos += take;
|
||||
}
|
||||
return parts.length === 1 ? parts[0] : Buffer.concat(parts);
|
||||
}
|
||||
}
|
||||
|
||||
const COMPRESSION = ['none', 'LZMA', 'LZ4', 'LZ4HC', 'LZHAM'];
|
||||
function decompress(buf, kind, size, what) {
|
||||
if (kind === 0) return buf;
|
||||
if (kind === 2 || kind === 3) return lz4Block(buf, size);
|
||||
if (kind === 1) return lzmaDecode(buf.subarray(0, 5), buf.subarray(5), size);
|
||||
throw new RPError('UNSUPPORTED_ARCHIVE', `${COMPRESSION[kind] ?? `compression ${kind}`} ${what}s are not supported (LZ4, LZ4HC, LZMA and stored are).`);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- UnityFS bundle
|
||||
const versionTuple = s => (String(s).match(/^(\d+)\.(\d+)\.(\d+)/) ?? [0, 0, 0, 0]).slice(1).map(Number);
|
||||
const vcmp = (a, b) => { for (let i = 0; i < 3; i++) if ((a[i] ?? 0) !== (b[i] ?? 0)) return (a[i] ?? 0) - (b[i] ?? 0); return 0; };
|
||||
|
||||
async function openBundle(abs, size) {
|
||||
const head = await readAt(abs, 0, Math.min(size, 4096));
|
||||
const r = new Bin(head, false);
|
||||
const sig = r.cstr();
|
||||
if (sig !== 'UnityFS') {
|
||||
return { format: 'unity-bundle', signature: sig, entries: [], note: `${sig} bundles (Unity 5.2 and older) are identified but not unpacked; only UnityFS bundles are.` };
|
||||
}
|
||||
const fmt = r.u32(), player = r.cstr(), revision = r.cstr();
|
||||
r.i64();
|
||||
const cInfo = r.u32(), uInfo = r.u32(), flags = r.u32();
|
||||
const ver = versionTuple(revision);
|
||||
const oldFlags = vcmp(ver, [2020, 0, 0]) < 0 || (ver[0] === 2020 && vcmp(ver, [2020, 3, 34]) < 0) || (ver[0] === 2021 && vcmp(ver, [2021, 3, 2]) < 0) || (ver[0] === 2022 && vcmp(ver, [2022, 1, 1]) < 0);
|
||||
const base = { format: 'unityfs', bundleFormat: fmt, unityVersion: player === '5.x.x' ? revision : player, unityRevision: revision, compression: COMPRESSION[flags & 0x3f] ?? flags & 0x3f };
|
||||
if (flags & (oldFlags ? 0x200 : 0x1400)) return { ...base, encrypted: true, entries: [], note: 'This bundle uses Unity asset-bundle encryption (as some UnityCN builds do): it is identified, never decrypted.' };
|
||||
if (fmt >= 7 || (ver[0] === 2019 && vcmp(ver, [2019, 4, 15]) >= 0)) r.align(16);
|
||||
let infoRaw, dataStart;
|
||||
if (flags & 0x80) { infoRaw = await readAt(abs, size - cInfo, cInfo); dataStart = r.p; }
|
||||
else { infoRaw = await readAt(abs, r.p, cInfo); dataStart = r.p + cInfo; }
|
||||
if (infoRaw.length < cInfo) throw new RPError('BAD_FORMAT', 'The bundle is truncated (block table).');
|
||||
if (!oldFlags && flags & 0x200) dataStart = Math.ceil(dataStart / 16) * 16;
|
||||
const info = new Bin(decompress(infoRaw, flags & 0x3f, uInfo, 'block table'), false);
|
||||
info.bytes(16);
|
||||
const blockCount = info.i32();
|
||||
if (blockCount < 0 || blockCount * 10 > info.left) throw new RPError('BAD_FORMAT', 'Bad block count in the bundle.');
|
||||
const blocks = [];
|
||||
for (let i = 0; i < blockCount; i++) blocks.push({ uSize: info.u32(), cSize: info.u32(), flags: info.u16() });
|
||||
const nodeCount = info.i32();
|
||||
if (nodeCount < 0 || nodeCount * 21 > info.left) throw new RPError('BAD_FORMAT', 'Bad file count in the bundle.');
|
||||
const nodes = [];
|
||||
for (let i = 0; i < nodeCount; i++) nodes.push({ offset: Number(info.i64()), size: Number(info.i64()), flags: info.u32(), path: info.cstr() });
|
||||
const stream = new BlockStream(abs, blocks, dataStart);
|
||||
for (const n of nodes) if (n.offset < 0 || n.size < 0 || n.offset + n.size > stream.size) throw new RPError('BAD_FORMAT', `Bundle entry ${n.path} lies outside the bundle data.`);
|
||||
const kinds = [...new Set(blocks.map(b => COMPRESSION[b.flags & 0x3f] ?? `compression ${b.flags & 0x3f}`))];
|
||||
return { ...base, blockCompression: kinds, blocks: blocks.length, nodes, stream };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- serialized files
|
||||
/** Does this look like a Unity serialized file of `size` bytes (it has no magic, so check the header's own sums)? */
|
||||
export function looksLikeSerializedFile(head, size) {
|
||||
if (head.length < 48) return false;
|
||||
const version = head.readUInt32BE(8);
|
||||
if (version < 14 || version > 40) return false;
|
||||
let metadataSize = head.readUInt32BE(0), fileSize = head.readUInt32BE(4), dataOffset = head.readUInt32BE(12);
|
||||
if (version >= 22) { metadataSize = head.readUInt32BE(20); fileSize = Number(head.readBigUInt64BE(24)); dataOffset = Number(head.readBigUInt64BE(32)); }
|
||||
return head[16] <= 1 && fileSize === size && dataOffset <= size && metadataSize > 0 && metadataSize < size && !head[17] && !head[18] && !head[19];
|
||||
}
|
||||
|
||||
function parseTypeTree(r, version) {
|
||||
if (version >= 23) {
|
||||
if (r.bytes(4).toString('latin1') !== 'mhtt') throw new RPError('BAD_FORMAT', 'Bad type tree blob.');
|
||||
r.i32();
|
||||
}
|
||||
const count = r.i32(), strSize = r.i32();
|
||||
const size = version >= 19 ? 32 : 24;
|
||||
if (count < 1 || count * size > r.left) throw new RPError('BAD_FORMAT', 'Bad type tree.');
|
||||
const raw = new Bin(r.bytes(count * size), r.le), strings = r.bytes(strSize);
|
||||
const name = v => {
|
||||
if (v & 0x80000000) return COMMON_STRINGS.get(v & 0x7fffffff) ?? String(v & 0x7fffffff);
|
||||
const e = strings.indexOf(0, v);
|
||||
return strings.toString('utf8', v, e < 0 ? strings.length : e);
|
||||
};
|
||||
const nodes = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
raw.u16(); const level = raw.u8(); raw.u8();
|
||||
const type = name(raw.u32()), field = name(raw.u32()), byteSize = raw.i32(); raw.i32(); const meta = raw.i32();
|
||||
if (size === 32) raw.u64();
|
||||
nodes.push({ level, type, name: field, byteSize, meta, children: [] });
|
||||
}
|
||||
const stack = [nodes[0]];
|
||||
for (const n of nodes.slice(1)) {
|
||||
while (stack.length && stack[stack.length - 1].level >= n.level) stack.pop();
|
||||
if (!stack.length) break;
|
||||
stack[stack.length - 1].children.push(n); stack.push(n);
|
||||
}
|
||||
return nodes[0];
|
||||
}
|
||||
|
||||
const ALIGNED = 0x4000;
|
||||
const PRIMS = {
|
||||
SInt8: r => r.i8(), UInt8: r => r.u8(), char: r => r.u8(), short: r => r.i16(), SInt16: r => r.i16(), 'unsigned short': r => r.u16(), UInt16: r => r.u16(),
|
||||
int: r => r.i32(), SInt32: r => r.i32(), 'unsigned int': r => r.u32(), UInt32: r => r.u32(), 'Type*': r => r.u32(),
|
||||
'long long': r => r.i64(), SInt64: r => r.i64(), 'unsigned long long': r => r.u64(), UInt64: r => r.u64(), FileSize: r => r.u64(),
|
||||
float: r => r.f32(), double: r => r.f64(), bool: r => r.u8() !== 0,
|
||||
};
|
||||
const BYTE_TYPES = new Set(['SInt8', 'UInt8', 'char']);
|
||||
|
||||
/** Read one value by its type-tree node. Strings come back as text when valid UTF-8, else as a Buffer. */
|
||||
function readValue(node, r) {
|
||||
let v, align = node.meta & ALIGNED;
|
||||
const prim = PRIMS[node.type];
|
||||
if (prim) v = prim(r);
|
||||
else if (node.type === 'string') { const b = r.strBytes(); v = b.length > 65536 ? b : textOf(b) ?? b; }
|
||||
else if (node.type === 'TypelessData') { const n = r.i32(); v = r.bytes(n); }
|
||||
else if (node.type === 'pair') v = [readValue(node.children[0], r), readValue(node.children[1], r)];
|
||||
else if (node.children[0]?.type === 'Array') {
|
||||
const arr = node.children[0];
|
||||
if (arr.meta & ALIGNED) align = true;
|
||||
const n = r.i32(), item = arr.children[1];
|
||||
if (n < 0 || n > r.left) throw new RangeError('bad array length');
|
||||
if (BYTE_TYPES.has(item.type)) v = r.bytes(n);
|
||||
else { v = new Array(n); for (let i = 0; i < n; i++) v[i] = readValue(item, r); }
|
||||
} else {
|
||||
v = {};
|
||||
for (const c of node.children) v[c.name] = readValue(c, r);
|
||||
}
|
||||
if (align) r.align();
|
||||
return v;
|
||||
}
|
||||
|
||||
/** Read the root's fields in order until every wanted field is found (or all fields are read). */
|
||||
function readFields(root, r, wanted) {
|
||||
const out = {};
|
||||
let left = wanted ? wanted.size : Infinity;
|
||||
for (const c of root.children) {
|
||||
out[c.name] = readValue(c, r);
|
||||
if (wanted?.has(c.name) && --left === 0) break;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async function parseSerialized(src, label) {
|
||||
const h = new Bin(await src.read(0, Math.min(48, src.size)), false);
|
||||
let metadataSize = h.u32(), fileSize = h.u32();
|
||||
const version = h.u32();
|
||||
let dataOffset = h.u32();
|
||||
if (version < 14) throw new RPError('UNSUPPORTED_ARCHIVE', `Serialized file version ${version} (Unity 4 or older) is not supported; Unity 5 and newer are.`);
|
||||
const big = h.u8() === 1; h.bytes(3);
|
||||
if (version >= 22) { metadataSize = h.u32(); fileSize = Number(h.i64()); dataOffset = Number(h.i64()); h.i64(); }
|
||||
const headerSize = h.p;
|
||||
if (metadataSize > src.size - headerSize || dataOffset > src.size) throw new RPError('BAD_FORMAT', `${label}: header sizes do not fit the file.`);
|
||||
const r = new Bin(await src.read(headerSize, metadataSize), !big, headerSize);
|
||||
const unityVersion = r.cstr(), platform = r.i32(), typeTree = r.u8() !== 0;
|
||||
const typeCount = r.i32();
|
||||
if (typeCount < 0 || typeCount > r.left) throw new RPError('BAD_FORMAT', `${label}: bad type count.`);
|
||||
const types = [];
|
||||
for (let i = 0; i < typeCount; i++) {
|
||||
const t = { classId: r.i32() };
|
||||
if (version >= 16) r.u8();
|
||||
if (version >= 17) t.scriptTypeIndex = r.i16();
|
||||
if ((version < 16 && t.classId < 0) || (version >= 16 && t.classId === 114)) r.bytes(16);
|
||||
r.bytes(16);
|
||||
if (typeTree) {
|
||||
let present = true;
|
||||
if (version >= 23) { r.bytes(16); present = r.i32() !== 0; }
|
||||
if (present) t.tree = parseTypeTree(r, version);
|
||||
if (version >= 21) { const n = r.i32(); if (n < 0 || n * 4 > r.left) throw new RPError('BAD_FORMAT', `${label}: bad type dependencies.`); r.bytes(n * 4); }
|
||||
}
|
||||
types.push(t);
|
||||
}
|
||||
const objectCount = r.i32();
|
||||
if (objectCount < 0 || objectCount > r.left) throw new RPError('BAD_FORMAT', `${label}: bad object count.`);
|
||||
const objects = [];
|
||||
for (let i = 0; i < objectCount; i++) {
|
||||
r.align();
|
||||
const pathId = r.i64();
|
||||
const start = (version >= 22 ? Number(r.i64()) : r.u32()) + dataOffset;
|
||||
const size = r.u32(), typeId = r.i32();
|
||||
let classId, type;
|
||||
if (version < 16) { classId = r.u16(); type = types.find(t => t.classId === typeId); }
|
||||
else { type = types[typeId]; classId = type?.classId; }
|
||||
if (version < 17) r.i16();
|
||||
if (version === 15 || version === 16) r.u8();
|
||||
if (classId == null || start + size > src.size) throw new RPError('BAD_FORMAT', `${label}: object ${pathId} is out of range.`);
|
||||
objects.push({ pathId, start, size, classId, tree: type?.tree });
|
||||
}
|
||||
const scriptCount = r.i32();
|
||||
if (scriptCount < 0 || scriptCount * 12 > r.left) throw new RPError('BAD_FORMAT', `${label}: bad script table.`);
|
||||
for (let i = 0; i < scriptCount; i++) { r.i32(); r.align(); r.i64(); }
|
||||
const extCount = r.i32();
|
||||
if (extCount < 0 || extCount > r.left) throw new RPError('BAD_FORMAT', `${label}: bad externals table.`);
|
||||
const externals = [];
|
||||
for (let i = 0; i < extCount; i++) { r.cstr(); r.bytes(16); r.i32(); externals.push(r.cstr()); }
|
||||
return { label, src, version, unityVersion, unityTuple: versionTuple(unityVersion), platform: PLATFORMS[platform] ?? platform, typeTree, le: !big, objects, externals };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- reading objects
|
||||
const PEEK = 64 * 1024;
|
||||
const objBin = (sf, obj, buf) => new Bin(buf, sf.le, obj.start);
|
||||
const pptr = r => ({ fileId: r.i32(), pathId: r.i64() });
|
||||
/** The bytes `inspect` needs: the whole object for script, bundle and build-settings objects, else the first 64 KiB. */
|
||||
const headOf = (sf, o) => sf.src.read(o.start, [115, 141, 142].includes(o.classId) ? Math.min(o.size, 64 * MiB) : Math.min(o.size, PEEK));
|
||||
|
||||
/** The parts of an object the inventory needs, from its first bytes: name, script pointer, script identity, container, scenes. */
|
||||
function inspect(sf, obj, buf) {
|
||||
const out = {};
|
||||
const r = objBin(sf, obj, buf);
|
||||
try {
|
||||
if (obj.tree) {
|
||||
const want = { 114: ['m_Script', 'm_Name'], 115: ['m_Name', 'm_ClassName', 'm_Namespace', 'm_AssemblyName'], 142: ['m_Name', 'm_Container'], 141: ['scenes', 'levels'] }[obj.classId] ?? ['m_Name'];
|
||||
const f = readFields(obj.tree, r, obj.classId === 141 ? new Set(['scenes']) : new Set(want));
|
||||
if (typeof f.m_Name === 'string') out.name = f.m_Name;
|
||||
if (f.m_Script && typeof f.m_Script === 'object' && 'm_PathID' in f.m_Script) out.script = { fileId: f.m_Script.m_FileID, pathId: f.m_Script.m_PathID };
|
||||
if (obj.classId === 115) out.monoScript = { className: f.m_ClassName, namespace: f.m_Namespace, assembly: f.m_AssemblyName };
|
||||
if (obj.classId === 142 && Array.isArray(f.m_Container)) out.container = f.m_Container.map(([p, info]) => ({ path: p, pathId: info?.asset?.m_PathID, fileId: info?.asset?.m_FileID }));
|
||||
if (obj.classId === 141) out.scenes = f.scenes ?? f.levels;
|
||||
if (obj.classId === 28) Object.assign(out, textureHeader(sf, obj, buf));
|
||||
return out;
|
||||
}
|
||||
if (obj.classId === 1) { // GameObject: components, layer, name
|
||||
const n = r.i32();
|
||||
if (n < 0 || n > 100000) return out;
|
||||
r.bytes(n * (vcmp(sf.unityTuple, [5, 5, 0]) >= 0 ? 12 : 16)); r.u32();
|
||||
const s = r.str(); if (looksName(s)) out.name = s;
|
||||
return out;
|
||||
}
|
||||
if (obj.classId === 114) { // MonoBehaviour: m_GameObject, m_Enabled, m_Script, m_Name
|
||||
pptr(r); r.u8(); r.align(); out.script = pptr(r);
|
||||
const s = r.str(); if (looksName(s)) out.name = s;
|
||||
return out;
|
||||
}
|
||||
if (obj.classId === 141) { // BuildSettings: the scene list comes first
|
||||
const n = r.i32(); if (n < 0 || n > 65536) return out;
|
||||
const scenes = []; for (let i = 0; i < n; i++) scenes.push(r.str());
|
||||
if (scenes.every(looksName)) out.scenes = scenes;
|
||||
return out;
|
||||
}
|
||||
const name = r.str();
|
||||
if (!looksName(name)) return out;
|
||||
out.name = name;
|
||||
if (obj.classId === 115) {
|
||||
r.i32(); if (sf.unityTuple[0] >= 5) r.bytes(16); else r.u32();
|
||||
const className = r.str(), namespace = r.str(), assembly = r.str();
|
||||
if ([className, namespace, assembly].every(looksName)) out.monoScript = { className, namespace, assembly };
|
||||
} else if (obj.classId === 142) {
|
||||
const pre = r.i32(); if (pre < 0 || pre * 12 > r.left) return out;
|
||||
r.bytes(pre * 12);
|
||||
const n = r.i32(); if (n < 0 || n > r.left) return out;
|
||||
out.container = [];
|
||||
for (let i = 0; i < n; i++) { const p = r.str(); r.i32(); r.i32(); const a = pptr(r); out.container.push({ path: p, pathId: a.pathId, fileId: a.fileId }); }
|
||||
} else if (obj.classId === 28) Object.assign(out, textureHeader(sf, obj, buf));
|
||||
} catch (e) { if (!(e instanceof RangeError)) throw e; }
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Texture2D size and format: from the type tree, or (no type tree) found by checking the size arithmetic. */
|
||||
function textureHeader(sf, obj, buf) {
|
||||
if (obj.tree) {
|
||||
const f = readFields(obj.tree, objBin(sf, obj, buf), new Set(['m_Width', 'm_Height', 'm_TextureFormat', 'm_MipCount', 'm_CompleteImageSize']));
|
||||
return { width: f.m_Width, height: f.m_Height, format: f.m_TextureFormat, mips: f.m_MipCount, imageSize: f.m_CompleteImageSize };
|
||||
}
|
||||
const r = objBin(sf, obj, buf);
|
||||
r.str();
|
||||
const from = r.p;
|
||||
for (let k = from; k <= from + 32 && k + 24 <= buf.length; k += 4) {
|
||||
const w = buf.readUInt32LE(k), h = buf.readUInt32LE(k + 4), size = buf.readUInt32LE(k + 8);
|
||||
if (!w || !h || w > 16384 || h > 16384 || !size) continue;
|
||||
for (const fo of [12, 16]) {
|
||||
if (k + fo + 8 > buf.length) continue;
|
||||
const format = buf.readInt32LE(k + fo), mips = buf.readInt32LE(k + fo + 4);
|
||||
if (!TEXTURE_FORMATS[format] || mips < 1 || mips > 15) continue;
|
||||
if (chainBytes(format, w, h, mips) === size) return { width: w, height: h, format, mips, imageSize: size };
|
||||
}
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
/** Full Texture2D: header plus where its pixels are (inline or in a .resS resource). */
|
||||
function textureData(sf, obj, buf) {
|
||||
if (obj.tree) {
|
||||
const f = readFields(obj.tree, objBin(sf, obj, buf));
|
||||
const sd = f.m_StreamData ?? {};
|
||||
return { width: f.m_Width, height: f.m_Height, format: f.m_TextureFormat, inline: Buffer.isBuffer(f['image data']) ? f['image data'] : Buffer.alloc(0), stream: sd.path ? { path: sd.path, offset: Number(sd.offset), size: Number(sd.size) } : null };
|
||||
}
|
||||
const head = textureHeader(sf, obj, buf);
|
||||
if (!head.format) return null;
|
||||
// Streamed: the object ends with offset, size and a .resS path. Inline: a length prefix equal to the image size,
|
||||
// with the pixels running to (nearly) the end of the object.
|
||||
const stream = trailingStream(buf, sf.unityTuple[0] >= 2020 ? 8 : 4);
|
||||
if (stream) return { ...head, inline: Buffer.alloc(0), stream };
|
||||
for (let p = buf.length - 4 - head.imageSize; p >= 0; p--) if (p % 4 === 0 && buf.readUInt32LE(p) === head.imageSize) return { ...head, inline: buf.subarray(p + 4, p + 4 + head.imageSize), stream: null };
|
||||
return { ...head, inline: Buffer.alloc(0), stream: null };
|
||||
}
|
||||
|
||||
/** A StreamingInfo at the end of an object: offset (4 or 8 bytes), size, aligned path string. */
|
||||
function trailingStream(buf, offBytes) {
|
||||
for (let q = buf.length - 4; q >= Math.max(0, buf.length - 1024); q -= 4) {
|
||||
const n = buf.readUInt32LE(q);
|
||||
if (n === 0 || q + 4 + n > buf.length || Math.ceil((q + 4 + n) / 4) * 4 !== buf.length) continue;
|
||||
const p = buf.toString('latin1', q + 4, q + 4 + n);
|
||||
if (!/^[\x20-\x7e]+$/.test(p) || !/\.(resS|resource)$|^archive:/.test(p)) continue;
|
||||
const size = buf.readUInt32LE(q - 4), offset = offBytes === 8 ? Number(buf.readBigUInt64LE(q - 12)) : buf.readUInt32LE(q - 8);
|
||||
return { path: p, offset, size };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** AudioClip's m_Resource { m_Source, m_Offset, m_Size }. */
|
||||
function audioResource(sf, obj, buf) {
|
||||
if (obj.tree) {
|
||||
const f = readFields(obj.tree, objBin(sf, obj, buf), new Set(['m_Resource', 'm_CompressionFormat']));
|
||||
const res = f.m_Resource;
|
||||
return res?.m_Source ? { path: res.m_Source, offset: Number(res.m_Offset), size: Number(res.m_Size) } : null;
|
||||
}
|
||||
// No type tree: find the resource path string, then the 8-byte offset and size after its alignment.
|
||||
for (let q = 0; q + 8 <= buf.length; q += 4) {
|
||||
const n = buf.readUInt32LE(q);
|
||||
if (n < 5 || n > 512 || q + 4 + n + 16 > buf.length) continue;
|
||||
const p = buf.toString('latin1', q + 4, q + 4 + n);
|
||||
if (!/^[\x20-\x7e]+$/.test(p) || !/\.(resource|resS)$/.test(p)) continue;
|
||||
const at = q + 4 + Math.ceil(n / 4) * 4;
|
||||
return { path: p, offset: Number(buf.readBigUInt64LE(at)), size: Number(buf.readBigUInt64LE(at + 8)) };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- JSON view of an object
|
||||
function jsonSafe(v, depth = 0) {
|
||||
if (typeof v === 'bigint') return Number.isSafeInteger(Number(v)) ? Number(v) : String(v);
|
||||
if (Buffer.isBuffer(v)) return `<${v.length} bytes>`;
|
||||
if (Array.isArray(v)) {
|
||||
const head = v.slice(0, 256).map(x => jsonSafe(x, depth + 1));
|
||||
return v.length > 256 ? [...head, `… ${v.length - 256} more`] : head;
|
||||
}
|
||||
if (v && typeof v === 'object') { if (depth > 40) return '…'; const o = {}; for (const [k, x] of Object.entries(v)) o[k] = jsonSafe(x, depth + 1); return o; }
|
||||
if (typeof v === 'number' && !Number.isFinite(v)) return String(v);
|
||||
return v;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- inventory and entries
|
||||
const ORDER = [49, 28, 83, 213, 43, 74, 21, 48, 128, 329, 687078895, 91, 115, 142, 141, 1, 114];
|
||||
const rank = id => { const i = ORDER.indexOf(id); return i < 0 ? ORDER.length : i; };
|
||||
const safeName = s => String(s).replace(/[\u0000-\u001f<>:"/\\|?*]/g, '_').replace(/^\.+/, '_').trim().slice(0, 120);
|
||||
const resourceBase = p => String(p).replace(/^archive:\/[^/]*\//, '').split(/[\\/]/).pop();
|
||||
|
||||
/**
|
||||
* Inventory of serialized files: one entry per object, with a producer that writes the useful form of it.
|
||||
* `files`: [{ sf, prefix }], `resource(name)`: async → source of a resource file (.resS/.resource) or null,
|
||||
* `resolveExternal(sf, fileId)`: the parsed file a PPtr's file id points to, or null.
|
||||
*/
|
||||
async function inventory(files, resource, resolveExternal) {
|
||||
const entries = [], classes = {}, used = new Map(), scenes = [], container = new Map();
|
||||
let named = 0;
|
||||
const scriptIndex = new Map(); // `${label}:${pathId}` → script identity
|
||||
for (const { sf } of files) for (const o of sf.objects) {
|
||||
if (o.classId !== 115) continue;
|
||||
const info = inspect(sf, o, await headOf(sf, o));
|
||||
if (info.monoScript) scriptIndex.set(`${sf.label}:${o.pathId}`, info.monoScript);
|
||||
}
|
||||
const lookupScript = async (sf, p) => {
|
||||
if (!p || (p.fileId === 0 && p.pathId === 0n)) return null;
|
||||
const target = p.fileId === 0 ? sf : await resolveExternal(sf, p.fileId);
|
||||
if (!target) return { unresolved: sf.externals[p.fileId - 1] ?? `file ${p.fileId}` };
|
||||
const key = `${target.label}:${p.pathId}`;
|
||||
if (!scriptIndex.has(key)) {
|
||||
const o = target.objects.find(x => x.pathId === p.pathId && x.classId === 115);
|
||||
const info = o ? inspect(target, o, await headOf(target, o)) : {};
|
||||
scriptIndex.set(key, info.monoScript ?? null);
|
||||
}
|
||||
return scriptIndex.get(key);
|
||||
};
|
||||
for (const { sf, prefix } of files) {
|
||||
const seen = new Set();
|
||||
const sorted = [...sf.objects].sort((a, b) => rank(a.classId) - rank(b.classId) || a.start - b.start);
|
||||
for (const o of sorted) {
|
||||
const cls = className(o.classId);
|
||||
classes[cls] = (classes[cls] ?? 0) + 1;
|
||||
const info = inspect(sf, o, await headOf(sf, o));
|
||||
if (info.name) named++;
|
||||
if (info.container) for (const c of info.container) if (c.fileId === 0 && c.pathId != null) container.set(`${sf.label}:${c.pathId}`, c.path);
|
||||
if (info.scenes) scenes.push(...info.scenes.filter(s => typeof s === 'string'));
|
||||
let script = null;
|
||||
if (o.classId === 114 && info.script) {
|
||||
script = await lookupScript(sf, info.script);
|
||||
const key = script?.className ? [script.namespace, script.className].filter(Boolean).join('.') : script?.unresolved ? `? (${script.unresolved})` : null;
|
||||
if (key) { const u = used.get(key) ?? { class: key, assembly: script.assembly, instances: 0 }; u.instances++; used.set(key, u); }
|
||||
}
|
||||
let ext = sf.typeTree ? '.json' : '.bin', kind = 'object';
|
||||
if (o.classId === 49) { kind = 'text'; ext = '.txt'; }
|
||||
else if (o.classId === 83) { kind = 'audio'; ext = '.fsb'; }
|
||||
else if (o.classId === 43 && o.tree) { kind = 'mesh'; ext = '.obj'; }
|
||||
else if (o.classId === 28 && info.format) { kind = 'texture'; ext = DECODABLE.has(info.format) ? '.png' : textureFile(Buffer.alloc(0), info.format, 1, 1).ext; }
|
||||
let base = safeName(info.name || (o.classId === 114 && script?.className) || String(o.pathId)) || String(o.pathId);
|
||||
let rel = `${prefix}${cls}/${base}${ext}`;
|
||||
if (seen.has(rel)) { base = `${base}~${o.pathId}`; rel = `${prefix}${cls}/${base}${ext}`; }
|
||||
seen.add(rel);
|
||||
const entry = { path: rel, size: o.size, pathId: String(o.pathId) };
|
||||
if (o.classId === 28 && info.format) Object.assign(entry, { width: info.width, height: info.height, textureFormat: TEXTURE_FORMATS[info.format] ?? info.format });
|
||||
if (o.classId === 114 && script?.className) entry.script = [script.namespace, script.className].filter(Boolean).join('.');
|
||||
entry.produce = () => produce(sf, o, kind, resource);
|
||||
entry.label = `${sf.label}:${o.pathId}`;
|
||||
entries.push(entry);
|
||||
}
|
||||
}
|
||||
for (const e of entries) { const c = container.get(e.label); if (c) e.asset = c; delete e.label; }
|
||||
const scripts = [...used.values()].sort((a, b) => b.instances - a.instances);
|
||||
return {
|
||||
entries,
|
||||
assets: {
|
||||
objects: entries.length, named,
|
||||
classes: Object.fromEntries(Object.entries(classes).sort((a, b) => b[1] - a[1])),
|
||||
...(scripts.length ? { scripts: scripts.slice(0, 100), scriptsTotal: scripts.length } : {}),
|
||||
...(scenes.length ? { scenes } : {}),
|
||||
...(container.size ? { containerPaths: container.size } : {}),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function produce(sf, o, kind, resource) {
|
||||
const buf = await sf.src.read(o.start, o.size);
|
||||
const fromResource = async (ref, what) => {
|
||||
if (!ref?.path || !ref.size) return null;
|
||||
const res = await resource(resourceBase(ref.path), sf);
|
||||
if (!res) throw new RPError('NOT_FOUND', `${what} data is in ${resourceBase(ref.path)}, which is not next to this file.`);
|
||||
return res.read(ref.offset, ref.size);
|
||||
};
|
||||
if (kind === 'text') {
|
||||
if (o.tree) { const f = readFields(o.tree, objBin(sf, o, buf), new Set(['m_Script'])); const s = f.m_Script; return { data: Buffer.isBuffer(s) ? s : Buffer.from(s ?? '', 'utf8') }; }
|
||||
const r = objBin(sf, o, buf); r.str(); const n = r.i32(); return { data: r.bytes(n) };
|
||||
}
|
||||
if (kind === 'audio') {
|
||||
const data = await fromResource(audioResource(sf, o, buf), 'AudioClip');
|
||||
if (!data) throw new RPError('NOT_FOUND', 'This AudioClip has no streamed resource (Unity 4-style inline audio is not exported).');
|
||||
return { data, note: data.subarray(0, 4).toString('latin1') === 'FSB5' ? undefined : 'not an FSB5 bank' };
|
||||
}
|
||||
if (kind === 'texture') {
|
||||
const t = textureData(sf, o, buf);
|
||||
if (!t?.format) throw new RPError('BAD_FORMAT', 'Texture2D header not understood.');
|
||||
const pixels = t.inline.length ? t.inline : await fromResource(t.stream, 'Texture2D');
|
||||
if (!pixels?.length) throw new RPError('NOT_FOUND', 'This texture has no pixel data in the build.');
|
||||
const file = textureFile(pixels, t.format, t.width, t.height);
|
||||
return { data: file.data, note: file.note };
|
||||
}
|
||||
if (kind === 'mesh') {
|
||||
const fields = readFields(o.tree, objBin(sf, o, buf));
|
||||
const obj = await meshToObj(fields, sf.unityTuple, (p, offset, size) => fromResource({ path: p, offset, size }, 'Mesh'));
|
||||
if (obj) return { data: Buffer.from(obj.text), note: `${obj.vertices} vertices, ${obj.faces} triangles` };
|
||||
return { data: Buffer.from(JSON.stringify(jsonSafe(fields), null, 1)), note: 'no readable vertex data: written as JSON' };
|
||||
}
|
||||
if (o.tree) return { data: Buffer.from(JSON.stringify(jsonSafe(readFields(o.tree, objBin(sf, o, buf))), null, 1)) };
|
||||
return { data: buf };
|
||||
}
|
||||
|
||||
const SUMMARY = sf => ({ file: sf.label, serializedVersion: sf.version, unityVersion: sf.unityVersion, platform: sf.platform, typeTree: sf.typeTree, objects: sf.objects.length, ...(sf.externals.length ? { externals: sf.externals.slice(0, 20) } : {}) });
|
||||
|
||||
const NO_TREE_NOTE = 'This file has no type trees (player builds strip them): names, scripts, TextAssets, audio and most textures still come out; other objects extract as raw bytes instead of JSON.';
|
||||
|
||||
/** A UnityFS bundle: its files, and the objects of the serialized files inside. */
|
||||
export async function readUnityBundle(abs, size) {
|
||||
const b = await openBundle(abs, size);
|
||||
if (!b.stream) return b;
|
||||
const { stream, nodes, ...head } = b;
|
||||
const entries = nodes.map(n => ({ path: `raw/${n.path}`, size: n.size, produce: async () => ({ chunks: (async function* () { for (let o = 0; o < n.size; o += 16 * MiB) yield await stream.read(n.offset + o, Math.min(16 * MiB, n.size - o)); })() }) }));
|
||||
const serialized = [];
|
||||
for (const n of nodes) {
|
||||
const src = new SliceSource(stream, n.offset, n.size);
|
||||
if (!looksLikeSerializedFile(await src.read(0, Math.min(48, n.size)), n.size)) continue;
|
||||
serialized.push({ sf: await parseSerialized(src, n.path), node: n });
|
||||
}
|
||||
const multi = serialized.length > 1;
|
||||
const byName = new Map(nodes.map(n => [n.path.split('/').pop(), n]));
|
||||
const resource = async name => { const n = byName.get(name); return n ? new SliceSource(stream, n.offset, n.size) : null; };
|
||||
const resolveExternal = async (sf, fileId) => { const ext = sf.externals[fileId - 1]; const hit = serialized.find(s => s.node.path === resourceBase(ext)); return hit?.sf ?? null; };
|
||||
const inv = await inventory(serialized.map(({ sf, node }) => ({ sf, prefix: multi ? `${node.path}/` : '' })), resource, resolveExternal);
|
||||
return {
|
||||
...head,
|
||||
entries: [...inv.entries, ...entries],
|
||||
assets: { ...inv.assets, serializedFiles: serialized.map(s => SUMMARY(s.sf)), bundleFiles: nodes.map(n => ({ path: n.path, size: n.size })) },
|
||||
...(serialized.some(s => !s.sf.typeTree) ? { note: NO_TREE_NOTE } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
/** A loose serialized file from a player build (sharedassets0.assets, level0, globalgamemanagers…). */
|
||||
export async function readUnityAssets(abs, size) {
|
||||
const dir = path.dirname(abs);
|
||||
const sf = await parseSerialized(new FileSource(abs, size), path.basename(abs));
|
||||
const opened = new Map([[path.basename(abs), sf]]);
|
||||
const sibling = async name => {
|
||||
if (!name || name !== path.basename(name)) return null;
|
||||
const file = path.join(dir, name);
|
||||
const st = await statOrNull(file);
|
||||
return st?.isFile() ? { file, size: st.size } : null;
|
||||
};
|
||||
const resource = async name => { const s = await sibling(name); return s ? new FileSource(s.file, s.size) : null; };
|
||||
const resolveExternal = async (from, fileId) => {
|
||||
const name = resourceBase(from.externals[fileId - 1] ?? '');
|
||||
if (opened.has(name)) return opened.get(name);
|
||||
opened.set(name, null);
|
||||
const s = await sibling(name);
|
||||
if (!s || opened.size > 16) return null;
|
||||
try {
|
||||
const src = new FileSource(s.file, s.size);
|
||||
if (!looksLikeSerializedFile(await src.read(0, Math.min(48, s.size)), s.size)) return null;
|
||||
const other = await parseSerialized(src, name);
|
||||
opened.set(name, other);
|
||||
return other;
|
||||
} catch { return null; }
|
||||
};
|
||||
const inv = await inventory([{ sf, prefix: '' }], resource, resolveExternal);
|
||||
return { format: 'unity-assets', unityVersion: sf.unityVersion, entries: inv.entries, assets: { ...inv.assets, serializedFiles: [SUMMARY(sf)] }, ...(sf.typeTree ? {} : { note: NO_TREE_NOTE }) };
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import zlib from 'node:zlib';
|
||||
import { tempDir } from './helpers.mjs';
|
||||
import { archiveTool, openArchive } from '../src/archive.mjs';
|
||||
import { unpickle } from '../src/gamearchives.mjs';
|
||||
import { lzmaLiteral } from './unity-fixture.mjs';
|
||||
|
||||
const u16 = n => { const b = Buffer.alloc(2); b.writeUInt16LE(n); return b; };
|
||||
const u32 = n => { const b = Buffer.alloc(4); b.writeUInt32LE(n); return b; };
|
||||
@@ -208,9 +209,14 @@ test('FMOD bank: event paths and FSB5 samples; Wwise bank: media, objects and na
|
||||
assert.equal(fmod.format, 'fmod-bank');
|
||||
assert.deepEqual(fmod.events, ['event:/Player/Jump']);
|
||||
assert.equal(fmod.fsb.codec, 'PCM16');
|
||||
assert.equal(fmod.entries[0].path, 'samples/jump.pcm');
|
||||
assert.equal(fmod.entries[0].path, 'samples/jump.wav');
|
||||
assert.equal(fmod.entries[0].frequency, 44100);
|
||||
assert.equal(fmod.entries[0].size, 64);
|
||||
assert.equal(fmod.entries[0].size, 64 + 44, 'the PCM data plus a WAV header');
|
||||
const pcm = await archiveTool({ file: path.join(dir, 'Master.bank'), action: 'extract', out: path.join(dir, 'pcm') });
|
||||
const wav = await fs.readFile(pcm.files.find(f => f.path === 'samples/jump.wav').file);
|
||||
assert.equal(wav.length, 108);
|
||||
assert.equal(wav.toString('latin1', 0, 4) + wav.toString('latin1', 8, 12), 'RIFFWAVE');
|
||||
assert.deepEqual([wav.readUInt16LE(22), wav.readUInt32LE(24), wav.readUInt16LE(34), wav.readUInt32LE(40)], [1, 44100, 16, 64]);
|
||||
const ww = await archiveTool({ file: await buildWwise(dir) });
|
||||
assert.equal(ww.format, 'wwise-bank');
|
||||
assert.equal(ww.bankVersion, 134);
|
||||
@@ -237,7 +243,7 @@ test('Unreal pak v8: footer, index, stored and zlib entries extract; Oodle/encry
|
||||
await fs.rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('SWF: stage, tags, AS3 classes, symbols, assets; compressed SWF inflates on extract', async () => {
|
||||
test('SWF: stage, tags, AS3 classes, symbols, assets; zlib and LZMA SWFs inflate on extract', async () => {
|
||||
const dir = await tempDir();
|
||||
const list = await archiveTool({ file: await buildSwf(dir, false) });
|
||||
assert.equal(list.format, 'swf');
|
||||
@@ -257,6 +263,16 @@ test('SWF: stage, tags, AS3 classes, symbols, assets; compressed SWF inflates on
|
||||
assert.ok(cex.files.some(f => f.path === 'swf-inflated.swf'));
|
||||
const again = await archiveTool({ file: path.join(dir, 'out2', 'swf-inflated.swf') });
|
||||
assert.equal(again.compression, 'none');
|
||||
// ZWS: "ZWS", version, uncompressed length, LZMA data length, 5 property bytes, LZMA data.
|
||||
const body = swfBody();
|
||||
const { props, data } = lzmaLiteral(body);
|
||||
const zws = path.join(dir, 'game-z.swf');
|
||||
await fs.writeFile(zws, Buffer.concat([Buffer.from('ZWS'), Buffer.from([13]), u32(8 + body.length), u32(data.length), props, data]));
|
||||
const z = await archiveTool({ file: zws });
|
||||
assert.deepEqual([z.compression, z.documentClass, z.entries.length], ['LZMA', 'com.game.Main', 3]);
|
||||
const zex = await archiveTool({ file: zws, action: 'extract', out: path.join(dir, 'out3') });
|
||||
const inflated = await fs.readFile(zex.files.find(f => f.path === 'swf-inflated.swf').file);
|
||||
assert.deepEqual(inflated.subarray(8), body);
|
||||
await fs.rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
// Test fixtures for Unity data, built in memory: LZ4 and LZMA streams (literal-only encoders: valid streams the
|
||||
// decoders must read like any other), serialized files with and without type trees, UnityFS bundles and a small
|
||||
// player-build folder. No Unity, no downloads.
|
||||
import fs from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
// ---------------------------------------------------------------- compressors
|
||||
/** One LZ4 block with a single literal run. */
|
||||
export function lz4Literal(buf) {
|
||||
const len = buf.length, head = [Math.min(len, 15) << 4];
|
||||
if (len >= 15) { let n = len - 15; while (n >= 255) { head.push(255); n -= 255; } head.push(n); }
|
||||
return Buffer.concat([Buffer.from(head), buf]);
|
||||
}
|
||||
|
||||
/** Raw LZMA (lc=3 lp=0 pb=2) with literals only: { props, data }. */
|
||||
export function lzmaLiteral(buf) {
|
||||
const probs = new Uint16Array(0x300 * 8 + 16 * 12).fill(1024);
|
||||
const IS_MATCH = 0x300 * 8; // isMatch[state 0 << 4 + posState] after the literal table
|
||||
const out = [];
|
||||
let low = 0, range = 0xffffffff, cache = 0, cacheSize = 1;
|
||||
const shiftLow = () => {
|
||||
if (low < 0xff000000 || low >= 0x100000000) {
|
||||
const carry = low >= 0x100000000 ? 1 : 0;
|
||||
let temp = cache;
|
||||
do { out.push((temp + carry) & 0xff); temp = 0xff; } while (--cacheSize !== 0);
|
||||
cache = Math.floor(low / 0x1000000) & 0xff;
|
||||
}
|
||||
cacheSize++;
|
||||
low = (low % 0x1000000) * 256;
|
||||
};
|
||||
const bit = (i, b) => {
|
||||
const p = probs[i], bound = (range >>> 11) * p;
|
||||
if (b === 0) { range = bound; probs[i] = p + ((2048 - p) >> 5); }
|
||||
else { low += bound; range = (range - bound) >>> 0; probs[i] = p - (p >> 5); }
|
||||
while (range < 0x1000000) { range = (range * 256) >>> 0; shiftLow(); }
|
||||
};
|
||||
let prev = 0;
|
||||
for (let o = 0; o < buf.length; o++) {
|
||||
bit(IS_MATCH + (o & 3), 0);
|
||||
const base = 0x300 * (prev >> 5);
|
||||
let ctx = 1;
|
||||
for (let i = 7; i >= 0; i--) { const b = (buf[o] >> i) & 1; bit(base + ctx, b); ctx = (ctx << 1) | b; }
|
||||
prev = buf[o];
|
||||
}
|
||||
for (let i = 0; i < 5; i++) shiftLow();
|
||||
const props = Buffer.from([93, 0, 0, 1, 0]); // (pb*5+lp)*9+lc, 64 KiB dictionary
|
||||
return { props, data: Buffer.from(out) };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- little writers
|
||||
const le32 = n => { const b = Buffer.alloc(4); b.writeInt32LE(n); return b; };
|
||||
const leU32 = n => { const b = Buffer.alloc(4); b.writeUInt32LE(n); return b; };
|
||||
const le64 = n => { const b = Buffer.alloc(8); b.writeBigInt64LE(BigInt(n)); return b; };
|
||||
const be32 = n => { const b = Buffer.alloc(4); b.writeUInt32BE(n); return b; };
|
||||
const be64 = n => { const b = Buffer.alloc(8); b.writeBigInt64BE(BigInt(n)); return b; };
|
||||
const pad4 = b => Buffer.concat([b, Buffer.alloc((4 - (b.length % 4)) % 4)]);
|
||||
|
||||
/** Object field writers (little-endian, Unity's aligned layout; objects start 8-aligned). */
|
||||
export const W = {
|
||||
i32: le32, u32: leU32, i64: le64, f32: n => { const b = Buffer.alloc(4); b.writeFloatLE(n); return b; },
|
||||
u8: n => Buffer.from([n]),
|
||||
str: s => pad4(Buffer.concat([le32(Buffer.byteLength(s)), Buffer.from(s)])),
|
||||
bytes: b => pad4(Buffer.concat([le32(b.length), b])),
|
||||
pptr: (fileId, pathId) => Buffer.concat([le32(fileId), le64(pathId)]),
|
||||
bool: v => Buffer.from([v ? 1 : 0, 0, 0, 0]),
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------- type trees
|
||||
/**
|
||||
* A type-tree blob from [level, type, name, byteSize, meta] rows. Names listed in `common` use Unity's shared
|
||||
* string table (offset | 0x80000000), the rest go in the local string buffer.
|
||||
*/
|
||||
export function typeTree(rows, version, common = {}) {
|
||||
const strings = [];
|
||||
const local = s => { const at = Buffer.concat(strings).length; strings.push(Buffer.from(`${s}\0`)); return at; };
|
||||
const cache = new Map();
|
||||
const ref = s => { if (s in common) return (0x80000000 | common[s]) >>> 0; if (!cache.has(s)) cache.set(s, local(s)); return cache.get(s); };
|
||||
const nodes = rows.map(([level, type, name, byteSize, meta = 0], i) => {
|
||||
const b = Buffer.alloc(version >= 19 ? 32 : 24);
|
||||
b.writeUInt16LE(1, 0); b[2] = level; b[3] = 0; b.writeUInt32LE(ref(type), 4); b.writeUInt32LE(ref(name), 8);
|
||||
b.writeInt32LE(byteSize, 12); b.writeInt32LE(i, 16); b.writeInt32LE(meta, 20);
|
||||
return b;
|
||||
});
|
||||
const str = Buffer.concat(strings);
|
||||
return Buffer.concat([le32(rows.length), le32(str.length), ...nodes, str]);
|
||||
}
|
||||
const ALIGN = 0x4000;
|
||||
/** Type-tree rows for an aligned string field at `level`. */
|
||||
export const stringRows = (level, name) => [[level, 'string', name, -1, ALIGN], [level + 1, 'Array', 'Array', -1, ALIGN], [level + 2, 'int', 'size', 4], [level + 2, 'char', 'data', 1]];
|
||||
export const pptrRows = (level, name, type) => [[level, `PPtr<${type}>`, name, 12], [level + 1, 'int', 'm_FileID', 4], [level + 1, 'SInt64', 'm_PathID', 8]];
|
||||
|
||||
// ---------------------------------------------------------------- serialized file
|
||||
/**
|
||||
* Unity serialized file. types: [{ classId, tree? }]; objects: [{ pathId, type (index), data }]; externals: names.
|
||||
*/
|
||||
export function serializedFile({ version = 22, unityVersion = '2022.3.10f1', platform = 19, typeTree: withTrees = false, types, objects, externals = [] }) {
|
||||
const headerSize = version >= 22 ? 48 : 20;
|
||||
const meta = [];
|
||||
meta.push(Buffer.from(`${unityVersion}\0`), le32(platform), Buffer.from([withTrees ? 1 : 0]), le32(types.length));
|
||||
for (const t of types) {
|
||||
meta.push(le32(t.classId), Buffer.from([0]), Buffer.alloc(2, 0xff));
|
||||
if (t.classId === 114) meta.push(Buffer.alloc(16, 1));
|
||||
meta.push(Buffer.alloc(16, 2));
|
||||
if (withTrees) {
|
||||
if (version >= 23) meta.push(Buffer.alloc(16, 3), le32(t.tree ? 1 : 0));
|
||||
if (t.tree) meta.push(t.tree);
|
||||
if (version >= 21) meta.push(le32(0));
|
||||
}
|
||||
}
|
||||
const metaLen = () => meta.reduce((n, b) => n + b.length, 0);
|
||||
meta.push(le32(objects.length));
|
||||
// Object data, each 8-aligned after dataOffset (computed once the metadata size is known).
|
||||
const placed = []; let off = 0;
|
||||
for (const o of objects) { placed.push(off); off = Math.ceil((off + o.data.length) / 8) * 8; }
|
||||
const objectRows = [];
|
||||
for (let i = 0; i < objects.length; i++) {
|
||||
const at = headerSize + metaLen() + objectRows.reduce((n, b) => n + b.length, 0);
|
||||
const pad = Buffer.alloc((4 - (at % 4)) % 4);
|
||||
const row = [pad, le64(objects[i].pathId), version >= 22 ? le64(placed[i]) : leU32(placed[i]), leU32(objects[i].data.length), le32(objects[i].type)];
|
||||
objectRows.push(Buffer.concat(row));
|
||||
}
|
||||
meta.push(...objectRows, le32(0), le32(externals.length));
|
||||
for (const e of externals) meta.push(Buffer.from('\0'), Buffer.alloc(16), le32(0), Buffer.from(`${e}\0`));
|
||||
if (version >= 20) meta.push(le32(0));
|
||||
meta.push(Buffer.from('\0'));
|
||||
const metadata = Buffer.concat(meta);
|
||||
const dataOffset = Math.ceil((headerSize + metadata.length) / 16) * 16;
|
||||
const data = Buffer.alloc(off);
|
||||
objects.forEach((o, i) => o.data.copy(data, placed[i]));
|
||||
const fileSize = dataOffset + data.length;
|
||||
const header = version >= 22
|
||||
? Buffer.concat([be32(0), be32(0), be32(version), be32(0), Buffer.from([0, 0, 0, 0]), be32(metadata.length), be64(fileSize), be64(dataOffset), be64(0)])
|
||||
: Buffer.concat([be32(metadata.length), be32(fileSize), be32(version), be32(dataOffset), Buffer.from([0, 0, 0, 0])]);
|
||||
return Buffer.concat([header, metadata, Buffer.alloc(dataOffset - headerSize - metadata.length), data]);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- UnityFS bundle
|
||||
/**
|
||||
* UnityFS bundle with the given files ([{ path, data, flags }]) in one block. compression: 'lz4' | 'lzma' | 'none';
|
||||
* infoAtEnd puts the block table at the end of the file.
|
||||
*/
|
||||
export function unityBundle(files, { compression = 'lz4', format = 7, revision = '2022.3.10f1', infoAtEnd = false, encrypted = false } = {}) {
|
||||
const payload = Buffer.concat(files.map(f => f.data));
|
||||
let block, kind;
|
||||
if (compression === 'lz4') { block = lz4Literal(payload); kind = 2; }
|
||||
else if (compression === 'lzma') { const { props, data } = lzmaLiteral(payload); block = Buffer.concat([props, data]); kind = 1; }
|
||||
else { block = payload; kind = 0; }
|
||||
let off = 0;
|
||||
const nodes = files.map(f => { const n = Buffer.concat([be64(off), be64(f.data.length), be32(f.flags ?? 0), Buffer.from(`${f.path}\0`)]); off += f.data.length; return n; });
|
||||
const blockRow = Buffer.alloc(10); blockRow.writeUInt32BE(payload.length, 0); blockRow.writeUInt32BE(block.length, 4); blockRow.writeUInt16BE(kind, 8);
|
||||
const info = Buffer.concat([Buffer.alloc(16), be32(1), blockRow, be32(files.length), ...nodes]);
|
||||
const infoPacked = lz4Literal(info);
|
||||
let flags = 2 | 0x40 | (infoAtEnd ? 0x80 : 0) | (encrypted ? 0x1000 : 0);
|
||||
const fixed = Buffer.concat([Buffer.from('UnityFS\0'), be32(format), Buffer.from('5.x.x\0'), Buffer.from(`${revision}\0`)]);
|
||||
const headLen = fixed.length + 8 + 12;
|
||||
const aligned = format >= 7 ? Math.ceil(headLen / 16) * 16 : headLen;
|
||||
const parts = [fixed];
|
||||
const total = aligned + infoPacked.length + block.length;
|
||||
parts.push(be64(total), be32(infoPacked.length), be32(info.length), be32(flags), Buffer.alloc(aligned - headLen));
|
||||
if (infoAtEnd) parts.push(block, infoPacked); else parts.push(infoPacked, block);
|
||||
return Buffer.concat(parts);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- objects
|
||||
export const obj = {
|
||||
gameObject: (name, components = []) => Buffer.concat([W.i32(components.length), ...components.map(([f, p]) => W.pptr(f, p)), W.u32(0), W.str(name)]),
|
||||
monoBehaviour: (go, script, name = '', extra = Buffer.alloc(0)) => Buffer.concat([W.pptr(0, go), W.bool(true), W.pptr(...script), W.str(name), extra]),
|
||||
monoScript: (name, className, namespace, assembly) => Buffer.concat([W.str(name), W.i32(0), Buffer.alloc(16, 9), W.str(className), W.str(namespace), W.str(assembly)]),
|
||||
textAsset: (name, content) => Buffer.concat([W.str(name), W.bytes(Buffer.from(content))]),
|
||||
assetBundle: (name, container) => Buffer.concat([W.str(name), W.i32(0), W.i32(container.length), ...container.map(([p, pathId]) => Buffer.concat([W.str(p), W.i32(0), W.i32(0), W.pptr(0, pathId)]))]),
|
||||
buildSettings: scenes => Buffer.concat([W.i32(scenes.length), ...scenes.map(W.str)]),
|
||||
/** Texture2D as Unity 2019 lays it out (no type tree): pixels inline or in a .resS. */
|
||||
texture2D: (name, { width, height, format, image = Buffer.alloc(0), stream = null }) => Buffer.concat([
|
||||
W.str(name), W.i32(-1), W.bool(false), W.i32(width), W.i32(height), W.i32(stream ? stream.size : image.length), W.i32(format), W.i32(1),
|
||||
W.bool(false), W.bool(false), W.i32(0), W.i32(1), W.i32(0), W.i32(0), W.i32(1), W.i32(1), W.i32(1), W.f32(0), W.f32(0), W.i32(0), W.i32(0), W.i32(0), W.i32(1),
|
||||
W.bytes(image), stream ? Buffer.concat([W.u32(stream.offset), W.u32(stream.size), W.str(stream.path)]) : Buffer.concat([W.u32(0), W.u32(0), W.str('')]),
|
||||
]),
|
||||
/** AudioClip (Unity 5+) whose data is in a .resource file. */
|
||||
audioClip: (name, res) => Buffer.concat([W.str(name), W.i32(1), W.i32(1), W.i32(44100), W.i32(16), W.f32(1.5), W.bool(false), W.i32(0), W.bool(true), W.str(res.path), W.i64(res.offset), W.i64(res.size), W.i32(2)]),
|
||||
};
|
||||
|
||||
/**
|
||||
* A small Windows player build: Game.exe, UnityPlayer.dll, Game_Data/{globalgamemanagers, globalgamemanagers.assets,
|
||||
* level0, sharedassets0.assets, sharedassets0.assets.resS, sharedassets0.resource}. Returns paths and the payloads.
|
||||
*/
|
||||
export async function playerBuild(dir) {
|
||||
const data = path.join(dir, 'Game_Data');
|
||||
await fs.mkdir(data, { recursive: true });
|
||||
await fs.writeFile(path.join(dir, 'Game.exe'), Buffer.from('MZ'));
|
||||
await fs.writeFile(path.join(dir, 'UnityPlayer.dll'), Buffer.from('MZ'));
|
||||
const v = { version: 17, unityVersion: '2019.4.40f1' };
|
||||
await fs.writeFile(path.join(data, 'globalgamemanagers'), serializedFile({ ...v, types: [{ classId: 141 }], objects: [{ pathId: 1, type: 0, data: obj.buildSettings(['Assets/Scenes/Menu.unity', 'Assets/Scenes/Level1.unity']) }] }));
|
||||
await fs.writeFile(path.join(data, 'globalgamemanagers.assets'), serializedFile({ ...v, types: [{ classId: 115 }], objects: [
|
||||
{ pathId: 10, type: 0, data: obj.monoScript('Player', 'Player', 'Game', 'Assembly-CSharp.dll') },
|
||||
{ pathId: 11, type: 0, data: obj.monoScript('Enemy', 'Enemy', 'Game.AI', 'Assembly-CSharp.dll') },
|
||||
] }));
|
||||
await fs.writeFile(path.join(data, 'level0'), serializedFile({ ...v, types: [{ classId: 1 }, { classId: 114 }, { classId: 4 }], externals: ['globalgamemanagers.assets'], objects: [
|
||||
{ pathId: 1, type: 0, data: obj.gameObject('Hero', [[0, 2], [0, 3]]) },
|
||||
{ pathId: 2, type: 2, data: Buffer.concat([W.pptr(0, 1), Buffer.alloc(40)]) },
|
||||
{ pathId: 3, type: 1, data: obj.monoBehaviour(1, [1, 10], '', W.f32(5)) },
|
||||
{ pathId: 4, type: 0, data: obj.gameObject('Goblin', [[0, 5]]) },
|
||||
{ pathId: 5, type: 1, data: obj.monoBehaviour(4, [1, 11]) },
|
||||
{ pathId: 6, type: 0, data: obj.gameObject('Goblin', [[0, 7]]) },
|
||||
{ pathId: 7, type: 1, data: obj.monoBehaviour(6, [1, 11]) },
|
||||
] }));
|
||||
// 4x2 RGBA32 inline (rows bottom-up) and a 4x4 DXT1 block in the .resS; audio bytes in the .resource.
|
||||
const rgba = Buffer.from([...Array(8)].flatMap((_, i) => (i < 4 ? [255, 0, 0, 255] : [0, 0, 255, 255])));
|
||||
const dxt = Buffer.from([0x00, 0xf8, 0x1f, 0x00, 0, 0, 0, 0]); // color0 red, color1 blue, all pixels index 0
|
||||
const resS = Buffer.concat([Buffer.alloc(16), dxt]);
|
||||
const fsb = Buffer.concat([Buffer.from('FSB5'), Buffer.alloc(60)]);
|
||||
const resource = Buffer.concat([Buffer.alloc(32), fsb]);
|
||||
await fs.writeFile(path.join(data, 'sharedassets0.assets.resS'), resS);
|
||||
await fs.writeFile(path.join(data, 'sharedassets0.resource'), resource);
|
||||
await fs.writeFile(path.join(data, 'sharedassets0.assets'), serializedFile({ ...v, types: [{ classId: 49 }, { classId: 28 }, { classId: 83 }], objects: [
|
||||
{ pathId: 1, type: 0, data: obj.textAsset('dialogue', '{"hello":"wereld"}') },
|
||||
{ pathId: 2, type: 1, data: obj.texture2D('flag', { width: 4, height: 2, format: 4, image: rgba }) },
|
||||
{ pathId: 3, type: 1, data: obj.texture2D('tile', { width: 4, height: 4, format: 10, stream: { offset: 16, size: 8, path: 'sharedassets0.assets.resS' } }) },
|
||||
{ pathId: 4, type: 2, data: obj.audioClip('jump', { path: 'sharedassets0.resource', offset: 32, size: fsb.length }) },
|
||||
] }));
|
||||
return { root: dir, data, rgba, fsb };
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
// Unity bundles and serialized files through re_archive and re_scan, on fixtures built in memory: a player build
|
||||
// without type trees, and UnityFS bundles (LZ4 and LZMA blocks) whose serialized file carries type trees.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import zlib from 'node:zlib';
|
||||
import { archiveTool } from '../src/archive.mjs';
|
||||
import { scanProject } from '../src/scan.mjs';
|
||||
import { lz4Block, lzmaDecode, lzmaAlone } from '../src/decompress.mjs';
|
||||
import { decodeTexture, encodePng, textureFile } from '../src/texture.mjs';
|
||||
import { meshToObj } from '../src/mesh.mjs';
|
||||
import { tempDir } from './helpers.mjs';
|
||||
import { lz4Literal, lzmaLiteral, serializedFile, unityBundle, typeTree, stringRows, pptrRows, playerBuild, obj, W } from './unity-fixture.mjs';
|
||||
|
||||
/** PNG → { width, height, rgba } (8-bit RGBA, filter 0 rows: what encodePng writes). */
|
||||
function readPng(buf) {
|
||||
assert.equal(buf.toString('latin1', 1, 4), 'PNG');
|
||||
let p = 8, width, height; const idat = [];
|
||||
while (p < buf.length) {
|
||||
const len = buf.readUInt32BE(p), type = buf.toString('latin1', p + 4, p + 8), data = buf.subarray(p + 8, p + 8 + len);
|
||||
if (type === 'IHDR') { width = data.readUInt32BE(0); height = data.readUInt32BE(4); }
|
||||
if (type === 'IDAT') idat.push(data);
|
||||
p += 12 + len;
|
||||
}
|
||||
const raw = zlib.inflateSync(Buffer.concat(idat)), rgba = Buffer.alloc(width * height * 4);
|
||||
for (let y = 0; y < height; y++) { assert.equal(raw[y * (width * 4 + 1)], 0); raw.copy(rgba, y * width * 4, y * (width * 4 + 1) + 1, (y + 1) * (width * 4 + 1)); }
|
||||
return { width, height, rgba };
|
||||
}
|
||||
const pixel = (img, x, y) => [...img.rgba.subarray((y * img.width + x) * 4, (y * img.width + x) * 4 + 4)];
|
||||
|
||||
test('LZ4 and LZMA decoders: literal runs, overlapping matches, a real encoder\'s stream, corrupt input', () => {
|
||||
const text = Buffer.from('ReversePort '.repeat(40));
|
||||
assert.deepEqual(lz4Block(lz4Literal(text), text.length), text);
|
||||
// "abc" then a match of 9 at offset 3 (overlapping copy), then the final literal "!".
|
||||
const block = Buffer.from([0x35, 0x61, 0x62, 0x63, 0x03, 0x00, 0x10, 0x21]);
|
||||
assert.equal(lz4Block(block, 13).toString(), 'abcabcabcabc!');
|
||||
assert.throws(() => lz4Block(Buffer.from([0x35, 0x61, 0x62, 0x63, 0x09, 0x00, 0x10, 0x21]), 13), { code: 'BAD_FORMAT' }, 'offset before the start');
|
||||
assert.throws(() => lz4Block(block, 12), { code: 'BAD_FORMAT' }, 'output larger than declared');
|
||||
const { props, data } = lzmaLiteral(text);
|
||||
assert.deepEqual(lzmaDecode(props, data, text.length), text);
|
||||
// xz --format=lzma output (size unknown, end marker, rep and normal matches).
|
||||
const xz = Buffer.from('XQAAgAD//////////wAqGgiiAyVm8Ut4xaIF/y7m2dIgGq00+OId6EE2+twGabs85BA0Jwnrs2bj7TeIqO86YLg17JzgVCRkTp0oFlmYQJr5P1JJZ1NQlrN39rYJIax//Z+MAA==', 'base64');
|
||||
assert.equal(lzmaAlone(xz, 4096).toString(), 'The quick brown fox jumps over the lazy dog. '.repeat(6) + 'ReversePort reads Unity bundles. '.repeat(4));
|
||||
assert.throws(() => lzmaDecode(props, data.subarray(0, data.length >> 1), text.length), { code: 'BAD_FORMAT' });
|
||||
});
|
||||
|
||||
test('textures: DXT1, RGBA32 and ETC2 decode upside-up; ASTC and BC7 become .astc and .dds files', () => {
|
||||
const dxt = Buffer.from([0x00, 0xf8, 0x1f, 0x00, 0x55, 0x55, 0xff, 0xff]); // rows 0-1 index 1 (blue), rows 2-3 index 3
|
||||
const img = decodeTexture(dxt, 10, 4, 4);
|
||||
assert.deepEqual([...img.subarray(0, 4)], [85, 0, 170, 255], 'top row is the last stored row (index 3: one third red, two thirds blue)');
|
||||
assert.deepEqual([...img.subarray(12 * 4, 12 * 4 + 4)], [0, 0, 255, 255], 'bottom row is stored row 0 (blue)');
|
||||
const rgba = Buffer.from([1, 2, 3, 4, 5, 6, 7, 8]);
|
||||
assert.deepEqual([...decodeTexture(rgba, 4, 1, 2)], [5, 6, 7, 8, 1, 2, 3, 4]);
|
||||
assert.equal(decodeTexture(Buffer.alloc(4), 4, 4, 4), null, 'too little data');
|
||||
const png = readPng(encodePng(Buffer.from([255, 0, 0, 255, 0, 255, 0, 128]), 2, 1));
|
||||
assert.deepEqual([png.width, png.height, ...png.rgba], [2, 1, 255, 0, 0, 255, 0, 255, 0, 128]);
|
||||
const astc = textureFile(Buffer.alloc(16 * 4), 54, 8, 8);
|
||||
assert.equal(astc.ext, '.astc');
|
||||
assert.equal(astc.data.readUInt32LE(0), 0x5ca1ab13);
|
||||
assert.deepEqual([astc.data[4], astc.data[5], astc.data.readUIntLE(7, 3), astc.data.readUIntLE(10, 3)], [4, 4, 8, 8]);
|
||||
const dds = textureFile(Buffer.alloc(16), 25, 4, 4);
|
||||
assert.equal(dds.ext, '.dds');
|
||||
assert.equal(dds.data.toString('latin1', 84, 88), 'DX10');
|
||||
assert.equal(dds.data.readUInt32LE(128), 98, 'DXGI_FORMAT_BC7_UNORM');
|
||||
assert.equal(textureFile(Buffer.alloc(8), 28, 4, 4).ext, '.tex', 'crunched: raw');
|
||||
});
|
||||
|
||||
test('player build without type trees: scenes, scripts in use, names; TextAsset, PNG and FSB5 extract', async () => {
|
||||
const dir = await tempDir();
|
||||
const build = await playerBuild(dir);
|
||||
const level = await archiveTool({ file: path.join(build.data, 'level0') });
|
||||
assert.equal(level.format, 'unity-assets');
|
||||
assert.equal(level.unityVersion, '2019.4.40f1');
|
||||
assert.deepEqual(level.assets.classes, { MonoBehaviour: 3, GameObject: 3, Transform: 1 });
|
||||
assert.deepEqual(level.assets.scripts, [{ class: 'Game.AI.Enemy', assembly: 'Assembly-CSharp.dll', instances: 2 }, { class: 'Game.Player', assembly: 'Assembly-CSharp.dll', instances: 1 }]);
|
||||
assert.equal(level.assets.serializedFiles[0].typeTree, false);
|
||||
assert.match(level.note, /no type trees/);
|
||||
const paths = level.entries.map(e => e.path);
|
||||
assert.ok(paths.includes('GameObject/Hero.bin') && paths.includes('GameObject/Goblin.bin') && paths.some(p => /^GameObject\/Goblin~\d+\.bin$/.test(p)), paths.join(', '));
|
||||
assert.ok(level.entries.some(e => e.path === 'MonoBehaviour/Player.bin' && e.script === 'Game.Player'), 'unnamed MonoBehaviours take their script name');
|
||||
|
||||
const ggm = await archiveTool({ file: path.join(build.data, 'globalgamemanagers') });
|
||||
assert.deepEqual(ggm.assets.scenes, ['Assets/Scenes/Menu.unity', 'Assets/Scenes/Level1.unity']);
|
||||
|
||||
const shared = await archiveTool({ file: path.join(build.data, 'sharedassets0.assets') });
|
||||
assert.deepEqual(shared.entries.map(e => e.path), ['TextAsset/dialogue.txt', 'Texture2D/flag.png', 'Texture2D/tile.png', 'AudioClip/jump.fsb']);
|
||||
assert.deepEqual(shared.entries.filter(e => e.width).map(e => [e.width, e.height, e.textureFormat]), [[4, 2, 'RGBA32'], [4, 4, 'DXT1']]);
|
||||
const out = path.join(dir, 'out');
|
||||
const ex = await archiveTool({ file: path.join(build.data, 'sharedassets0.assets'), action: 'extract', out });
|
||||
assert.deepEqual(ex.skipped, []);
|
||||
assert.equal(await fs.readFile(path.join(out, 'TextAsset/dialogue.txt'), 'utf8'), '{"hello":"wereld"}');
|
||||
const flag = readPng(await fs.readFile(path.join(out, 'Texture2D/flag.png')));
|
||||
assert.deepEqual([flag.width, flag.height, pixel(flag, 0, 0), pixel(flag, 3, 1)], [4, 2, [0, 0, 255, 255], [255, 0, 0, 255]], 'stored bottom-up, written top-down');
|
||||
const tile = readPng(await fs.readFile(path.join(out, 'Texture2D/tile.png')));
|
||||
assert.deepEqual(pixel(tile, 2, 2), [255, 0, 0, 255], 'DXT1 from the .resS');
|
||||
assert.deepEqual(await fs.readFile(path.join(out, 'AudioClip/jump.fsb')), build.fsb);
|
||||
|
||||
const scan = await scanProject({ path: dir });
|
||||
const unity = scan.engines.find(e => e.id === 'unity-build');
|
||||
assert.ok(unity.details.assetFiles.includes('Game_Data/level0') && unity.details.assetFiles.includes('Game_Data/sharedassets0.assets'));
|
||||
assert.ok(scan.nextSteps.some(s => s.startsWith('re_archive') && s.includes('globalgamemanagers')), scan.nextSteps.join('\n'));
|
||||
await fs.rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------- a bundle with type trees
|
||||
const COMMON = { m_Name: 427, Base: 55, string: 840, int: 222 };
|
||||
const ALIGN = 0x4000;
|
||||
function treeBundleCab() {
|
||||
const v = 22;
|
||||
const textTree = typeTree([[0, 'TextAsset', 'Base', -1], ...stringRows(1, 'm_Name'), ...stringRows(1, 'm_Script')], v, COMMON);
|
||||
const scriptTree = typeTree([[0, 'MonoScript', 'Base', -1], ...stringRows(1, 'm_Name'), [1, 'int', 'm_ExecutionOrder', 4], [1, 'Hash128', 'm_PropertiesHash', 16], [2, 'UInt32', 'a', 4], [2, 'UInt32', 'b', 4], [2, 'UInt32', 'c', 4], [2, 'UInt32', 'd', 4],
|
||||
...stringRows(1, 'm_ClassName'), ...stringRows(1, 'm_Namespace'), ...stringRows(1, 'm_AssemblyName')], v, COMMON);
|
||||
const behaviourTree = typeTree([[0, 'MonoBehaviour', 'Base', -1], ...pptrRows(1, 'm_GameObject', 'GameObject'), [1, 'UInt8', 'm_Enabled', 1, ALIGN], ...pptrRows(1, 'm_Script', 'MonoScript'), ...stringRows(1, 'm_Name'),
|
||||
[1, 'int', 'speed', 4], [1, 'float', 'jumpHeight', 4], ...stringRows(1, 'title'), [1, 'vector', 'waypoints', -1], [2, 'Array', 'Array', -1, ALIGN], [3, 'int', 'size', 4], [3, 'Vector2f', 'data', 8], [4, 'float', 'x', 4], [4, 'float', 'y', 4]], v, COMMON);
|
||||
const bundleTree = typeTree([[0, 'AssetBundle', 'Base', -1], ...stringRows(1, 'm_Name'), [1, 'vector', 'm_PreloadTable', -1], [2, 'Array', 'Array', -1, ALIGN], [3, 'int', 'size', 4], ...pptrRows(3, 'data', 'Object'),
|
||||
[1, 'map', 'm_Container', -1], [2, 'Array', 'Array', -1, ALIGN], [3, 'int', 'size', 4], [3, 'pair', 'data', -1], ...stringRows(4, 'first'), [4, 'AssetInfo', 'second', -1], [5, 'int', 'preloadIndex', 4], [5, 'int', 'preloadSize', 4], ...pptrRows(5, 'asset', 'Object')], v, COMMON);
|
||||
const texTree = typeTree([[0, 'Texture2D', 'Base', -1], ...stringRows(1, 'm_Name'), [1, 'int', 'm_Width', 4], [1, 'int', 'm_Height', 4], [1, 'int', 'm_CompleteImageSize', 4], [1, 'int', 'm_TextureFormat', 4], [1, 'int', 'm_MipCount', 4],
|
||||
[1, 'TypelessData', 'image data', -1, ALIGN], [2, 'int', 'size', 4], [2, 'UInt8', 'data', 1], [1, 'StreamingInfo', 'm_StreamData', -1], [2, 'UInt64', 'offset', 8], [2, 'unsigned int', 'size', 4], ...stringRows(2, 'path')], v, COMMON);
|
||||
const dxt5 = Buffer.concat([Buffer.from([255, 255, 0, 0, 0, 0, 0, 0]), Buffer.from([0xe0, 0x07, 0xe0, 0x07, 0, 0, 0, 0])]); // opaque green
|
||||
const resS = Buffer.concat([Buffer.alloc(32), dxt5]);
|
||||
const cab = serializedFile({ version: v, unityVersion: '2022.3.10f1', typeTree: true,
|
||||
types: [{ classId: 49, tree: textTree }, { classId: 115, tree: scriptTree }, { classId: 114, tree: behaviourTree }, { classId: 142, tree: bundleTree }, { classId: 28, tree: texTree }],
|
||||
objects: [
|
||||
{ pathId: 100, type: 0, data: obj.textAsset('story', 'Er was eens…') },
|
||||
{ pathId: 200, type: 1, data: Buffer.concat([W.str('Player'), W.i32(0), Buffer.alloc(16), W.str('Player'), W.str('Game'), W.str('Assembly-CSharp.dll')]) },
|
||||
{ pathId: 300, type: 2, data: obj.monoBehaviour(0, [0, 200], 'PlayerConfig', Buffer.concat([W.i32(7), W.f32(2.5), W.str('Held'), W.i32(2), W.f32(1), W.f32(2), W.f32(3), W.f32(4)])) },
|
||||
{ pathId: 1, type: 3, data: obj.assetBundle('characters', [['assets/story.txt', 100], ['assets/config/player.asset', 300]]) },
|
||||
{ pathId: 400, type: 4, data: Buffer.concat([W.str('grass'), W.i32(4), W.i32(4), W.i32(16), W.i32(12), W.i32(1), W.i32(0), W.i64(32), W.u32(16), W.str('archive:/CAB-test/CAB-test.resS')]) },
|
||||
] });
|
||||
return { cab, resS };
|
||||
}
|
||||
|
||||
test('UnityFS bundle with type trees (LZ4): container paths, scripts, JSON objects, PNG from the .resS, raw files', async () => {
|
||||
const dir = await tempDir();
|
||||
const { cab, resS } = treeBundleCab();
|
||||
const file = path.join(dir, 'characters.bundle');
|
||||
await fs.writeFile(file, unityBundle([{ path: 'CAB-test', data: cab, flags: 4 }, { path: 'CAB-test.resS', data: resS }]));
|
||||
const list = await archiveTool({ file });
|
||||
assert.equal(list.format, 'unityfs');
|
||||
assert.deepEqual([list.unityVersion, list.bundleFormat, list.blockCompression], ['2022.3.10f1', 7, ['LZ4']]);
|
||||
assert.equal(list.assets.serializedFiles[0].typeTree, true);
|
||||
assert.equal(list.assets.containerPaths, 2);
|
||||
assert.deepEqual(list.assets.scripts, [{ class: 'Game.Player', assembly: 'Assembly-CSharp.dll', instances: 1 }]);
|
||||
const byPath = Object.fromEntries(list.entries.map(e => [e.path, e]));
|
||||
assert.equal(byPath['TextAsset/story.txt'].asset, 'assets/story.txt');
|
||||
assert.equal(byPath['MonoBehaviour/PlayerConfig.json'].asset, 'assets/config/player.asset');
|
||||
assert.deepEqual([byPath['Texture2D/grass.png'].width, byPath['Texture2D/grass.png'].textureFormat], [4, 'DXT5']);
|
||||
assert.ok(byPath['raw/CAB-test'] && byPath['raw/CAB-test.resS']);
|
||||
const out = path.join(dir, 'out');
|
||||
const ex = await archiveTool({ file, action: 'extract', out });
|
||||
assert.deepEqual(ex.skipped, []);
|
||||
assert.equal(await fs.readFile(path.join(out, 'TextAsset/story.txt'), 'utf8'), 'Er was eens…');
|
||||
const config = JSON.parse(await fs.readFile(path.join(out, 'MonoBehaviour/PlayerConfig.json'), 'utf8'));
|
||||
assert.deepEqual([config.speed, config.jumpHeight, config.title, config.waypoints, config.m_Script], [7, 2.5, 'Held', [{ x: 1, y: 2 }, { x: 3, y: 4 }], { m_FileID: 0, m_PathID: 200 }]);
|
||||
const grass = readPng(await fs.readFile(path.join(out, 'Texture2D/grass.png')));
|
||||
assert.deepEqual(pixel(grass, 1, 1), [0, 255, 0, 255]);
|
||||
assert.deepEqual(await fs.readFile(path.join(out, 'raw/CAB-test')), cab);
|
||||
assert.deepEqual(await fs.readFile(path.join(out, 'raw/CAB-test.resS')), resS);
|
||||
const only = await archiveTool({ file, action: 'extract', match: 'TextAsset/*', out: path.join(dir, 'only') });
|
||||
assert.deepEqual(only.files.map(f => f.path), ['TextAsset/story.txt']);
|
||||
await fs.rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('UnityFS variants: LZMA blocks with the table at the end, format 6, encrypted bundles, corrupt data', async () => {
|
||||
const dir = await tempDir();
|
||||
const { cab, resS } = treeBundleCab();
|
||||
const files = [{ path: 'CAB-test', data: cab }, { path: 'CAB-test.resS', data: resS }];
|
||||
const lzma = path.join(dir, 'lzma.bundle');
|
||||
await fs.writeFile(lzma, unityBundle(files, { compression: 'lzma', infoAtEnd: true, format: 6, revision: '2018.4.11f1' }));
|
||||
const list = await archiveTool({ file: lzma });
|
||||
assert.deepEqual([list.bundleFormat, list.blockCompression, list.assets.objects], [6, ['LZMA'], 5]);
|
||||
const ex = await archiveTool({ file: lzma, action: 'extract', match: 'TextAsset/*', out: path.join(dir, 'out') });
|
||||
assert.equal(await fs.readFile(path.join(dir, 'out', 'TextAsset/story.txt'), 'utf8'), 'Er was eens…');
|
||||
assert.equal(ex.extracted, 1);
|
||||
|
||||
const enc = path.join(dir, 'enc.bundle');
|
||||
await fs.writeFile(enc, unityBundle(files, { encrypted: true }));
|
||||
const e = await archiveTool({ file: enc });
|
||||
assert.equal(e.encrypted, true);
|
||||
assert.match(e.note, /never decrypted/);
|
||||
await assert.rejects(archiveTool({ file: enc, action: 'extract', out: path.join(dir, 'x') }), { code: 'UNSUPPORTED_ARCHIVE' });
|
||||
|
||||
const good = unityBundle(files, { compression: 'none' });
|
||||
const bad = path.join(dir, 'bad.bundle');
|
||||
await fs.writeFile(bad, good.subarray(0, good.length - 200));
|
||||
await assert.rejects(archiveTool({ file: bad }), { code: 'BAD_FORMAT' });
|
||||
await fs.rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('meshes → OBJ: vertex streams (float and half UVs), external vertex data, compressed meshes, quads', async () => {
|
||||
const data = Buffer.alloc(3 * 16);
|
||||
[[0, 0, 0], [1, 0, 0], [0, 2, 0]].forEach((p, i) => { p.forEach((v, k) => data.writeFloatLE(v, i * 16 + k * 4)); data.writeUInt16LE(i === 1 ? 0x3c00 : 0, i * 16 + 12); data.writeUInt16LE(i === 2 ? 0x3800 : 0, i * 16 + 14); });
|
||||
const channels = Array.from({ length: 14 }, () => ({ stream: 0, offset: 0, format: 0, dimension: 0 }));
|
||||
channels[0] = { stream: 0, offset: 0, format: 0, dimension: 3 };
|
||||
channels[4] = { stream: 0, offset: 12, format: 1, dimension: 2 };
|
||||
const mesh = { m_Name: 'tri', m_IndexFormat: 0, m_IndexBuffer: Buffer.from([0, 0, 1, 0, 2, 0]), m_SubMeshes: [{ firstByte: 0, indexCount: 3, topology: 0, baseVertex: 0 }], m_VertexData: { m_VertexCount: 3, m_Channels: channels, m_DataSize: data } };
|
||||
const obj = await meshToObj(mesh, [2021, 3, 0], () => { throw new Error('not external'); });
|
||||
assert.equal(obj.text, 'g tri\nv 0 0 0\nv -1 0 0\nv 0 2 0\nvt 0 0\nvt 1 0\nvt 0 0.5\ng tri_0\nf 3/3 2/2 1/1\n');
|
||||
// The same vertices kept in a .resS (m_StreamData), drawn as one quad.
|
||||
const quad = { ...mesh, m_IndexBuffer: Buffer.from([0, 0, 1, 0, 2, 0, 1, 0]), m_SubMeshes: [{ firstByte: 0, indexCount: 4, topology: 2 }], m_VertexData: { ...mesh.m_VertexData, m_DataSize: Buffer.alloc(0) }, m_StreamData: { offset: 0n, size: 48, path: 'archive:/CAB-x/CAB-x.resS' } };
|
||||
const fromRes = await meshToObj(quad, [2021, 3, 0], async (p, offset, size) => { assert.deepEqual([p, offset, size], ['archive:/CAB-x/CAB-x.resS', 0, 48]); return data; });
|
||||
assert.equal(fromRes.faces, 2);
|
||||
assert.match(fromRes.text, /f 3\/3 2\/2 1\/1\nf 2\/2 3\/3 1\/1\n$/);
|
||||
// Compressed: 8-bit packed positions and triangles.
|
||||
const packed = (items, bytes, range = 1, start = 0) => ({ m_NumItems: items, m_Range: range, m_Start: start, m_BitSize: 8, m_Data: Buffer.from(bytes) });
|
||||
const compressed = { m_Name: 'c', m_SubMeshes: [{ firstByte: 0, indexCount: 3, topology: 0 }], m_VertexData: { m_VertexCount: 0, m_Channels: [], m_DataSize: Buffer.alloc(0) },
|
||||
m_CompressedMesh: { m_Vertices: packed(9, [0, 0, 0, 255, 0, 0, 0, 255, 0]), m_UV: packed(0, []), m_Normals: packed(0, []), m_Triangles: packed(3, [0, 1, 2]) } };
|
||||
const c = await meshToObj(compressed, [2019, 4, 0], null);
|
||||
assert.equal(c.text, 'g c\nv 0 0 0\nv -1 0 0\nv 0 1 0\ng c_0\nf 3 2 1\n');
|
||||
assert.equal(await meshToObj({ m_Name: 'empty', m_SubMeshes: [] }, [2022, 1, 0], null), null);
|
||||
});
|
||||
@@ -29,13 +29,19 @@ re_archive {file:"Master.bank"} / {file:"Init.bnk"} → FMOD events + FSB5 samp
|
||||
re_archive {file:"Game.pak"} → Unreal: entries, compression, stored/Zlib extract; encrypted = listed only
|
||||
re_archive {file:"Build/game.data.br"} → Unity WebGL: data files, StreamingAssets, IL2CPP metadata; Emscripten .data via its loader .js
|
||||
re_archive {file:"game.swf"} → Flash: stage, AS2/AS3, classes, symbols, images/sounds/ABC extract
|
||||
re_archive {file:"<Game>_Data/data.unity3d"} → Unity build: objects by class and name, scenes, C# scripts in use (+ counts)
|
||||
re_archive {file:"<Game>_Data/sharedassets0.assets"} → the same for one serialized file (level0, globalgamemanagers, …)
|
||||
re_archive {file:"StreamingAssets/<x>.bundle", action:"extract"} → asset bundle: textures → PNG, meshes → OBJ, audio (FSB5), text, objects as JSON
|
||||
re_binary {file:"<Game>_Data/Managed/Assembly-CSharp.dll"} → Unity Mono build: game classes and methods
|
||||
re_binary {file:"Game.exe"} → arch, imports (graphics API, Steam, FMOD/Wwise…), hints
|
||||
```
|
||||
|
||||
Godot exports store imported resources (`.ctex`, `.scn`, `.res` under `.godot/imported/`) rather than the original
|
||||
files; scripts may be compiled or encrypted. Encrypted packs are reported as such — use the project source instead.
|
||||
Unity bundles and Unreal paks are identified but not unpacked.
|
||||
Unity bundles (LZ4/LZMA) and serialized files unpack without Unity. Player builds strip type trees: names, scripts in
|
||||
use, TextAssets, audio and textures still come out, other objects extract as raw bytes (bundles give JSON). ASTC and BC7
|
||||
textures are written as `.astc`/`.dds`, crunched textures as raw bytes. Encrypted bundles and Unreal paks with an
|
||||
encrypted index are listed only.
|
||||
|
||||
Only extract builds the user made or has the right to inspect. To recover a lost project, prefer backups and version
|
||||
control; a build is a last resort and gives imported (not original) assets.
|
||||
|
||||
@@ -49,7 +49,7 @@ then follow edges. Cycles mean tightly coupled code: name them in the report.
|
||||
```text
|
||||
re_binary {file} → format, arch, sections + entropy, imports/exports, PDB path, overlay, hints
|
||||
re_binary {file, strings:{match:"/http|\\.dll|res:\\/\\//i", limit:300}}
|
||||
re_archive {file} → Godot .pck (also inside the .exe), zip/apk/jar, tar.gz/.unitypackage, asar
|
||||
re_archive {file} → Godot .pck (also inside the .exe), Unity bundles/.assets, zip/apk/jar, tar.gz/.unitypackage, asar
|
||||
re_probe {file, offset?} → hex view, entropy map, embedded files, record strides
|
||||
re_extract {file, offset, length?} → carve an embedded PNG/WAV/glb/ZIP into the workspace
|
||||
```
|
||||
|
||||
@@ -13,6 +13,9 @@ Use the port-project skill for the loop; this skill holds the Unity-specific jud
|
||||
it `re_scene` cannot read them.
|
||||
- Keep the `.meta` files: they map GUIDs to scripts and prefabs. Run `re_refs` on the project first: a dangling
|
||||
GUID (missing script, pink material) is a component the port would silently lose; fix it in Unity before planning.
|
||||
- Only a build, no project? `re_archive` on `<Game>_Data/data.unity3d`, `sharedassets*.assets` or an asset bundle
|
||||
lists every object, the scene list and the C# scripts each MonoBehaviour uses; `action:"extract"` writes textures as
|
||||
PNG, meshes as OBJ, audio and TextAssets. Code still comes from `re_binary` (Mono) or `re_il2cpp` (IL2CPP).
|
||||
- GDScript or C#? Large, idiomatic C# codebases port faster to **godot-csharp** (only engine API calls change);
|
||||
smaller or designer-facing projects read better as GDScript (`to:"godot"`).
|
||||
|
||||
|
||||
Reference in new issue
Block a user