diff --git a/docs/ghidra-nativeaot.md b/docs/ghidra-nativeaot.md index 90dbf99b4..d9505a226 100644 --- a/docs/ghidra-nativeaot.md +++ b/docs/ghidra-nativeaot.md @@ -1,16 +1,25 @@ # Optional NativeAOT metadata recovery -REA integrates [Washi1337/ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot) -through an optional headless adapter. The original source is an opt-in submodule, -pinned to `effeb734fc570c32650f88b159608979dc7b423e`; its MIT license and existing -source notices are retained. No upstream source or algorithms are rewritten. +REA has a built-in read-only snapshot parser for x64 PE targets with .NET 8 RTR +9.1 metadata. It parses the exact admitted Ghidra session snapshot and uses an +in-memory overlay for rehydrated metadata; it does not define Ghidra DataTypes, +labels, functions, or memory bytes. The hydration semantics are adapted from +[Washi1337/ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot) at +`effeb734fc570c32650f88b159608979dc7b423e` (MIT), cross-checked against +.NET's `MethodTable.h` at runtime commit +`a3b2d40328be0be3f8dd43f0e5cc925b8827b044` (MIT). The required notice ships in +[`licenses/nativeaot-parser-MIT.txt`](../licenses/nativeaot-parser-MIT.txt). -The initial tested layout is .NET runtime **8.0.22**, NativeAOT RTR **9.1**, -x86-64 ELF and native Windows PE **targets analyzed on Linux x64**, using -Ghidra **12.1.4** and JDK **21**. The build accepts any Ghidra 12.1.x release and -javac 21 or newer, and it compiles with `--release 21`. Real verification of -this layout remains the Ghidra 12.1.4 and JDK 21 pair. Other NativeAOT layouts, -architectures, and hosts are unsupported. +The optional headless Ghidra adapter remains available for database annotations +and broader recovery. It uses the same pinned upstream commit as an opt-in +submodule and is a separate, more capable workflow. + +The parser currently handles x64 PE32+ and RTR 9.1, including bounded +DEHYDRATED_DATA processing, MethodTable relationships, virtual slots, and +frozen strings. Other RTR versions and PE architectures are reported as +unsupported or partial. The matching .NET 8.0.22 fixture is source-built and +the parser's Windows-host Ghidra workflow has not yet had a real Windows-host +run. This does not establish Windows Ghidra import or decompiler behavior. Windows Ghidra P0 has no database mutation authority. A PE/CLI or ReadyToRun assembly should use `inspect_managed_artifact`; it is not NativeAOT. @@ -43,42 +52,62 @@ arbitrary caller-supplied bytes. The bounded loader accepts regular JARs up to ## Use existing tools -Open the executable with provider `ghidra`. `inspect_native_load_image` returns -optional `observations.metadata_recovery` inline, including the detected header, -layout, recovery status, type category paths/addresses, diagnostics, annotation -coverage and derived-memory SHA. The enclosing `unsupported` load-image status -for PE/ELF concerns independent DOS loader verification; metadata recovery has -its own status. It is not a failed import. +Open the executable with provider `ghidra`. For an x64 PE, `inspect_native_load_image` +returns `observations.metadata_recovery` inline from the captured source snapshot, +including the RTR header, MethodTable addresses, frozen strings, derived overlay +identity, and examined/recovered counts. Its enclosing `unsupported` load-image +status means independent PE load-image verification was not performed; it does +not mean the metadata parser failed or the executable failed to load. -Pass one returned type path or its typed address to `inspect_native_data_type`. -Its optional `metadata_recovery` includes related/base type, implemented -interfaces and virtual slots with targets and known procedure names. Follow -these with `analyze_function`, string searches and ordinary cross-references. -For CLI use: +Pass a returned MethodTable address to `inspect_native_data_type`. A direct PE +query also derives the snapshot metadata from the authenticated Ghidra session +handshake, so it does not require a preceding load-image call. In a persistent +MCP session, load-image followed by type inspection reuses that same derivation. +The database result remains `unavailable` when Ghidra has no DataType at that +address; its `metadata_recovery.source` is `read-only-derived-overlay` and +carries related type, interface, slot, and target observations. Generated +address identities do not claim original managed names. The parser does not +merge frozen literals into Ghidra's database string inventory. Follow slot +targets with `analyze_function` and ordinary cross-references. + +The built-in CLI path uses a MethodTable address returned by the parser or +another observation: ```sh rea inspect-native-load-image ./NativeAotFixture --provider ghidra --json -rea inspect-native-data-type ./NativeAotFixture --type /NativeAOT/MethodTables/Class_ADDRESS_MT --provider ghidra --json +rea inspect-native-data-type ./NativeAotFixture --address 0xMETHODTABLE_ADDRESS --provider ghidra --json rea function ./NativeAotFixture 0xADDRESS --provider ghidra --json ``` -No new MCP workflow or approval flag is required. Configuration opts into the -adapter; without it ordinary Ghidra profiles/results remain unchanged. +The generated `/NativeAOT/...` category path belongs to the optional headless +adapter. Use `--type /NativeAOT/MethodTables/Class_ADDRESS_MT` only when that +adapter is configured and has created the corresponding Ghidra DataType. +Built-in PE analysis includes its parser contract and resource policy in the +Ghidra analysis profile; non-PE profiles do not include that contract. ## Evidence boundaries -Discovery prefers an imported RTR symbol, otherwise the upstream initialized, -non-executable-memory signature heuristic. REA rejects ambiguous candidates, -unknown layout versions, malformed directory rows, unsupported flags and -unmapped ranges before recovery. It requires the verified dehydrated layout. +The built-in parser uses an initialized non-executable PE signature scan and +requires a unique RTR 9.1 header and exactly one DEHYDRATED_DATA row. Rehydrated +bytes live only in a parser-owned overlay, identified by virtual address, size, +and SHA-256; the caller's source Buffer remains unchanged. Limits derive from +captured section extents and format field bounds. Separate resource policies +admit up to 128 MiB of captured PE bytes, 64 MiB-equivalent parser work, 64 MiB +of exact parser-owned overlay/index Buffers, and 4 MiB of serialized NativeAOT +report JSON. These are local parser limits, independent of MCP delivery; the +buffer counter does not claim to cap V8 heap or total process memory. Rows that +do not fit the report budget are omitted with explicit partial status and +coverage. Coverage counts describe the pointer scan, recovered MethodTables, +and frozen-string candidates examined, not every possible runtime type or +object. Cancellation is honored during snapshot I/O and checked immediately +before and after parsing; the synchronous parser call itself is bounded but +cannot be interrupted mid-call. Parser output never claims that a Ghidra +database type was defined. -The adapter performs rehydration, method-table recovery and frozen-object -annotation in one ephemeral Program transaction. Fatal failure/cancellation -rolls it back. Recovery may change analysis-memory bytes, never the original -executable file. Derived ranges have a digest and **no original file offset**; -they are not captured runtime state. Candidate/committed instance annotation -counts distinguish partial frozen-object coverage. Coverage concerns the -recovered candidate set, not every possible object or every runtime type. +The optional adapter performs its annotations in an ephemeral Program +transaction. That workflow may change analysis-memory bytes, never the original +executable file. It is unavailable on Windows P0, which has no database +mutation authority. REA preserves pre-recovery calling conventions and uses the loaded compiler specification default for newly created methods. The upstream universal diff --git a/docs/ghidra-provider.md b/docs/ghidra-provider.md index b1e87a188..a79afecbd 100644 --- a/docs/ghidra-provider.md +++ b/docs/ghidra-provider.md @@ -8,9 +8,9 @@ Executable bytes remain unchanged. Use [installation](installation.md#ghidra) for supported hosts, prerequisites, configuration, startup deadlines, resource controls, and cleanup recovery. [Windows P0](windows-ghidra-p0.md) has its own admission and lifecycle boundary; -[NativeAOT recovery](ghidra-nativeaot.md) is an optional, separately verified -extension. Check the [release boundary](installation.md#released-package-and-main) -when using npm. +[NativeAOT observations](ghidra-nativeaot.md) include a built-in read-only PE +snapshot parser and a separate optional annotation extension. Check the +[release boundary](installation.md#released-package-and-main) when using npm. See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds) to override the header-detected language or seed analysis. diff --git a/docs/testing.md b/docs/testing.md index f1d43ee3a..300bf473e 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -672,6 +672,13 @@ boundaries and budgets. ## Optional NativeAOT Ghidra analysis +The source parser lane is independent of the optional annotation JAR. Its +synthetic RTR 9.1 PE golden checks captured-byte digest admission, rehydration +into an immutable parser overlay, MethodTable/slot relationships, and frozen +literal extraction; provider tests exercise `inspect_native_load_image` and +address-based `inspect_native_data_type` with the database result still +unavailable. These tests do not establish Windows Ghidra host behavior. + This lane is separate from the default native lane. `build:fixtures:nativeaot` requires an existing .NET SDK 8.0.416, the platform NativeAOT compiler/linker and runtime pack 8.0.22. It builds benign sources into ignored `_reference/` and diff --git a/docs/windows-ghidra-p0.md b/docs/windows-ghidra-p0.md index 5f50d832c..7d72056f6 100644 --- a/docs/windows-ghidra-p0.md +++ b/docs/windows-ghidra-p0.md @@ -21,9 +21,12 @@ additional permission flag or degraded mode is required. - the 25 read-only Ghidra inventory, memory, and function-analysis operations. Loaded memory reads and file offsets preserve Ghidra's observed source mappings. -PE load-image inspection returns measurements with an explicit unsupported -attestation status; independent format-specific verification currently supports -DOS MZ only. +PE load-image inspection returns an explicit unsupported attestation status; +independent load-image verification currently supports DOS MZ only. For x64 PE +NativeAOT RTR 9.1, the read-only snapshot parser can report metadata through +`inspect_native_load_image` and `inspect_native_data_type` without a JAR or +database mutation. A real Windows-host NativeAOT Ghidra run is still required +to establish the complete host workflow. This boundary does not establish general Windows feature parity. Ghidra GUI state, annotations, other target architectures and formats, and Hopper-only diff --git a/licenses/nativeaot-parser-MIT.txt b/licenses/nativeaot-parser-MIT.txt new file mode 100644 index 000000000..cd793e5ba --- /dev/null +++ b/licenses/nativeaot-parser-MIT.txt @@ -0,0 +1,28 @@ +The MIT License (MIT) +===================== + +Copyright © 2023-2025 Washi + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the “Software”), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +The read-only NativeAOT parser adapts format and hydration semantics from +Washi1337/ghidra-nativeaot commit +effeb734fc570c32650f88b159608979dc7b423e (MIT). MethodTable field layout was +cross-checked against .NET runtime src/coreclr/nativeaot/Runtime/inc/MethodTable.h +at dotnet/runtime commit a3b2d40328be0be3f8dd43f0e5cc925b8827b044 (MIT). diff --git a/package.json b/package.json index 0592256db..f96e53d19 100644 --- a/package.json +++ b/package.json @@ -81,7 +81,8 @@ "bridge/pwndbg/launch_impl.py", "bridge/pwndbg/core_context.py", "third_party/pwndbg/README.md", - "third_party/pwndbg/LICENSE.md" + "third_party/pwndbg/LICENSE.md", + "licenses/nativeaot-parser-MIT.txt" ], "scripts": { "release:notes": "node scripts/release-notes.mjs", diff --git a/src/domain/native/nativeAotPe.budget.test.ts b/src/domain/native/nativeAotPe.budget.test.ts new file mode 100644 index 000000000..7e5c02af3 --- /dev/null +++ b/src/domain/native/nativeAotPe.budget.test.ts @@ -0,0 +1,176 @@ +import { describe, expect, it } from "vitest"; +import type { NativeAotParserLimits } from "./nativeAotPe.js"; +import { + NATIVE_AOT_MAX_REPORT_BYTES, + parseNativeAotPe, +} from "./nativeAotPe.js"; +import { + nativeAotPeDigest, + nativeAotPeFixture, +} from "../../../tests/fixtures/nativeaotPe.js"; + +const REPORT_BUDGET = NATIVE_AOT_MAX_REPORT_BYTES; + +describe("NativeAOT parser resource budgets", () => { + it("reports source-cap rejection without hashing or inventing omitted-byte coverage", () => { + const bytes = nativeAotPeFixture(); + const sha = nativeAotPeDigest(bytes); + const limits: NativeAotParserLimits = { + sourceBytes: bytes.length - 1, + workUnits: 1024, + workingMemoryBytes: 1024, + }; + const report = parseNativeAotPe( + bytes, + sha, + REPORT_BUDGET, + undefined, + limits, + ).summary; + expect(report).toMatchObject({ + status: "partial", + analysis_artifact_sha256: sha, + header_address: null, + coverage: { + source_bytes_examined: 0, + source_byte_budget_bytes: bytes.length - 1, + truncation_reason: "source-byte-budget", + }, + }); + expect(report.coverage.source_bytes_omitted).toBeUndefined(); + }); +}); + +describe("NativeAOT parser report admission", () => { + it("never returns an over-budget report from early exits", () => { + const bytes = nativeAotPeFixture(); + const sha = nativeAotPeDigest(bytes); + expect(() => + parseNativeAotPe(bytes, sha, 0, undefined, { + sourceBytes: bytes.length - 1, + workUnits: 1024, + workingMemoryBytes: 1024, + }), + ).toThrow(/minimum truthful report/iu); + const nonPe = Buffer.from("not a PE image"); + expect(() => parseNativeAotPe(nonPe, nativeAotPeDigest(nonPe), 0)).toThrow( + /minimum truthful report/iu, + ); + }); + + it("does not claim a missing RTR header when signature coverage hits the work limit", () => { + const bytes = nativeAotPeFixture(); + const limits: NativeAotParserLimits = { + sourceBytes: bytes.length, + workUnits: 8, + workingMemoryBytes: 1024, + }; + const report = parseNativeAotPe( + bytes, + nativeAotPeDigest(bytes), + REPORT_BUDGET, + undefined, + limits, + ).summary; + expect(report).toMatchObject({ + status: "partial", + header_address: null, + truncated: true, + coverage: { + rtr_signature_scan_bytes_examined: 8, + rtr_signature_scan_bytes_omitted: expect.any(Number), + truncation_reason: "work-unit-budget", + }, + }); + expect(report.reason).not.toMatch(/No supported ReadyToRun directory/iu); + }); + + it("does not claim a missing root when table discovery is incomplete", () => { + const bytes = nativeAotPeFixture(); + const report = parseNativeAotPe( + bytes, + nativeAotPeDigest(bytes), + REPORT_BUDGET, + undefined, + { + sourceBytes: bytes.length, + workUnits: 10_000, + workingMemoryBytes: 1024, + }, + ).summary; + expect(report.truncated).toBe(true); + expect(report.reason).toMatch(/before determining whether.*root exists/iu); + expect(report.reason).not.toMatch( + /no unique System\.Object.*fully examined/iu, + ); + }); +}); + +describe("NativeAOT detail and summary byte accounting", () => { + it("admits serialized MethodTable memory descriptors before emitting them", () => { + const bytes = nativeAotPeFixture(); + const sha = nativeAotPeDigest(bytes); + const detail = (budget: number) => + parseNativeAotPe(bytes, sha, REPORT_BUDGET).readTypeDetail( + "0x140002280", + budget, + ); + const complete = detail(REPORT_BUDGET); + expect(complete?.derived_memory).not.toBeNull(); + let low = 0; + let high = Buffer.byteLength(JSON.stringify(complete)); + while (low < high) { + const middle = Math.floor((low + high) / 2); + if (detail(middle)?.derived_memory != null) high = middle; + else low = middle + 1; + } + const atBoundary = detail(low); + const belowBoundary = detail(low - 1); + expect(atBoundary?.derived_memory).not.toBeNull(); + expect(Buffer.byteLength(JSON.stringify(atBoundary))).toBeLessThanOrEqual( + low, + ); + expect(belowBoundary?.derived_memory ?? null).toBeNull(); + expect(belowBoundary?.diagnostics.join(" ")).toContain( + "omitted because the final serialized report exceeded its byte budget", + ); + expect( + belowBoundary === undefined + ? 0 + : Buffer.byteLength(JSON.stringify(belowBoundary)), + ).toBeLessThanOrEqual(low - 1); + }); + + it("truncates report rows before exceeding its JSON byte budget", () => { + const bytes = nativeAotPeFixture(); + const sha = nativeAotPeDigest(bytes); + const complete = parseNativeAotPe(bytes, sha, REPORT_BUDGET); + const fullBytes = Buffer.byteLength(JSON.stringify(complete.summary)); + let budget = fullBytes - 1; + let summary: ReturnType["summary"] | undefined; + while (budget > 0) { + try { + const candidate = parseNativeAotPe(bytes, sha, budget); + if (candidate.summary.truncated) { + summary = candidate.summary; + break; + } + } catch { + // Budgets below the fixed report header cannot carry an honest summary. + } + budget--; + } + if (summary === undefined) + throw new Error("No partial summary budget found"); + expect(summary.truncated).toBe(true); + expect( + (summary.types_omitted ?? 0) + + (summary.frozen_strings_omitted ?? 0) + + (summary.coverage.scan_bytes_omitted ?? 0) + + (summary.coverage.frozen_scan_bytes_omitted ?? 0), + ).toBeGreaterThan(0); + expect(Buffer.byteLength(JSON.stringify(summary))).toBeLessThanOrEqual( + budget, + ); + }); +}); diff --git a/src/domain/native/nativeAotPe.test.ts b/src/domain/native/nativeAotPe.test.ts new file mode 100644 index 000000000..bdefbab98 --- /dev/null +++ b/src/domain/native/nativeAotPe.test.ts @@ -0,0 +1,105 @@ +import { describe, expect, it } from "vitest"; +import { nativeMetadataRecoverySummarySchema } from "./nativeMetadataRecovery.js"; +import { + NATIVE_AOT_MAX_REPORT_BYTES, + parseNativeAotPe, +} from "./nativeAotPe.js"; +import { + nativeAotPeDigest, + nativeAotPeFixture, +} from "../../../tests/fixtures/nativeaotPe.js"; +const SUMMARY_BUDGET = NATIVE_AOT_MAX_REPORT_BYTES; + +describe("read-only NativeAOT PE parser", () => { + it("recovers a bounded RTR9.1 type graph and frozen literal from captured bytes", () => { + const bytes = nativeAotPeFixture(); + const before = Buffer.from(bytes); + const parsed = parseNativeAotPe( + bytes, + nativeAotPeDigest(bytes), + SUMMARY_BUDGET, + "0x140000000", + ); + expect(bytes).toEqual(before); + expect(parsed.summary).toMatchObject({ + format: "dotnet-nativeaot", + status: "complete", + analysis_mode: "read-only-derived-overlay", + format_major: 9, + format_minor: 1, + method_tables: 3, + frozen_strings: [ + { address: "0x140002300", value: "REA_NATIVEAOT_FROZEN" }, + ], + derived_overlay: { address: "0x140002380", size_bytes: 1 }, + coverage: { + basis: "raw-image-and-rehydrated-pointer-scan", + method_tables_recovered: 3, + frozen_string_candidates: 1, + }, + }); + expect(parsed.readTypeDetail("0x140002280", SUMMARY_BUDGET)).toMatchObject({ + source: "read-only-derived-overlay", + method_table_address: "0x140002280", + related_type: { address: "0x140002200", type: null }, + virtual_slots: [{ slot: 0, target_address: "0x140001040" }], + }); + expect(nativeMetadataRecoverySummarySchema.parse(parsed.summary)).toEqual( + parsed.summary, + ); + }); + + it("fails closed on snapshot identity mismatch", () => { + const bytes = nativeAotPeFixture(); + expect(() => + parseNativeAotPe(bytes, "0".repeat(64), SUMMARY_BUDGET), + ).toThrow(/digest/iu); + }); + + it("reports a detected but unsupported RTR version as partial without guessed types", () => { + const bytes = nativeAotPeFixture(10, 0); + const parsed = parseNativeAotPe( + bytes, + nativeAotPeDigest(bytes), + SUMMARY_BUDGET, + ); + expect(parsed.summary).toMatchObject({ + status: "partial", + format_major: 10, + format_minor: 0, + method_tables: 0, + types: [], + }); + }); + + it("does not claim an image mapping when Ghidra rebased the image", () => { + const bytes = nativeAotPeFixture(); + expect( + parseNativeAotPe( + bytes, + nativeAotPeDigest(bytes), + SUMMARY_BUDGET, + "0x180000000", + ).summary.reason, + ).toMatch(/preferred image base/iu); + }); + + it("reports an unsupported hydration opcode as partial without mutating the snapshot", () => { + const bytes = nativeAotPeFixture(); + bytes.writeUInt8(6, 0x704); + const before = Buffer.from(bytes); + const parsed = parseNativeAotPe( + bytes, + nativeAotPeDigest(bytes), + SUMMARY_BUDGET, + ); + expect(bytes).toEqual(before); + expect(parsed.summary).toMatchObject({ + status: "partial", + derived_overlay: null, + diagnostics: [ + expect.stringContaining("Unsupported DEHYDRATED_DATA command"), + ], + }); + }); +}); diff --git a/src/domain/native/nativeAotPe.ts b/src/domain/native/nativeAotPe.ts new file mode 100644 index 000000000..b86e3f2b8 --- /dev/null +++ b/src/domain/native/nativeAotPe.ts @@ -0,0 +1,1800 @@ +import { createHash } from "node:crypto"; +import type { NativeMetadataRecovery } from "./nativeMetadataRecovery.js"; + +const RTR_SIGNATURE = 0x00525452; +const RTR_ROW_SIZE = 24; +const RTR_ROW_TYPE = 1; +const DEHYDRATED_DATA = 207; +const FROZEN_OBJECT_REGION = 206; +export const NATIVE_AOT_MAX_SOURCE_BYTES = 128 * 1024 * 1024; +/** Bound one parser-produced JSON report independently of any transport. */ +export const NATIVE_AOT_MAX_REPORT_BYTES = 4 * 1024 * 1024; +export const NATIVE_AOT_MAX_WORK_UNITS = 64 * 1024 * 1024; +export const NATIVE_AOT_MAX_WORKING_MEMORY_BYTES = 64 * 1024 * 1024; +/** Version of the built-in read-only PE metadata interpretation in profiles. */ +export const NATIVE_AOT_PROFILE_CONTRACT_REVISION = + "rtr-9.1-x64-pe-read-only-v1"; + +export type NativeAotParserLimits = { + readonly sourceBytes: number; + readonly workUnits: number; + readonly workingMemoryBytes: number; +}; +export const DEFAULT_NATIVE_AOT_PARSER_LIMITS: NativeAotParserLimits = { + sourceBytes: NATIVE_AOT_MAX_SOURCE_BYTES, + workUnits: NATIVE_AOT_MAX_WORK_UNITS, + workingMemoryBytes: NATIVE_AOT_MAX_WORKING_MEMORY_BYTES, +}; + +class NativeAotParserBudget { + workUnits = 0; + workingMemoryBytes = 0; + workingMemoryBytesPeak = 0; + truncationReason: + | "work-unit-budget" + | "working-memory-budget" + | "response-budget" + | undefined; + + constructor(readonly limits: NativeAotParserLimits) {} + + consumeWork(bytes: number): boolean { + if ( + !Number.isSafeInteger(bytes) || + bytes < 0 || + bytes > this.limits.workUnits - this.workUnits + ) { + this.truncationReason ??= "work-unit-budget"; + return false; + } + this.workUnits += bytes; + return true; + } + + reserveWorkingMemory(bytes: number): boolean { + if ( + !Number.isSafeInteger(bytes) || + bytes < 0 || + bytes > this.limits.workingMemoryBytes - this.workingMemoryBytes + ) { + this.truncationReason ??= "working-memory-budget"; + return false; + } + this.workingMemoryBytes += bytes; + this.workingMemoryBytesPeak = Math.max( + this.workingMemoryBytesPeak, + this.workingMemoryBytes, + ); + return true; + } + + releaseWorkingMemory(bytes: number): void { + this.workingMemoryBytes = Math.max(0, this.workingMemoryBytes - bytes); + } +} + +type Section = { + readonly name: string; + readonly rva: number; + readonly virtualSize: number; + readonly rawOffset: number; + readonly rawSize: number; + readonly characteristics: number; +}; +type RtrSection = { + readonly type: number; + readonly flags: number; + readonly start: bigint; + readonly end: bigint; +}; +type MethodTable = { + readonly address: bigint; + readonly flags: number; + readonly baseSize: number; + readonly related: bigint; + readonly slotsCount: number; + readonly interfacesCount: number; +}; +const METHOD_TABLE_RECORD_BYTES = 32; +class MethodTableIndex { + #records = Buffer.alloc(0); + #count = 0; + #includedCount = 0; + + constructor(private readonly budget: NativeAotParserBudget) {} + + get count(): number { + return this.#count; + } + get includedCount(): number { + return this.#includedCount; + } + + append(table: MethodTable): boolean { + if (!this.#ensureCapacity(this.#count + 1)) return false; + const offset = this.#count * METHOD_TABLE_RECORD_BYTES; + this.#records.writeBigUInt64LE(table.address, offset); + this.#records.writeUInt32LE(table.flags, offset + 8); + this.#records.writeUInt32LE(table.baseSize, offset + 12); + this.#records.writeBigUInt64LE(table.related, offset + 16); + this.#records.writeUInt16LE(table.slotsCount, offset + 24); + this.#records.writeUInt16LE(table.interfacesCount, offset + 26); + this.#records.writeUInt8(0, offset + 28); + this.#records.writeUInt8(0, offset + 29); + this.#records.writeUInt16LE(0, offset + 30); + this.#count++; + return true; + } + + at(index: number): MethodTable | undefined { + if (!Number.isSafeInteger(index) || index < 0 || index >= this.#count) + return undefined; + const offset = index * METHOD_TABLE_RECORD_BYTES; + return { + address: this.#records.readBigUInt64LE(offset), + flags: this.#records.readUInt32LE(offset + 8), + baseSize: this.#records.readUInt32LE(offset + 12), + related: this.#records.readBigUInt64LE(offset + 16), + slotsCount: this.#records.readUInt16LE(offset + 24), + interfacesCount: this.#records.readUInt16LE(offset + 26), + }; + } + + isIncluded(index: number): boolean { + return ( + index >= 0 && + index < this.#count && + this.#records.readUInt8(index * METHOD_TABLE_RECORD_BYTES + 28) !== 0 + ); + } + + include(index: number): void { + if (!this.isIncluded(index)) { + this.#records.writeUInt8(1, index * METHOD_TABLE_RECORD_BYTES + 28); + this.#includedCount++; + } + } + + find(address: bigint, budget: NativeAotParserBudget): number { + let low = 0; + let high = this.#count; + while (low < high) { + if (!budget.consumeWork(8)) return -1; + const middle = low + ((high - low) >>> 1); + const record = middle * METHOD_TABLE_RECORD_BYTES; + const candidate = this.#records.readBigUInt64LE(record); + if (candidate < address) low = middle + 1; + else high = middle; + } + return low < this.#count && + this.#records.readBigUInt64LE(low * METHOD_TABLE_RECORD_BYTES) === address + ? low + : -1; + } + + forEach(callback: (table: MethodTable, index: number) => void): void { + for (let index = 0; index < this.#count; index++) { + const table = this.at(index); + if (table !== undefined) callback(table, index); + } + } + + forEachIncluded(callback: (table: MethodTable, index: number) => void): void { + for (let index = 0; index < this.#count; index++) { + if (!this.isIncluded(index)) continue; + const table = this.at(index); + if (table !== undefined) callback(table, index); + } + } + + #ensureCapacity(requiredRecords: number): boolean { + const requiredBytes = requiredRecords * METHOD_TABLE_RECORD_BYTES; + if (requiredBytes <= this.#records.byteLength) return true; + const capacityBytes = Math.min( + this.budget.limits.workingMemoryBytes, + Math.max(requiredBytes, this.#records.byteLength * 2 || 4096), + ); + if ( + capacityBytes < requiredBytes || + !this.budget.reserveWorkingMemory(capacityBytes) + ) + return false; + const expanded = Buffer.allocUnsafe(capacityBytes); + this.#records.copy(expanded, 0, 0, this.#count * METHOD_TABLE_RECORD_BYTES); + this.budget.releaseWorkingMemory(this.#records.byteLength); + this.#records = expanded; + return true; + } +} +type ImageOverlay = { + readonly base: bigint; + readonly sizeBytes: number; + readonly sha256: string; + readonly bytes: Buffer; +}; +export type NativeAotType = { + readonly address: string; + readonly type: null; +}; +export type NativeAotString = { + readonly address: string; + readonly value: string; +}; +export type NativeAotReport = { + readonly format: "dotnet-nativeaot"; + readonly status: "complete" | "partial" | "not_applicable"; + readonly reason: string | null; + readonly analysis_artifact_sha256: string; + readonly analysis_mode: "read-only-derived-overlay"; + readonly header_address: string | null; + readonly discovery: "signature-heuristic" | null; + readonly format_major: number | null; + readonly format_minor: number | null; + readonly method_tables: number; + readonly types: readonly NativeAotType[]; + readonly frozen_strings: readonly NativeAotString[]; + readonly truncated?: boolean; + readonly types_omitted?: number; + readonly frozen_strings_omitted?: number; + readonly output_budget_bytes?: number; + readonly derived_overlay: { + readonly address: string; + readonly size_bytes: number; + readonly sha256: string; + } | null; + readonly derived_memory: null; + readonly coverage: { + readonly pointer_candidates_examined: number; + readonly pointer_slots_examined?: number; + readonly scan_bytes_examined?: number; + readonly scan_bytes_omitted?: number; + readonly rtr_signature_scan_bytes_examined?: number; + readonly rtr_signature_scan_bytes_omitted?: number; + readonly method_table_validation_items_examined?: number; + readonly work_units_examined?: number; + readonly method_tables_recovered: number; + readonly frozen_string_candidates: number; + readonly frozen_scan_bytes_examined?: number; + readonly frozen_scan_bytes_omitted?: number; + readonly frozen_string_data_bytes_examined?: number; + readonly source_bytes_examined?: number; + readonly source_bytes_omitted?: number; + readonly work_budget_units?: number; + readonly source_byte_budget_bytes?: number; + readonly working_memory_budget_bytes?: number; + readonly working_memory_bytes_peak?: number; + readonly truncation_reason?: + | "source-byte-budget" + | "work-unit-budget" + | "working-memory-budget" + | "response-budget"; + readonly basis: + | "raw-image-pointer-scan" + | "raw-image-and-rehydrated-pointer-scan" + | "unsupported-layout" + | "no-rtr-header"; + }; + readonly diagnostics: readonly string[]; + readonly limitations: readonly string[]; +}; +export type NativeAotPeResult = { + readonly summary: NativeAotReport; + readonly readTypeDetail: ( + address: string, + maximumBytes: number, + ) => NativeMetadataRecovery | undefined; +}; + +export const nativeAotSnapshotOverCapacity = ( + sha256: string, + sourceBytesAtLeast: number, + maximumSummaryBytes: number, + limits: NativeAotParserLimits = DEFAULT_NATIVE_AOT_PARSER_LIMITS, +): NativeAotReport => + assertNativeAotReportFits( + { + format: "dotnet-nativeaot", + status: "partial", + reason: + "The captured PE exceeds the read-only NativeAOT snapshot byte budget.", + analysis_artifact_sha256: sha256, + analysis_mode: "read-only-derived-overlay", + header_address: null, + discovery: null, + format_major: null, + format_minor: null, + method_tables: 0, + types: [], + frozen_strings: [], + truncated: true, + types_omitted: 0, + frozen_strings_omitted: 0, + output_budget_bytes: maximumSummaryBytes, + derived_overlay: null, + derived_memory: null, + coverage: { + pointer_candidates_examined: 0, + scan_bytes_examined: 0, + scan_bytes_omitted: 0, + method_table_validation_items_examined: 0, + work_units_examined: 0, + method_tables_recovered: 0, + frozen_string_candidates: 0, + frozen_scan_bytes_examined: 0, + frozen_scan_bytes_omitted: 0, + frozen_string_data_bytes_examined: 0, + source_bytes_examined: 0, + work_budget_units: limits.workUnits, + source_byte_budget_bytes: limits.sourceBytes, + working_memory_budget_bytes: limits.workingMemoryBytes, + working_memory_bytes_peak: 0, + truncation_reason: "source-byte-budget", + basis: "unsupported-layout", + }, + diagnostics: [ + `The private snapshot was observed to contain at least ${sourceBytesAtLeast} bytes, above the ${limits.sourceBytes} byte NativeAOT source limit; no parser snapshot was admitted.`, + ], + limitations: [ + "NativeAOT metadata was not inspected because the captured snapshot exceeded the local source-byte budget.", + "The artifact digest identifies the selected target; no metadata facts are inferred from the omitted snapshot.", + "No Ghidra database types, labels, functions, or bytes are created or changed.", + ], + }, + maximumSummaryBytes, + ); + +const assertNativeAotReportFits = ( + report: NativeAotReport, + maximumBytes: number, +): NativeAotReport => { + if ( + !Number.isSafeInteger(maximumBytes) || + maximumBytes < 0 || + Buffer.byteLength(JSON.stringify(report)) > maximumBytes + ) + throw new Error( + "NativeAOT report-byte budget is too small for the minimum truthful report", + ); + return report; +}; + +const parsedSummary = ( + summary: NativeAotReport, + maximumBytes: number, +): NativeAotPeResult => ({ + summary: assertNativeAotReportFits(summary, maximumBytes), + readTypeDetail: () => undefined, +}); + +class PeImage { + readonly imageBase: bigint; + readonly sizeOfImage: number; + readonly sections: readonly Section[]; + readonly #bytes: Buffer; + readonly #sizeOfHeaders: number; + readonly #machine: number; + readonly #overlays: ImageOverlay[] = []; + #budget: NativeAotParserBudget | undefined; + + constructor(bytes: Buffer) { + this.#bytes = bytes; + if (bytes.length < 0x40 || bytes.toString("ascii", 0, 2) !== "MZ") + throw new Error("Not a PE image"); + const peOffset = this.#u32(0x3c); + this.#range(peOffset, 24); + if (bytes.toString("ascii", peOffset, peOffset + 4) !== "PE\0\0") + throw new Error("Invalid PE signature"); + this.#machine = this.#u16(peOffset + 4); + const count = this.#u16(peOffset + 6); + // Microsoft documents a 96-section Windows image-loader limit. This + // parser accepts executable PE images, not arbitrary COFF object files. + if (count > 96) + throw new Error("PE image exceeds the Windows 96-section limit"); + const optionalSize = this.#u16(peOffset + 20); + const optional = peOffset + 24; + this.#range(optional, optionalSize); + const magic = this.#u16(optional); + if (magic !== 0x20b || this.#machine !== 0x8664) + throw new Error("Only x64 PE32+ is supported"); + this.imageBase = this.#u64(optional + 24); + this.sizeOfImage = this.#u32(optional + 56); + this.#sizeOfHeaders = this.#u32(optional + 60); + if (count < 1 || this.sizeOfImage === 0) + throw new Error("Unsupported PE image extent"); + const sectionOffset = optional + optionalSize; + this.#range(sectionOffset, count * 40); + this.sections = Array.from({ length: count }, (_, index) => { + const p = sectionOffset + index * 40; + const name = bytes.toString("ascii", p, p + 8).replace(/\0.*$/u, ""); + return { + name, + virtualSize: this.#u32(p + 8), + rva: this.#u32(p + 12), + rawSize: this.#u32(p + 16), + rawOffset: this.#u32(p + 20), + characteristics: this.#u32(p + 36), + }; + }); + for (const section of this.sections) { + this.#range(section.rawOffset, section.rawSize); + if ( + section.rva + Math.max(section.virtualSize, section.rawSize) > + this.sizeOfImage + ) + throw new Error("PE section exceeds the declared image extent"); + } + } + + addressForRva(rva: number): bigint { + return this.imageBase + BigInt(rva); + } + setBudget(budget: NativeAotParserBudget): void { + this.#budget = budget; + } + hasImageRange(start: bigint, end: bigint): boolean { + return ( + end >= start && + start >= this.imageBase && + end - this.imageBase <= BigInt(this.sizeOfImage) + ); + } + read(address: bigint, size: number): Buffer | null { + if (address < this.imageBase || !Number.isSafeInteger(size) || size < 0) + return null; + for (const overlay of this.#overlays) { + const offset = address - overlay.base; + if (offset >= 0n && offset <= BigInt(overlay.sizeBytes - size)) { + const start = Number(offset); + return overlay.bytes.subarray(start, start + size); + } + } + const rva = address - this.imageBase; + if (rva > BigInt(Number.MAX_SAFE_INTEGER)) return null; + const relative = Number(rva); + if (relative < this.#sizeOfHeaders && relative <= this.#bytes.length - size) + return this.#bytes.subarray(relative, relative + size); + for (const section of this.sections) { + if (this.#budget !== undefined && !this.#budget.consumeWork(1)) + return null; + const extent = Math.max(section.virtualSize, section.rawSize); + if (relative < section.rva || relative - section.rva > extent - size) + continue; + const delta = relative - section.rva; + if (delta > section.rawSize - size) return null; + const offset = section.rawOffset + delta; + return this.#bytes.subarray(offset, offset + size); + } + return null; + } + isExecutable(address: bigint): boolean { + if (address < this.imageBase) return false; + const rva = address - this.imageBase; + for (const section of this.sections) { + if (this.#budget !== undefined && !this.#budget.consumeWork(1)) + return false; + if ( + (section.characteristics & 0x20000000) !== 0 && + rva >= BigInt(section.rva) && + rva < BigInt(section.rva + section.rawSize) + ) + return true; + } + return false; + } + addOverlay(overlay: ImageOverlay): void { + this.#overlays.push(overlay); + } + virtualBytesRemaining(address: bigint): number { + if (address < this.imageBase) return 0; + const rva = address - this.imageBase; + let section: Section | undefined; + for (const candidate of this.sections) { + if (this.#budget !== undefined && !this.#budget.consumeWork(1)) return 0; + if ( + rva >= BigInt(candidate.rva) && + rva < + BigInt( + candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize), + ) + ) { + section = candidate; + break; + } + } + return section === undefined + ? 0 + : section.rva + + Math.max(section.virtualSize, section.rawSize) - + Number(rva); + } + #range(offset: number, size: number): void { + if ( + !Number.isSafeInteger(offset) || + !Number.isSafeInteger(size) || + offset < 0 || + size < 0 || + offset > this.#bytes.length - size + ) + throw new Error("PE field leaves the captured file"); + } + #u16(offset: number): number { + this.#range(offset, 2); + return this.#bytes.readUInt16LE(offset); + } + #u32(offset: number): number { + this.#range(offset, 4); + return this.#bytes.readUInt32LE(offset); + } + #u64(offset: number): bigint { + this.#range(offset, 8); + return this.#bytes.readBigUInt64LE(offset); + } +} + +const hex = (value: bigint): string => `0x${value.toString(16)}`; +const readU64 = (bytes: Buffer | null, offset = 0): bigint | null => + bytes === null || offset > bytes.length - 8 + ? null + : bytes.readBigUInt64LE(offset); +const readU32 = (bytes: Buffer | null, offset = 0): number | null => + bytes === null || offset > bytes.length - 4 + ? null + : bytes.readUInt32LE(offset); +const notApplicable = ( + sha: string, + reason: string, + basis: NativeAotReport["coverage"]["basis"], +): NativeAotReport => ({ + format: "dotnet-nativeaot", + status: "not_applicable", + reason, + analysis_artifact_sha256: sha, + analysis_mode: "read-only-derived-overlay", + header_address: null, + discovery: null, + format_major: null, + format_minor: null, + method_tables: 0, + types: [], + frozen_strings: [], + derived_overlay: null, + derived_memory: null, + coverage: { + pointer_candidates_examined: 0, + method_tables_recovered: 0, + frozen_string_candidates: 0, + basis, + }, + diagnostics: [], + limitations: [ + "This report is derived from the admitted file snapshot; it does not define or annotate Ghidra database types.", + ], +}); + +type RtrDirectory = { + readonly address: bigint; + readonly major: number; + readonly minor: number; + readonly sections: readonly RtrSection[]; +}; +type RtrDirectorySearch = { + readonly directory: RtrDirectory | null; + readonly matches: number; + readonly scanBytesExamined: number; + readonly scanBytesOmitted: number; + readonly truncated: boolean; +}; + +const findDirectory = ( + image: PeImage, + bytes: Buffer, + budget: NativeAotParserBudget, +): RtrDirectorySearch => { + let found: RtrDirectory | null = null; + let matches = 0; + let scanBytesExamined = 0; + let totalScanBytes = 0; + let truncated = false; + scan: for (const section of image.sections) { + if ( + (section.characteristics & 0x40000000) === 0 || + (section.characteristics & 0x20000000) !== 0 + ) + continue; + const begin = section.rawOffset; + const end = begin + section.rawSize; + const first = (begin + 7) & ~7; + const candidateBytes = + first <= end - 16 ? (Math.floor((end - 16 - first) / 8) + 1) * 8 : 0; + totalScanBytes += candidateBytes; + for (let offset = first; offset <= end - 16; offset += 8) { + if (!budget.consumeWork(8)) { + truncated = true; + break scan; + } + scanBytesExamined += 8; + if (bytes.readUInt32LE(offset) !== RTR_SIGNATURE) continue; + if (!budget.consumeWork(16)) { + truncated = true; + break scan; + } + const count = bytes.readUInt16LE(offset + 12); + if ( + bytes.readUInt8(offset + 14) !== RTR_ROW_SIZE || + bytes.readUInt8(offset + 15) !== RTR_ROW_TYPE || + count < 1 || + offset > end - (16 + count * RTR_ROW_SIZE) + ) + continue; + const rows: RtrSection[] = []; + let valid = true; + for (let row = 0; row < count; row++) { + if (!budget.consumeWork(RTR_ROW_SIZE)) { + truncated = true; + break scan; + } + const p = offset + 16 + row * RTR_ROW_SIZE; + const type = bytes.readUInt32LE(p); + const start = bytes.readBigUInt64LE(p + 8); + const finish = bytes.readBigUInt64LE(p + 16); + const isConsumedRange = + type === FROZEN_OBJECT_REGION || type === DEHYDRATED_DATA; + const isTypeManagerIndirection = type === 204; + if ( + isConsumedRange && + finish !== start && + !image.hasImageRange(start, finish) + ) { + valid = false; + break; + } + if ( + isTypeManagerIndirection && + start !== 0n && + image.read(start, 8) === null + ) { + valid = false; + break; + } + if (isConsumedRange || isTypeManagerIndirection) { + if (rows.some((existing) => existing.type === type)) { + valid = false; + break; + } + rows.push({ + type, + flags: bytes.readUInt32LE(p + 4), + start, + end: finish, + }); + } + } + if (valid) { + matches++; + if (found === null) + found = { + address: image.addressForRva(section.rva + offset - begin), + major: bytes.readUInt16LE(offset + 4), + minor: bytes.readUInt16LE(offset + 6), + sections: rows, + }; + } + } + } + return { + directory: found, + matches, + scanBytesExamined, + scanBytesOmitted: Math.max(0, totalScanBytes - scanBytesExamined), + truncated, + }; +}; + +const parseMethodTable = ( + image: PeImage, + address: bigint, + remainingValidationItems: number, +): MethodTable | null => { + const header = image.read(address, 16); + if (header === null) return null; + const flags = header.readUInt32LE(0), + baseSize = header.readUInt32LE(4), + related = header.readBigUInt64LE(8); + const counts = image.read(address + 16n, 4); + if (counts === null) return null; + const slotsCount = counts.readUInt16LE(0), + interfacesCount = counts.readUInt16LE(2); + if (baseSize > 0x10000000 || (baseSize !== 0 && baseSize < 16)) return null; + const itemCount = slotsCount + interfacesCount; + if (itemCount > remainingValidationItems) return null; + const tableBytes = 24 + itemCount * 8; + const table = image.read(address, tableBytes); + if (table === null) return null; + for (let slot = 0; slot < slotsCount; slot++) { + const target = table.readBigUInt64LE(24 + slot * 8); + if (target !== 0n && !image.isExecutable(target)) return null; + } + for (let index = 0; index < interfacesCount; index++) { + const iface = table.readBigUInt64LE(24 + (slotsCount + index) * 8); + if (iface !== 0n && image.read(iface, 20) === null) return null; + } + return { address, flags, baseSize, related, slotsCount, interfacesCount }; +}; + +const readRelativePointer = ( + image: PeImage, + address: bigint, +): bigint | null => { + const raw = image.read(address, 4); + return raw === null ? null : address + BigInt(raw.readInt32LE(0)); +}; + +/** Exact UTF-8 length of JSON.stringify for a UTF-16 string without allocating it. */ +const jsonUtf16StringBytes = (bytes: Buffer, codeUnits: number): number => { + let total = 2; + for (let index = 0; index < codeUnits; index++) { + const value = bytes.readUInt16LE(index * 2); + if (value === 0x22 || value === 0x5c) total += 2; + else if (value < 0x20) + total += + value === 8 || + value === 9 || + value === 10 || + value === 12 || + value === 13 + ? 2 + : 6; + else if (value >= 0xd800 && value <= 0xdbff) { + const next = + index + 1 < codeUnits ? bytes.readUInt16LE((index + 1) * 2) : 0; + if (next >= 0xdc00 && next <= 0xdfff) { + total += 4; + index++; + } else total += 6; + } else if (value >= 0xdc00 && value <= 0xdfff) total += 6; + else if (value <= 0x7f) total++; + else if (value <= 0x7ff) total += 2; + else total += 3; + } + return total; +}; + +/** Versioned .NET 8 RTR9.1 hydration, adapted from Washi's MIT-licensed + * `ghidra-nativeaot` at effeb734fc570c32650f88b159608979dc7b423e. The + * MethodTable layout is cross-checked against the .NET Foundation MIT-licensed + * MethodTable.h at runtime commit a3b2d40328be0be3f8dd43f0e5cc925b8827b044. + * See licenses/nativeaot-parser-MIT.txt. Output is an isolated parser overlay; + * input snapshot bytes stay immutable. */ +const hydrate = ( + image: PeImage, + dehydrated: RtrSection, + budget: NativeAotParserBudget, +): { + readonly base: bigint; + readonly sizeBytes: number; + readonly sha256: string; +} | null => { + const length = dehydrated.end - dehydrated.start; + if ( + length < 5n || + length > BigInt(image.virtualBytesRemaining(dehydrated.start)) + ) + throw new Error("DEHYDRATED_DATA length is outside its mapped section"); + const first = image.read(dehydrated.start, Number(length)); + if (first === null) + throw new Error("DEHYDRATED_DATA leaves captured PE sections"); + const base = readRelativePointer(image, dehydrated.start); + if (base === null) + throw new Error("DEHYDRATED_DATA has no hydration base pointer"); + const outputLimit = Math.min( + image.virtualBytesRemaining(base), + budget.limits.workUnits - budget.workUnits, + budget.limits.workingMemoryBytes - budget.workingMemoryBytes, + ); + if (outputLimit === 0) + throw new Error( + "DEHYDRATED_DATA hydration base is outside mapped PE sections", + ); + let output = Buffer.alloc(Math.min(outputLimit, 64 * 1024)); + if (!budget.reserveWorkingMemory(output.byteLength)) + throw new Error("NativeAOT working-memory budget is exhausted"); + let outputLength = 0; + let cursor = 4; + const ensureOutput = (length: number): number => { + if ( + !Number.isSafeInteger(length) || + length < 0 || + length > outputLimit - outputLength + ) { + if (length > budget.limits.workUnits - budget.workUnits) + throw new Error("NativeAOT parser work budget is exhausted"); + throw new Error("Hydrated metadata exceeds its mapped PE section"); + } + const start = outputLength; + const required = start + length; + if (required > output.length) { + const capacity = Math.min( + outputLimit, + Math.max( + required, + Math.min(outputLimit, output.length * 2 || 64 * 1024), + ), + ); + if (!budget.reserveWorkingMemory(capacity)) + throw new Error("NativeAOT working-memory budget is exhausted"); + const expanded = Buffer.allocUnsafe(capacity); + output.copy(expanded, 0, 0, outputLength); + budget.releaseWorkingMemory(output.byteLength); + output = expanded; + } + outputLength = required; + return start; + }; + const append = (chunk: Buffer): void => { + if (!budget.consumeWork(chunk.length)) + throw new Error("NativeAOT parser work budget is exhausted"); + chunk.copy(output, ensureOutput(chunk.length)); + }; + const appendZero = (length: number): void => { + if (!budget.consumeWork(length)) + throw new Error("NativeAOT parser work budget is exhausted"); + const start = ensureOutput(length); + output.fill(0, start, outputLength); + }; + const appendRel = (target: bigint): void => { + if (!budget.consumeWork(4)) + throw new Error("NativeAOT parser work budget is exhausted"); + const delta = target - (base + BigInt(outputLength)); + if (delta < -0x80000000n || delta > 0x7fffffffn) + throw new Error("Hydrated relative pointer does not fit int32"); + const offset = ensureOutput(4); + output.writeInt32LE(Number(delta), offset); + }; + const appendPtr = (target: bigint): void => { + if (!budget.consumeWork(8)) + throw new Error("NativeAOT parser work budget is exhausted"); + const offset = ensureOutput(8); + output.writeBigUInt64LE(target, offset); + }; + const readFixup = (index: number): bigint => { + const fixupLocation = dehydrated.end + BigInt(index * 4); + const target = readRelativePointer(image, fixupLocation); + if (target === null) + throw new Error("DEHYDRATED_DATA fixup leaves captured PE sections"); + return target; + }; + try { + while (cursor < first.length) { + if (!budget.consumeWork(1)) + throw new Error("NativeAOT parser work budget is exhausted"); + const encoded = first[cursor]; + if (encoded === undefined) break; + cursor++; + const command = encoded & 7; + let payload = encoded >>> 3; + const extraBytes = payload - 28; + if (extraBytes > 0) { + if (extraBytes > 3 || cursor > first.length - extraBytes) + throw new Error("Invalid DEHYDRATED_DATA payload encoding"); + payload = 0; + for (let i = 0; i < extraBytes; i++) + payload += (first[cursor++] ?? 0) * 2 ** (8 * i); + payload += 28; + } + switch (command) { + case 0: { + if (cursor > first.length - payload) + throw new Error("DEHYDRATED_DATA COPY leaves its section"); + append(first.subarray(cursor, cursor + payload)); + cursor += payload; + break; + } + case 1: + appendZero(payload); + break; + case 2: + appendRel(readFixup(payload)); + break; + case 3: + appendPtr(readFixup(payload)); + break; + case 4: + if (!budget.consumeWork(payload * 4)) + throw new Error("NativeAOT parser work budget is exhausted"); + if ( + payload > Math.floor((first.length - cursor) / 4) || + payload > Math.floor((outputLimit - outputLength) / 4) + ) + throw new Error("Inline relative fixups leave DEHYDRATED_DATA"); + for (let i = 0; i < payload; i++) { + const target = readRelativePointer( + image, + dehydrated.start + BigInt(cursor), + ); + if (target === null) + throw new Error("Invalid inline relative pointer"); + cursor += 4; + appendRel(target); + } + break; + case 5: + if (!budget.consumeWork(payload * 4)) + throw new Error("NativeAOT parser work budget is exhausted"); + if ( + payload > Math.floor((first.length - cursor) / 4) || + payload > Math.floor((outputLimit - outputLength) / 8) + ) + throw new Error("Inline pointer fixups leave DEHYDRATED_DATA"); + for (let i = 0; i < payload; i++) { + const target = readRelativePointer( + image, + dehydrated.start + BigInt(cursor), + ); + if (target === null) throw new Error("Invalid inline pointer"); + cursor += 4; + appendPtr(target); + } + break; + default: + throw new Error(`Unsupported DEHYDRATED_DATA command ${command}`); + } + } + } catch (cause: unknown) { + budget.releaseWorkingMemory(output.byteLength); + throw cause; + } + const hydrated = output.subarray(0, outputLength); + const sha256 = createHash("sha256").update(hydrated).digest("hex"); + image.addOverlay({ base, sizeBytes: outputLength, sha256, bytes: hydrated }); + return { base, sizeBytes: outputLength, sha256 }; +}; + +const discoverTables = ( + image: PeImage, + hydrated: { readonly base: bigint; readonly sizeBytes: number } | null, + budget: NativeAotParserBudget, +): { + tables: MethodTableIndex; + pointerCount: number; + pointerSlotsExamined: number; + scanBytesExamined: number; + scanBytesOmitted: number; + methodTableValidationItemsExamined: number; + workBytesExamined: number; + truncated: boolean; +} => { + const ranges: { start: bigint; end: bigint }[] = []; + for (const section of image.sections) { + if ( + (section.characteristics & 0x40000000) === 0 || + (section.characteristics & 0x20000000) !== 0 + ) + continue; + if (section.rawSize > 0) { + ranges.push({ + start: image.addressForRva(section.rva), + end: image.addressForRva(section.rva + section.rawSize), + }); + } + } + if (hydrated !== null) { + const section = image.sections.find( + (candidate) => + (candidate.characteristics & 0x40000000) !== 0 && + (candidate.characteristics & 0x20000000) === 0 && + hydrated.base >= image.addressForRva(candidate.rva) && + hydrated.base + BigInt(hydrated.sizeBytes) <= + image.addressForRva( + candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize), + ), + ); + if (section !== undefined) + ranges.push({ + start: hydrated.base, + end: hydrated.base + BigInt(hydrated.sizeBytes), + }); + } + ranges.sort((left, right) => (left.start < right.start ? -1 : 1)); + const merged: { start: bigint; end: bigint }[] = []; + for (const range of ranges) { + const previous = merged.at(-1); + if (previous !== undefined && range.start <= previous.end) { + if (range.end > previous.end) previous.end = range.end; + } else { + merged.push({ ...range }); + } + } + const totalScanBytes = merged.reduce( + (total, range) => total + Number(range.end - range.start), + 0, + ); + let pointerCount = 0, + pointerSlotsExamined = 0, + scanBytesExamined = 0, + validationItems = 0, + workBytesExamined = 0, + truncated = false; + const candidates = new MethodTableIndex(budget); + scan: for (const range of merged) { + let address = (range.start + 7n) & ~7n; + while (address + 8n <= range.end) { + if (!budget.consumeWork(8)) { + truncated = true; + break scan; + } + const value = readU64(image.read(address, 8)); + pointerSlotsExamined++; + scanBytesExamined += 8; + workBytesExamined += 8; + if (value !== null && value !== 0n) pointerCount++; + const candidateAddress = address - 8n; + const header = + candidateAddress >= image.imageBase + ? image.read(candidateAddress, 24) + : null; + if (header !== null) { + const flags = header.readUInt32LE(0); + const elementType = (flags & 0x7c000000) >>> 26; + const baseSize = header.readUInt32LE(4); + const related = header.readBigUInt64LE(8); + const plausibleHeader = + (elementType === 0x14 || elementType === 0x15) && + baseSize >= 16 && + baseSize <= 0x10000000 && + (related === 0n || image.read(related, 24) !== null); + if (!plausibleHeader) { + address += 8n; + continue; + } + const counts = image.read(candidateAddress + 16n, 4); + if (counts === null) { + address += 8n; + continue; + } + const itemCount = counts.readUInt16LE(0) + counts.readUInt16LE(2); + if (!budget.consumeWork(itemCount * 8)) { + truncated = true; + break scan; + } + validationItems += itemCount; + workBytesExamined += itemCount * 8; + const candidate = parseMethodTable(image, candidateAddress, itemCount); + if (candidate !== null) { + if (!candidates.append(candidate)) { + truncated = true; + break scan; + } + } + } + address += 8n; + } + } + let objectRoot = -1; + let objectCount = 0; + candidates.forEach((candidate, index) => { + if ( + candidate.flags !== 0x50000000 || + candidate.baseSize !== 0x18 || + candidate.related !== 0n || + candidate.slotsCount !== 3 || + candidate.interfacesCount !== 0 + ) + return; + if (!budget.consumeWork(32)) { + truncated = true; + return; + } + const vtable = image.read(candidate.address + 24n, 32); + if ( + vtable !== null && + image.isExecutable(vtable.readBigUInt64LE(0)) && + image.isExecutable(vtable.readBigUInt64LE(8)) && + image.isExecutable(vtable.readBigUInt64LE(16)) && + !image.isExecutable(vtable.readBigUInt64LE(24)) + ) { + objectRoot = index; + objectCount++; + } + }); + if (objectCount !== 1 || objectRoot < 0 || truncated) + return { + tables: candidates, + pointerCount, + pointerSlotsExamined, + scanBytesExamined, + scanBytesOmitted: Math.max(0, totalScanBytes - scanBytesExamined), + methodTableValidationItemsExamined: validationItems, + workBytesExamined, + truncated, + }; + candidates.include(objectRoot); + let changed = true; + while (changed && !truncated) { + changed = false; + candidates.forEach((base, baseIndex) => { + if (truncated || !candidates.isIncluded(baseIndex)) return; + for ( + let candidateIndex = 0; + candidateIndex < candidates.count; + candidateIndex++ + ) { + if (!budget.consumeWork(8)) { + truncated = true; + return; + } + const candidate = candidates.at(candidateIndex); + if ( + candidate !== undefined && + candidate.related === base.address && + !candidates.isIncluded(candidateIndex) + ) { + candidates.include(candidateIndex); + changed = true; + } + } + for (let index = 0; index < base.interfacesCount; index++) { + if (!budget.consumeWork(8)) { + truncated = true; + return; + } + const ifaceAddress = readU64( + image.read( + base.address + 24n + BigInt((base.slotsCount + index) * 8), + 8, + ), + ); + if (ifaceAddress === null) continue; + const interfaceIndex = candidates.find(ifaceAddress, budget); + if (budget.truncationReason !== undefined) { + truncated = true; + return; + } + if (interfaceIndex >= 0 && !candidates.isIncluded(interfaceIndex)) { + candidates.include(interfaceIndex); + changed = true; + } + } + }); + } + return { + tables: candidates, + pointerCount, + pointerSlotsExamined, + scanBytesExamined, + scanBytesOmitted: Math.max(0, totalScanBytes - scanBytesExamined), + methodTableValidationItemsExamined: validationItems, + workBytesExamined, + truncated, + }; +}; + +/** Parse only the captured x64 PE snapshot. No file reads or Ghidra database writes occur here. */ +export const parseNativeAotPe = ( + bytes: Buffer, + expectedSha256: string, + maximumSummaryBytes: number, + loadedImageBase?: string, + limits = DEFAULT_NATIVE_AOT_PARSER_LIMITS, +): NativeAotPeResult => { + if (!Number.isSafeInteger(maximumSummaryBytes) || maximumSummaryBytes < 0) + throw new Error( + "NativeAOT summary budget must be a non-negative safe integer", + ); + for (const limit of Object.values(limits)) + if (!Number.isSafeInteger(limit) || limit < 0) + throw new Error( + "NativeAOT parser limits must be non-negative safe integers", + ); + if (bytes.byteLength > limits.sourceBytes) + return parsedSummary( + nativeAotSnapshotOverCapacity( + expectedSha256, + bytes.byteLength, + maximumSummaryBytes, + limits, + ), + maximumSummaryBytes, + ); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + if (sha256 !== expectedSha256) + throw new Error( + "NativeAOT snapshot digest does not match the admitted artifact identity", + ); + let image: PeImage; + try { + image = new PeImage(bytes); + } catch (cause: unknown) { + return parsedSummary( + notApplicable( + sha256, + cause instanceof Error ? cause.message : "Unsupported PE image", + "unsupported-layout", + ), + maximumSummaryBytes, + ); + } + if ( + loadedImageBase !== undefined && + BigInt(loadedImageBase) !== image.imageBase + ) + return parsedSummary( + notApplicable( + sha256, + "Ghidra loaded the PE at a base different from its preferred image base; relocation-aware metadata mapping is not implemented.", + "unsupported-layout", + ), + maximumSummaryBytes, + ); + const budget = new NativeAotParserBudget(limits); + image.setBudget(budget); + const directorySearch = findDirectory(image, bytes, budget); + if (directorySearch.truncated) + return parsedSummary( + { + ...notApplicable( + sha256, + "RTR signature scanning reached the NativeAOT parser work budget; a unique ReadyToRun directory was not established.", + "no-rtr-header", + ), + status: "partial", + truncated: true, + output_budget_bytes: maximumSummaryBytes, + coverage: { + pointer_candidates_examined: 0, + pointer_slots_examined: 0, + scan_bytes_examined: directorySearch.scanBytesExamined, + scan_bytes_omitted: directorySearch.scanBytesOmitted, + rtr_signature_scan_bytes_examined: directorySearch.scanBytesExamined, + rtr_signature_scan_bytes_omitted: directorySearch.scanBytesOmitted, + method_tables_recovered: 0, + frozen_string_candidates: 0, + work_units_examined: budget.workUnits, + work_budget_units: budget.limits.workUnits, + source_byte_budget_bytes: budget.limits.sourceBytes, + working_memory_budget_bytes: budget.limits.workingMemoryBytes, + working_memory_bytes_peak: budget.workingMemoryBytesPeak, + ...(budget.truncationReason === undefined + ? {} + : { truncation_reason: budget.truncationReason }), + basis: "no-rtr-header", + }, + diagnostics: [ + "No metadata hydration was attempted because signature coverage was incomplete.", + ], + }, + maximumSummaryBytes, + ); + if (directorySearch.directory === null && directorySearch.matches === 0) + return parsedSummary( + notApplicable( + sha256, + "No supported ReadyToRun directory signature was found in initialized non-executable PE sections.", + "no-rtr-header", + ), + maximumSummaryBytes, + ); + if (directorySearch.matches !== 1 || directorySearch.directory === null) + return parsedSummary( + notApplicable( + sha256, + "Multiple plausible ReadyToRun directories were found; metadata identity is ambiguous.", + "unsupported-layout", + ), + maximumSummaryBytes, + ); + const directory = directorySearch.directory; + if (directory.major !== 9 || directory.minor !== 1) + return parsedSummary( + { + ...notApplicable( + sha256, + `RTR ${directory.major}.${directory.minor} is detected but this parser supports only the .NET 8 RTR 9.1 method-table layout.`, + "unsupported-layout", + ), + status: "partial", + header_address: hex(directory.address), + discovery: "signature-heuristic", + format_major: directory.major, + format_minor: directory.minor, + }, + maximumSummaryBytes, + ); + const dehydrated = directory.sections.filter( + (section) => section.type === DEHYDRATED_DATA, + ); + if (dehydrated.length !== 1) + return parsedSummary( + { + ...notApplicable( + sha256, + "RTR 9.1 directory does not contain exactly one DEHYDRATED_DATA section; metadata recovery is incomplete.", + "unsupported-layout", + ), + status: "partial", + header_address: hex(directory.address), + discovery: "signature-heuristic", + format_major: directory.major, + format_minor: directory.minor, + }, + maximumSummaryBytes, + ); + let overlay: ReturnType = null; + let hydrationError: string | null = null; + try { + overlay = hydrate(image, dehydrated[0] as RtrSection, budget); + } catch (cause: unknown) { + hydrationError = + cause instanceof Error + ? cause.message + : "DEHYDRATED_DATA could not be hydrated"; + } + const discovered = discoverTables(image, overlay, budget); + const frozen = directory.sections.find( + (section) => section.type === FROZEN_OBJECT_REGION, + ); + const isRecoveredStringTable = (address: bigint): boolean => { + const index = discovered.tables.find(address, budget); + if ( + budget.truncationReason !== undefined || + index < 0 || + !discovered.tables.isIncluded(index) + ) + return false; + const table = discovered.tables.at(index); + if ( + table === undefined || + (table.flags & 0x7c000000) >>> 26 !== 0x14 || + table.baseSize !== 0x16 || + table.related === 0n + ) + return false; + const related = discovered.tables.find(table.related, budget); + if (budget.truncationReason !== undefined) return false; + return related >= 0 && discovered.tables.isIncluded(related); + }; + const types: NativeAotType[] = []; + const frozenStrings: NativeAotString[] = []; + let frozenCandidates = 0; + let frozenStringsOmitted = 0; + let frozenScanBytesExamined = 0; + let frozenStringDataBytesExamined = 0; + let frozenScanTruncated = false; + const frozenFirstCandidate = + frozen === undefined ? 0n : (frozen.start + 7n) & ~7n; + const frozenCandidateBound = + frozen === undefined || frozen.end < frozenFirstCandidate + 14n + ? 0 + : Number((frozen.end - 14n - frozenFirstCandidate) / 8n + 1n); + const recoveryReason = + discovered.truncated && discovered.tables.includedCount === 0 + ? "RTR 9.1 was found, but method-table discovery reached its resource budget before determining whether a unique System.Object root exists." + : discovered.tables.includedCount === 0 + ? "RTR 9.1 was found, but no unique System.Object method-table root could be recovered from the fully examined image pointers." + : hydrationError; + const reportLimitations = [ + "Only x64 PE32+ with RTR 9.1 is supported.", + "No Ghidra database types, labels, functions, or bytes are created or changed.", + "Names are address identities; original managed type names are unavailable.", + "Metadata rows omitted to fit the NativeAOT report-byte budget are counted explicitly.", + "Parser work, parser-owned typed buffers, NativeAOT report JSON, and MCP delivery have separate byte budgets; incomplete scans report examined and unexamined coverage.", + ...(frozen === undefined + ? ["The RTR directory has no FROZEN_OBJECT_REGION section."] + : []), + ]; + const budgetSkeleton = { + format: "dotnet-nativeaot", + status: "complete", + reason: recoveryReason, + analysis_artifact_sha256: sha256, + analysis_mode: "read-only-derived-overlay", + header_address: hex(directory.address), + discovery: "signature-heuristic", + format_major: directory.major, + format_minor: directory.minor, + method_tables: discovered.tables.includedCount, + types: [], + frozen_strings: [], + truncated: true, + types_omitted: discovered.tables.includedCount, + frozen_strings_omitted: frozenCandidateBound, + output_budget_bytes: maximumSummaryBytes, + derived_overlay: + overlay === null + ? null + : { + address: hex(overlay.base), + size_bytes: overlay.sizeBytes, + sha256: overlay.sha256, + }, + derived_memory: null, + coverage: { + pointer_candidates_examined: discovered.pointerCount, + pointer_slots_examined: discovered.pointerSlotsExamined, + scan_bytes_examined: discovered.scanBytesExamined, + scan_bytes_omitted: discovered.scanBytesOmitted, + rtr_signature_scan_bytes_examined: directorySearch.scanBytesExamined, + rtr_signature_scan_bytes_omitted: directorySearch.scanBytesOmitted, + method_table_validation_items_examined: + discovered.methodTableValidationItemsExamined, + // Reserve the largest numeric encodings used by the eventual coverage + // fields so the summary budget remains valid after frozen-region work. + work_units_examined: budget.limits.workUnits, + work_budget_units: budget.limits.workUnits, + source_byte_budget_bytes: budget.limits.sourceBytes, + working_memory_budget_bytes: budget.limits.workingMemoryBytes, + working_memory_bytes_peak: budget.limits.workingMemoryBytes, + truncation_reason: "response-budget", + method_tables_recovered: discovered.tables.includedCount, + frozen_string_candidates: 0, + frozen_scan_bytes_examined: frozenCandidateBound * 8, + frozen_string_data_bytes_examined: + frozen === undefined ? 0 : Number(frozen.end - frozen.start), + frozen_scan_bytes_omitted: frozenCandidateBound * 8, + basis: + overlay === null + ? "raw-image-pointer-scan" + : "raw-image-and-rehydrated-pointer-scan", + }, + diagnostics: + hydrationError === null + ? [] + : [`Read-only hydration stopped: ${hydrationError}`], + limitations: reportLimitations, + }; + let remainingSummaryBytes = + maximumSummaryBytes - Buffer.byteLength(JSON.stringify(budgetSkeleton)); + if (remainingSummaryBytes < 0) + throw new Error( + "NativeAOT metadata summary exceeds its report-byte budget", + ); + let typesOmitted = 0; + discovered.tables.forEachIncluded((table) => { + const row: NativeAotType = { address: hex(table.address), type: null }; + const rowBytes = + Buffer.byteLength(JSON.stringify(row)) + (types.length > 0 ? 1 : 0); + if (rowBytes > remainingSummaryBytes) { + typesOmitted++; + return; + } + types.push(row); + remainingSummaryBytes -= rowBytes; + }); + if (frozen !== undefined) { + let address = frozenFirstCandidate; + while (address + 14n <= frozen.end) { + if (!budget.consumeWork(8)) { + frozenScanTruncated = true; + break; + } + frozenScanBytesExamined += 8; + const table = readU64(image.read(address, 8)); + if (table === null || !isRecoveredStringTable(table)) { + address += 8n; + continue; + } + frozenCandidates++; + const length = readU32(image.read(address + 8n, 4)); + if (length === null || address + 14n + BigInt(length) * 2n > frozen.end) { + address += 8n; + continue; + } + const stringBytes = length * 2 + 2; + if (!budget.consumeWork(stringBytes)) { + frozenScanTruncated = true; + break; + } + frozenStringDataBytesExamined += stringBytes; + // NativeAOT's System.String layout is the object header, 32-bit length, + // UTF-16 payload, then the terminating code unit. The payload starts at + // +12, not +16 (the latter silently shifts every decoded string by two + // characters and includes adjacent object padding). + const raw = image.read(address + 12n, stringBytes); + if (raw === null || raw.readUInt16LE(length * 2) !== 0) { + address += 8n; + continue; + } + if (!budget.consumeWork(stringBytes)) { + frozenScanTruncated = true; + break; + } + const rowPrefixBytes = Buffer.byteLength( + JSON.stringify({ address: hex(address), value: "" }), + ); + const rowBytes = + rowPrefixBytes - + 2 + + jsonUtf16StringBytes(raw, length) + + (frozenStrings.length > 0 ? 1 : 0); + if (rowBytes > remainingSummaryBytes) { + frozenStringsOmitted++; + address += 8n; + continue; + } + const row: NativeAotString = { + address: hex(address), + value: raw.toString("utf16le", 0, length * 2), + }; + frozenStrings.push(row); + remainingSummaryBytes -= rowBytes; + address += 8n; + } + } + const truncated = + discovered.truncated || + frozenScanTruncated || + typesOmitted > 0 || + frozenStringsOmitted > 0; + const truncationReason = + budget.truncationReason ?? + (typesOmitted > 0 || frozenStringsOmitted > 0 + ? "response-budget" + : undefined); + const summary: NativeAotReport = { + format: "dotnet-nativeaot", + status: + truncated || + hydrationError !== null || + discovered.tables.includedCount === 0 + ? "partial" + : "complete", + reason: recoveryReason, + analysis_artifact_sha256: sha256, + analysis_mode: "read-only-derived-overlay", + header_address: hex(directory.address), + discovery: "signature-heuristic", + format_major: directory.major, + format_minor: directory.minor, + method_tables: discovered.tables.includedCount, + types, + frozen_strings: frozenStrings, + truncated, + types_omitted: typesOmitted, + frozen_strings_omitted: frozenStringsOmitted, + output_budget_bytes: maximumSummaryBytes, + derived_overlay: + overlay === null + ? null + : { + address: hex(overlay.base), + size_bytes: overlay.sizeBytes, + sha256: overlay.sha256, + }, + derived_memory: null, + coverage: { + pointer_candidates_examined: discovered.pointerCount, + pointer_slots_examined: discovered.pointerSlotsExamined, + scan_bytes_examined: discovered.scanBytesExamined, + scan_bytes_omitted: discovered.scanBytesOmitted, + rtr_signature_scan_bytes_examined: directorySearch.scanBytesExamined, + rtr_signature_scan_bytes_omitted: directorySearch.scanBytesOmitted, + method_table_validation_items_examined: + discovered.methodTableValidationItemsExamined, + work_units_examined: budget.workUnits, + work_budget_units: budget.limits.workUnits, + source_byte_budget_bytes: budget.limits.sourceBytes, + working_memory_budget_bytes: budget.limits.workingMemoryBytes, + working_memory_bytes_peak: budget.workingMemoryBytesPeak, + ...(budget.truncationReason === undefined + ? truncationReason === undefined + ? {} + : { truncation_reason: truncationReason } + : { truncation_reason: budget.truncationReason }), + method_tables_recovered: discovered.tables.includedCount, + frozen_string_candidates: frozenCandidates, + frozen_scan_bytes_examined: frozenScanBytesExamined, + frozen_string_data_bytes_examined: frozenStringDataBytesExamined, + frozen_scan_bytes_omitted: Math.max( + 0, + frozenCandidateBound * 8 - frozenScanBytesExamined, + ), + basis: + overlay === null + ? "raw-image-pointer-scan" + : "raw-image-and-rehydrated-pointer-scan", + }, + diagnostics: [ + ...(hydrationError === null + ? [] + : [`Read-only hydration stopped: ${hydrationError}`]), + ], + limitations: reportLimitations, + }; + if (Buffer.byteLength(JSON.stringify(summary)) > maximumSummaryBytes) + throw new Error( + "NativeAOT metadata summary exceeded its report-byte budget after serialization", + ); + const readTypeDetail = ( + address: string, + maximumBytes: number, + ): NativeMetadataRecovery | undefined => { + if (!Number.isSafeInteger(maximumBytes) || maximumBytes < 0) + return undefined; + if (!/^0x[0-9a-f]+$/iu.test(address)) return undefined; + let tableAddress: bigint; + try { + tableAddress = BigInt(address); + } catch { + return undefined; + } + const detailBudget = new NativeAotParserBudget(limits); + const tableIndex = discovered.tables.find(tableAddress, detailBudget); + if (tableIndex < 0 || !discovered.tables.isIncluded(tableIndex)) + return undefined; + const table = discovered.tables.at(tableIndex); + if (table === undefined) return undefined; + const metadata: NativeMetadataRecovery = { + source: "read-only-derived-overlay", + format: "dotnet-nativeaot", + status: "complete", + header_address: hex(directory.address), + format_major: directory.major, + format_minor: directory.minor, + method_table_address: hex(table.address), + name_origin: "generated", + original_name: null, + base_size_bytes: table.baseSize, + related_type: + table.related === 0n + ? null + : { address: hex(table.related), type: null }, + interfaces: [], + virtual_slots: [], + derived_memory: null, + truncated: true, + virtual_slots_omitted: table.slotsCount, + interfaces_omitted: table.interfacesCount, + output_budget_bytes: maximumBytes, + diagnostics: [], + limitations: [ + "Address-derived MethodTable identity; no original managed type name is available.", + "This result is parser-derived and does not claim that Ghidra defined a database DataType.", + "Slot and interface details are generated only for the requested address and bounded by the NativeAOT report-byte budget.", + ], + }; + let remaining = maximumBytes - Buffer.byteLength(JSON.stringify(metadata)); + if (remaining < 0) { + metadata.diagnostics.push( + "The NativeAOT detail report budget is too small for the requested MethodTable summary.", + ); + metadata.truncated = true; + metadata.status = "partial"; + return Buffer.byteLength(JSON.stringify(metadata)) <= maximumBytes + ? metadata + : undefined; + } + const maxInterfaceRowBytes = Buffer.byteLength( + JSON.stringify({ address: `0x${"f".repeat(16)}`, type: null }), + ); + let interfacesOmitted = 0; + for (let index = 0; index < table.interfacesCount; index++) { + if (remaining < maxInterfaceRowBytes) { + interfacesOmitted += table.interfacesCount - index; + break; + } + if (!detailBudget.consumeWork(8)) { + interfacesOmitted += table.interfacesCount - index; + break; + } + const row = { + address: hex(table.address), + type: null, + }; + const value = readU64( + image.read( + table.address + 24n + BigInt((table.slotsCount + index) * 8), + 8, + ), + ); + if (value === null) { + interfacesOmitted += table.interfacesCount - index; + break; + } + row.address = hex(value); + const rowBytes = + Buffer.byteLength(JSON.stringify(row)) + + (metadata.interfaces.length > 0 ? 1 : 0); + if (rowBytes > remaining) { + interfacesOmitted++; + continue; + } + metadata.interfaces.push(row); + remaining -= rowBytes; + } + const maxVirtualSlotRowBytes = Buffer.byteLength( + JSON.stringify({ + slot: 65535, + slot_address: `0x${"f".repeat(16)}`, + target_address: `0x${"f".repeat(16)}`, + procedure_name: null, + basis: "method-table-pointer", + }), + ); + let slotsOmitted = 0; + for (let slot = 0; slot < table.slotsCount; slot++) { + if (remaining < maxVirtualSlotRowBytes) { + slotsOmitted += table.slotsCount - slot; + break; + } + if (!detailBudget.consumeWork(8)) { + slotsOmitted += table.slotsCount - slot; + break; + } + const value = readU64( + image.read(table.address + 24n + BigInt(slot * 8), 8), + ); + if (value === null) { + slotsOmitted += table.slotsCount - slot; + break; + } + const row = { + slot, + slot_address: hex(table.address + 24n + BigInt(slot * 8)), + target_address: value === 0n ? null : hex(value), + procedure_name: null, + basis: "method-table-pointer" as const, + }; + const rowBytes = + Buffer.byteLength(JSON.stringify(row)) + + (metadata.virtual_slots.length > 0 ? 1 : 0); + if (rowBytes > remaining) { + slotsOmitted++; + continue; + } + metadata.virtual_slots.push(row); + remaining -= rowBytes; + } + const tableBytes = 24 + (table.slotsCount + table.interfacesCount) * 8; + // The descriptor size is independent of the source table size. Admit the + // output representation first, then bound source hashing by parser work. + const descriptor = { + address: hex(table.address), + size_bytes: tableBytes, + sha256: "0".repeat(64), + file_offset: null, + }; + const withDescriptor = { ...metadata, derived_memory: descriptor }; + const descriptorBytes = + Buffer.byteLength(JSON.stringify(withDescriptor)) - + Buffer.byteLength(JSON.stringify(metadata)); + if (descriptorBytes <= remaining && detailBudget.consumeWork(tableBytes)) { + const raw = image.read(table.address, tableBytes); + if (raw !== null) { + metadata.derived_memory = { + ...descriptor, + sha256: createHash("sha256").update(raw).digest("hex"), + }; + remaining -= descriptorBytes; + } + } + if (detailBudget.truncationReason !== undefined) + metadata.diagnostics.push( + "MethodTable memory hashing stopped at the parser work budget.", + ); + metadata.interfaces_omitted = interfacesOmitted; + metadata.virtual_slots_omitted = slotsOmitted; + metadata.truncated = + discovered.truncated || + hydrationError !== null || + detailBudget.truncationReason !== undefined || + interfacesOmitted > 0 || + slotsOmitted > 0 || + metadata.derived_memory === null; + if (metadata.truncated) metadata.status = "partial"; + if (Buffer.byteLength(JSON.stringify(metadata)) > maximumBytes) { + metadata.derived_memory = null; + metadata.interfaces_omitted += metadata.interfaces.length; + metadata.interfaces.length = 0; + metadata.virtual_slots_omitted += metadata.virtual_slots.length; + metadata.virtual_slots.length = 0; + metadata.diagnostics.push( + "MethodTable detail rows were omitted because the final serialized report exceeded its byte budget.", + ); + metadata.truncated = true; + metadata.status = "partial"; + if (Buffer.byteLength(JSON.stringify(metadata)) > maximumBytes) + return undefined; + } + return metadata; + }; + return { summary, readTypeDetail }; +}; diff --git a/src/domain/native/nativeMetadataRecovery.ts b/src/domain/native/nativeMetadataRecovery.ts index 78217f27e..6582303a4 100644 --- a/src/domain/native/nativeMetadataRecovery.ts +++ b/src/domain/native/nativeMetadataRecovery.ts @@ -6,6 +6,8 @@ const relatedType = z.strictObject({ address, type: z.string().nullable() }); /** Runtime-format metadata recovered in an analysis image, with explicit inference boundaries. */ export const nativeMetadataRecoverySchema = z.strictObject({ format: z.literal("dotnet-nativeaot"), + /** Identifies parser-only facts without claiming Ghidra defined a DataType. */ + source: z.enum(["analysis-database", "read-only-derived-overlay"]).optional(), status: z.enum(["complete", "partial"]), header_address: address, format_major: z.number().int().nonnegative(), @@ -25,19 +27,29 @@ export const nativeMetadataRecoverySchema = z.strictObject({ basis: z.literal("method-table-pointer"), }), ), - derived_memory: z.strictObject({ - address, - size_bytes: z.number().int().nonnegative(), - sha256: z.string().regex(/^[a-f0-9]{64}$/u), - file_offset: z.null(), - }), + derived_memory: z + .strictObject({ + address, + size_bytes: z.number().int().nonnegative(), + sha256: z.string().regex(/^[a-f0-9]{64}$/u), + file_offset: z.null(), + }) + .nullable(), + truncated: z.boolean().optional(), + virtual_slots_omitted: z.number().int().nonnegative().optional(), + interfaces_omitted: z.number().int().nonnegative().optional(), + output_budget_bytes: z.number().int().nonnegative().optional(), diagnostics: z.array(z.string()), limitations: z.array(z.string()), }); +export type NativeMetadataRecovery = z.infer< + typeof nativeMetadataRecoverySchema +>; /** Discover a recovered runtime format and reusable type identities from the loaded image. */ export const nativeMetadataRecoverySummarySchema = z.strictObject({ format: z.literal("dotnet-nativeaot"), + analysis_mode: z.literal("read-only-derived-overlay").optional(), status: z.enum(["complete", "partial", "not_applicable"]), reason: z.string().nullable(), analysis_artifact_sha256: z.string().regex(/^[a-f0-9]{64}$/u), @@ -47,14 +59,76 @@ export const nativeMetadataRecoverySummarySchema = z.strictObject({ format_minor: z.number().int().nonnegative().nullable(), method_tables: z.number().int().nonnegative(), types: z.array(relatedType), - derived_memory: nativeMetadataRecoverySchema.shape.derived_memory.nullable(), + frozen_strings: z + .array(z.strictObject({ address, value: z.string() })) + .optional(), + truncated: z.boolean().optional(), + types_omitted: z.number().int().nonnegative().optional(), + frozen_strings_omitted: z.number().int().nonnegative().optional(), + output_budget_bytes: z.number().int().nonnegative().optional(), + derived_overlay: z + .strictObject({ + address, + size_bytes: z.number().int().nonnegative(), + sha256: z.string().regex(/^[a-f0-9]{64}$/u), + }) + .nullable() + .optional(), + derived_memory: nativeMetadataRecoverySchema.shape.derived_memory, coverage: z .strictObject({ - frozen_object_candidates: z.number().int().nonnegative(), - frozen_objects_annotated: z.number().int().nonnegative(), - basis: z.literal( + frozen_object_candidates: z.number().int().nonnegative().optional(), + frozen_objects_annotated: z.number().int().nonnegative().optional(), + pointer_candidates_examined: z.number().int().nonnegative().optional(), + pointer_slots_examined: z.number().int().nonnegative().optional(), + scan_bytes_examined: z.number().int().nonnegative().optional(), + scan_bytes_omitted: z.number().int().nonnegative().optional(), + rtr_signature_scan_bytes_examined: z + .number() + .int() + .nonnegative() + .optional(), + rtr_signature_scan_bytes_omitted: z + .number() + .int() + .nonnegative() + .optional(), + method_table_validation_items_examined: z + .number() + .int() + .nonnegative() + .optional(), + work_units_examined: z.number().int().nonnegative().optional(), + method_tables_recovered: z.number().int().nonnegative().optional(), + frozen_string_candidates: z.number().int().nonnegative().optional(), + frozen_scan_bytes_examined: z.number().int().nonnegative().optional(), + frozen_scan_bytes_omitted: z.number().int().nonnegative().optional(), + frozen_string_data_bytes_examined: z + .number() + .int() + .nonnegative() + .optional(), + source_bytes_examined: z.number().int().nonnegative().optional(), + source_bytes_omitted: z.number().int().nonnegative().optional(), + source_byte_budget_bytes: z.number().int().nonnegative().optional(), + work_budget_units: z.number().int().nonnegative().optional(), + working_memory_budget_bytes: z.number().int().nonnegative().optional(), + working_memory_bytes_peak: z.number().int().nonnegative().optional(), + truncation_reason: z + .enum([ + "source-byte-budget", + "work-unit-budget", + "working-memory-budget", + "response-budget", + ]) + .optional(), + basis: z.enum([ "rehydrated-pointer-candidates-and-committed-instance-types", - ), + "raw-image-pointer-scan", + "raw-image-and-rehydrated-pointer-scan", + "unsupported-layout", + "no-rtr-header", + ]), }) .nullable(), diagnostics: z.array(z.string()), diff --git a/src/ghidra/GhidraAnalysisProfile.ts b/src/ghidra/GhidraAnalysisProfile.ts index 4cd801ad0..fa5015dfc 100644 --- a/src/ghidra/GhidraAnalysisProfile.ts +++ b/src/ghidra/GhidraAnalysisProfile.ts @@ -17,6 +17,13 @@ import { ghidraMipsProfileParameters, ghidraMipsUnsupportedReason, } from "./GhidraMipsProfile.js"; +import { + NATIVE_AOT_MAX_REPORT_BYTES, + NATIVE_AOT_MAX_SOURCE_BYTES, + NATIVE_AOT_MAX_WORKING_MEMORY_BYTES, + NATIVE_AOT_MAX_WORK_UNITS, + NATIVE_AOT_PROFILE_CONTRACT_REVISION, +} from "../domain/native/nativeAotPe.js"; /** Resolve version-bound, deterministic semantics before Ghidra imports a target. */ export const resolveGhidraAnalysisProfile = ( @@ -69,6 +76,13 @@ export const resolveGhidraAnalysisProfile = ( ? { executable_role: target.executableRole ?? null, managed: target.managed ?? null, + native_aot_metadata: { + contract_revision: NATIVE_AOT_PROFILE_CONTRACT_REVISION, + source_bytes: NATIVE_AOT_MAX_SOURCE_BYTES, + work_units: NATIVE_AOT_MAX_WORK_UNITS, + working_memory_bytes: NATIVE_AOT_MAX_WORKING_MEMORY_BYTES, + report_bytes: NATIVE_AOT_MAX_REPORT_BYTES, + }, } : {}), architecture: target.architecture ?? null, diff --git a/src/ghidra/GhidraClient.ts b/src/ghidra/GhidraClient.ts index 1c1f262bd..51ea9f9b1 100644 --- a/src/ghidra/GhidraClient.ts +++ b/src/ghidra/GhidraClient.ts @@ -43,6 +43,21 @@ import { } from "./GhidraSessionValues.js"; import { createGhidraTargetSnapshot } from "./GhidraTargetSnapshot.js"; import { readFile } from "node:fs/promises"; +import { readBoundedFileBytes } from "../process/BoundedFileBytes.js"; +import { + openRegularFile, + RegularFileChangedError, + sameRegularFileState, +} from "../filesystem/RegularFile.js"; +import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; + +export type GhidraTargetSnapshot = + | { readonly kind: "captured"; readonly bytes: Buffer } + | { + readonly kind: "over-capacity"; + readonly sourceBytesAtLeast: number; + readonly maximumBytes: number; + }; import { connectGhidraSocket } from "./GhidraSocketConnection.js"; import { createGhidraEndpoint, @@ -79,6 +94,7 @@ export class GhidraClient { #socketRoot: PrivateRuntimeRoot | undefined; #endpointPath: string | undefined; #snapshotPath: string | undefined; + readonly #snapshotOwners = new Set(); #targetAdmission: JsonValue | undefined; #token: string | undefined; // Retain authentication identities for diagnostics from late request settlement. @@ -227,7 +243,10 @@ export class GhidraClient { } /** Read the private immutable source for independent import verification. */ - async readTargetSnapshot(): Promise> { + async readTargetSnapshot( + maximumBytes?: number, + signal?: AbortSignal, + ): Promise> { if (this.#snapshotPath === undefined) return err( this.#failure( @@ -236,11 +255,80 @@ export class GhidraClient { ), ); try { - return ok(await readFile(this.#snapshotPath)); + if (maximumBytes !== undefined) { + signal?.throwIfAborted(); + if (!Number.isSafeInteger(maximumBytes) || maximumBytes < 0) + throw new RangeError("Snapshot byte limit must be a safe integer"); + const handle = await openRegularFile(this.#snapshotPath, { + symlinks: "reject", + }); + const owner = new OwnedFileHandle(handle); + this.#snapshotOwners.add(owner); + let result: GhidraTargetSnapshot; + let readFailure: unknown; + try { + const metadata = await handle.stat(); + signal?.throwIfAborted(); + if (metadata.size > maximumBytes) + result = { + kind: "over-capacity", + sourceBytesAtLeast: metadata.size, + maximumBytes, + }; + else { + const bytes = await readBoundedFileBytes( + handle, + maximumBytes, + signal, + metadata.size, + ); + signal?.throwIfAborted(); + const after = await handle.stat(); + if ( + bytes !== undefined && + bytes.length === metadata.size && + sameRegularFileState(metadata, after) + ) { + result = { kind: "captured", bytes }; + } else if ( + bytes === undefined && + after.isFile() && + after.size > maximumBytes + ) { + result = { + kind: "over-capacity", + sourceBytesAtLeast: after.size, + maximumBytes, + }; + } else { + throw new RegularFileChangedError(this.#snapshotPath); + } + } + } catch (cause: unknown) { + readFailure = cause; + } + try { + await owner.close(); + this.#snapshotOwners.delete(owner); + } catch (closeFailure: unknown) { + throw new AggregateError( + readFailure === undefined + ? [closeFailure] + : [readFailure, closeFailure], + "Ghidra target snapshot read or descriptor cleanup failed", + ); + } + if (readFailure !== undefined) throw readFailure; + return ok(result!); + } + return ok({ + kind: "captured", + bytes: await readFile(this.#snapshotPath), + }); } catch (cause: unknown) { return err( this.#failure( - "protocol", + signal?.aborted === true ? "cancelled" : "protocol", "Ghidra target snapshot could not be read for load-image verification", cause, ), @@ -602,6 +690,10 @@ export class GhidraClient { this.#lastDiagnostics = this.#diagnostics(); this.#process = undefined; this.#launch = undefined; + for (const owner of this.#snapshotOwners) { + await owner.close(); + this.#snapshotOwners.delete(owner); + } try { await this.#removeRuntimeRoots(); } catch (cause: unknown) { diff --git a/src/ghidra/GhidraLoadImageAttest.ts b/src/ghidra/GhidraLoadImageAttest.ts index dd7aef655..956f570fa 100644 --- a/src/ghidra/GhidraLoadImageAttest.ts +++ b/src/ghidra/GhidraLoadImageAttest.ts @@ -1,4 +1,5 @@ import type { AnalysisOperation } from "../application/AnalysisProvider.js"; +import { AnalysisCancelledError } from "../domain/analysisErrorCore.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; import type { AnalysisError } from "../domain/analysisErrorBase.js"; import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; @@ -12,23 +13,91 @@ import { attestGhidraDosComLoadImage, attestGhidraDosLoadImage, } from "./GhidraLoadImageValues.js"; +import { type NativeAotPeResult } from "../domain/native/nativeAotPe.js"; +import type { GhidraTargetSnapshot } from "./GhidraClient.js"; import type { GhidraSessionError } from "./GhidraSessionError.js"; type LoadImageSnapshotClient = { - readonly readTargetSnapshot?: () => Promise< - Result - >; + readonly readTargetSnapshot?: ( + maximumBytes?: number, + signal?: AbortSignal, + ) => Promise>; }; +type NativeLoadImageAttestationInput = { + readonly target: BinaryTarget; + readonly operation: AnalysisOperation; + readonly measured: JsonValue; + readonly client: LoadImageSnapshotClient; + readonly mapSessionError: (failure: GhidraSessionError) => AnalysisError; + readonly nativeAotResult?: NativeAotPeResult | undefined; + readonly signal?: AbortSignal | undefined; +}; + +const isSignalAborted = (signal: AbortSignal | undefined): boolean => + signal?.aborted === true; + /** Independently verify measured load-image observations for admitted DOS targets. */ export const attestGhidraNativeLoadImage = async ( - target: BinaryTarget, - operation: AnalysisOperation, - measured: JsonValue, - client: LoadImageSnapshotClient, - mapSessionError: (failure: GhidraSessionError) => AnalysisError, + input: NativeLoadImageAttestationInput, ): Promise> => { + const { + target, + operation, + measured, + client, + mapSessionError, + nativeAotResult, + signal, + } = input; + if (isSignalAborted(signal)) + return err(new AnalysisCancelledError(operation)); const observations = nativeLoadImageObservationSchema.parse(measured); + if (target.format === "pe") { + if (nativeAotResult === undefined) + return err( + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason: + "The Ghidra provider did not produce read-only NativeAOT metadata for this admitted PE snapshot.", + }, + }), + ); + if (isSignalAborted(signal)) + return err(new AnalysisCancelledError(operation)); + try { + return ok( + jsonValueSchema.parse( + nativeLoadImageSchema.parse({ + status: "unsupported", + reason: + "Independent load-image verification currently supports DOS MZ and explicit DOS COM targets only.", + observations: { + ...observations, + metadata_recovery: [ + ...(observations.metadata_recovery ?? []), + nativeAotResult.summary, + ], + }, + limitations: [ + "Load-image verification is not performed for PE targets; NativeAOT metadata observations are derived separately from the exact admitted snapshot.", + ], + }), + ), + ); + } catch (cause: unknown) { + return err( + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason: + cause instanceof Error + ? cause.message + : "NativeAOT metadata could not be represented in the load-image result", + }, + }), + ); + } + } if (target.format !== "dos-mz" && target.format !== "dos-com") { return ok( jsonValueSchema.parse( @@ -55,11 +124,20 @@ export const attestGhidraNativeLoadImage = async ( ); const snapshot = await client.readTargetSnapshot(); if (!snapshot.ok) return err(mapSessionError(snapshot.error)); + if (snapshot.value.kind !== "captured") + return err( + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason: + "The uncapped DOS load-image snapshot was unexpectedly not captured.", + }, + }), + ); const attested = ( target.format === "dos-com" ? attestGhidraDosComLoadImage : attestGhidraDosLoadImage - )(snapshot.value, target.sha256 ?? "", observations); + )(snapshot.value.bytes, target.sha256 ?? "", observations); if (!attested.ok) return err( new ProviderAdapterError("ghidra", operation, { diff --git a/src/ghidra/GhidraProvider.behavior.test.ts b/src/ghidra/GhidraProvider.behavior.test.ts index 7686f837e..3dd108487 100644 --- a/src/ghidra/GhidraProvider.behavior.test.ts +++ b/src/ghidra/GhidraProvider.behavior.test.ts @@ -10,16 +10,22 @@ import { fileURLToPath } from "node:url"; import { promisify } from "node:util"; import { describe, expect, it } from "vitest"; import { fixtureDosLoadImage } from "./GhidraLoadImage.fixture.js"; +import { + nativeAotPeDigest, + nativeAotPeFixture, +} from "../../tests/fixtures/nativeaotPe.js"; import { jsonValueSchema } from "../domain/jsonValue.js"; import { parseConfig } from "../config/parseConfig.js"; import type { BinaryTarget } from "../domain/binaryTargetTypes.js"; import { parseExecutableHeader } from "../domain/binaryTarget.js"; import { GhidraProvider } from "./GhidraProvider.js"; +import type { GhidraTargetSnapshot } from "./GhidraClient.js"; import type { GhidraProviderClientFactory } from "./GhidraProviderClient.js"; import type { GhidraInstallationHost } from "./GhidraInstallation.js"; import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js"; import { err, ok } from "../domain/result.js"; +import type { Result } from "../domain/result.js"; import { GhidraSessionError } from "./GhidraSessionError.js"; import { silentLogger } from "../logger.js"; import { ProviderCleanupError } from "../domain/providerCleanupError.js"; @@ -219,10 +225,25 @@ describe("Ghidra platform support", () => { expect(profile.parameters).toMatchObject({ executable_role: target.executableRole, managed: target.managed, + native_aot_metadata: { + contract_revision: "rtr-9.1-x64-pe-read-only-v1", + source_bytes: 128 * 1024 * 1024, + work_units: 64 * 1024 * 1024, + working_memory_bytes: 64 * 1024 * 1024, + report_bytes: 4 * 1024 * 1024, + }, }); digests.add(profile.digest); } expect(digests.size).toBe(3); + const nonPe = await ghidra.resolveAnalysisProfile( + executableTarget("elf", "x86_64"), + ); + if (!nonPe.ok || nonPe.value.profile === null) + throw new Error("Expected a non-PE analysis profile"); + expect(nonPe.value.profile.parameters).not.toHaveProperty( + "native_aot_metadata", + ); }); it("keeps Windows annotation mutation unavailable independently of native controls", () => { @@ -717,7 +738,8 @@ describe("Ghidra measured load-image projection", () => { ...(state === "missing-snapshot" ? {} : { - readTargetSnapshot: () => Promise.resolve(ok(fixture.bytes)), + readTargetSnapshot: () => + Promise.resolve(ok({ kind: "captured", bytes: fixture.bytes })), }), }); const ghidra = provider(installationHost(), factory); @@ -757,6 +779,505 @@ describe("Ghidra measured load-image projection", () => { ); }); +it("returns read-only NativeAOT metadata inline and drills down by derived MethodTable address", async () => { + const bytes = nativeAotPeFixture(); + const observations = fixtureDosLoadImage().observation; + observations.image_base = "0x140000000"; + const factory: GhidraProviderClientFactory = () => ({ + start: () => + Promise.resolve( + ok({ + ...sessionInfo(), + target: { ...sessionInfo().target, image_base: "0x140000000" }, + }), + ), + callTool: (operation) => + Promise.resolve( + ok( + jsonValueSchema.parse( + operation === "inspect_native_load_image" + ? observations + : { + status: "unavailable", + reason: "No Ghidra DataType is defined at this address.", + id: null, + name: null, + kind: "unavailable", + source: "analysis-database", + source_archive: null, + address: null, + size_bytes: null, + alignment_bytes: null, + packing_enabled: null, + referenced_type: null, + array_count: null, + array_stride_bytes: null, + fields: [], + members: [], + total_fields: 0, + truncated: false, + limitations: [], + }, + ), + ), + ), + readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })), + close: () => Promise.resolve(ok(null)), + }); + const ghidra = provider(installationHost(), factory); + const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) }; + const profile = await ghidra.resolveAnalysisProfile(target); + if (!profile.ok || profile.value.profile === null) + throw new Error("Expected admitted PE profile"); + const client = ghidra.createClient(target, profile.value.profile); + const image = await client.execute("inspect_native_load_image", {}); + expect(image).toMatchObject({ + ok: true, + value: { + result: { + status: "unsupported", + observations: { + metadata_recovery: [ + { + status: "complete", + analysis_mode: "read-only-derived-overlay", + method_tables: 3, + frozen_strings: [{ value: "REA_NATIVEAOT_FROZEN" }], + }, + ], + }, + }, + rawResult: observations, + }, + }); + const detail = await client.execute("inspect_native_data_type", { + address: "0x140002280", + }); + expect(detail).toMatchObject({ + ok: true, + value: { + result: { + status: "unavailable", + source: "analysis-database", + metadata_recovery: { + source: "read-only-derived-overlay", + method_table_address: "0x140002280", + related_type: { address: "0x140002200", type: null }, + }, + }, + }, + }); + await client.close(); +}); + +it("derives PE metadata for a direct type-address query without a load-image call", async () => { + const bytes = nativeAotPeFixture(); + const operations: string[] = []; + const factory: GhidraProviderClientFactory = () => ({ + start: () => + Promise.resolve( + ok({ + ...sessionInfo(), + target: { ...sessionInfo().target, image_base: "0x140000000" }, + }), + ), + callTool: (operation) => { + operations.push(operation); + return Promise.resolve( + ok( + jsonValueSchema.parse({ + status: "unavailable", + reason: "No Ghidra DataType is defined at this address.", + id: null, + name: null, + kind: "unavailable", + source: "analysis-database", + source_archive: null, + address: null, + size_bytes: null, + alignment_bytes: null, + packing_enabled: null, + referenced_type: null, + array_count: null, + array_stride_bytes: null, + fields: [], + members: [], + total_fields: 0, + truncated: false, + limitations: [], + }), + ), + ); + }, + readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })), + close: () => Promise.resolve(ok(null)), + }); + const ghidra = provider(installationHost(), factory); + const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) }; + const profile = await ghidra.resolveAnalysisProfile(target); + if (!profile.ok || profile.value.profile === null) + throw new Error("Expected admitted PE profile"); + const client = ghidra.createClient(target, profile.value.profile); + const detail = await client.execute("inspect_native_data_type", { + address: "0x140002280", + }); + expect(detail).toMatchObject({ + ok: true, + value: { + result: { + metadata_recovery: { + source: "read-only-derived-overlay", + method_table_address: "0x140002280", + related_type: { address: "0x140002200", type: null }, + }, + }, + }, + }); + expect(operations).toEqual(["inspect_native_data_type"]); + await client.close(); +}); + +it("shares one session snapshot across callers while isolating cancellation and rejecting a changed base", async () => { + const bytes = nativeAotPeFixture(); + const baseObservation = fixtureDosLoadImage().observation; + baseObservation.image_base = "0x140000000"; + let calls = 0; + let snapshotReads = 0; + let snapshotSignal: AbortSignal | undefined; + let releaseSnapshot: + | ((value: Result) => void) + | undefined; + let markSnapshotStarted: (() => void) | undefined; + const snapshotStarted = new Promise((resolve) => { + markSnapshotStarted = resolve; + }); + const snapshotResult = new Promise< + Result + >((resolve) => { + releaseSnapshot = resolve; + }); + const factory: GhidraProviderClientFactory = () => ({ + start: () => Promise.resolve(ok(sessionInfo())), + callTool: (operation) => { + calls += 1; + const observation = { + ...baseObservation, + source_files: [ + { ...baseObservation.source_files[0]!, name: `caller-${calls}` }, + ], + }; + return Promise.resolve( + ok( + jsonValueSchema.parse( + operation === "inspect_native_load_image" + ? observation + : { + status: "unavailable", + reason: "No Ghidra DataType is defined at this address.", + id: null, + name: null, + kind: "unavailable", + source: "analysis-database", + source_archive: null, + address: null, + size_bytes: null, + alignment_bytes: null, + packing_enabled: null, + referenced_type: null, + array_count: null, + array_stride_bytes: null, + fields: [], + members: [], + total_fields: 0, + truncated: false, + limitations: [], + }, + ), + ), + ); + }, + readTargetSnapshot: (_maximumBytes, signal) => { + snapshotReads += 1; + snapshotSignal = signal; + markSnapshotStarted?.(); + return snapshotResult; + }, + close: () => Promise.resolve(ok(null)), + }); + const ghidra = provider(installationHost(), factory); + const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) }; + const profile = await ghidra.resolveAnalysisProfile(target); + if (!profile.ok || profile.value.profile === null) + throw new Error("Expected admitted PE profile"); + const client = ghidra.createClient(target, profile.value.profile); + const firstCaller = new AbortController(); + const secondCaller = new AbortController(); + const first = client.execute( + "inspect_native_load_image", + {}, + { + signal: firstCaller.signal, + }, + ); + const second = client.execute( + "inspect_native_load_image", + {}, + { + signal: secondCaller.signal, + }, + ); + await snapshotStarted; + expect(snapshotReads).toBe(1); + expect(snapshotSignal).toBeDefined(); + expect(snapshotSignal).not.toBe(firstCaller.signal); + expect(snapshotSignal).not.toBe(secondCaller.signal); + firstCaller.abort(); + await expect(first).resolves.toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError" }, + }); + releaseSnapshot?.(ok({ kind: "captured", bytes })); + const secondResult = await second; + expect(secondResult).toMatchObject({ + ok: true, + value: { + rawResult: { + source_files: [{ name: "caller-2" }], + }, + result: { + observations: { + source_files: [{ name: "caller-2" }], + }, + }, + }, + }); + expect(snapshotReads).toBe(1); + expect(calls).toBe(2); + + baseObservation.image_base = "0x150000000"; + const conflictingBase = await client.execute("inspect_native_load_image", {}); + expect(conflictingBase).toMatchObject({ + ok: false, + error: { + _tag: "ProviderAdapterError", + diagnostics: { + reason: expect.stringContaining("different loaded image base"), + }, + }, + }); + expect(snapshotReads).toBe(1); + await client.close(); + expect(snapshotSignal?.aborted).toBe(true); +}); + +it("retries a failed NativeAOT snapshot read without retaining a failed derivation", async () => { + const bytes = nativeAotPeFixture(); + const observations = fixtureDosLoadImage().observation; + observations.image_base = "0x140000000"; + let snapshotReads = 0; + const factory: GhidraProviderClientFactory = () => ({ + start: () => Promise.resolve(ok(sessionInfo())), + callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))), + readTargetSnapshot: () => { + snapshotReads += 1; + return Promise.resolve( + snapshotReads === 1 + ? err( + new GhidraSessionError( + "process", + "temporary snapshot read failure", + ), + ) + : ok({ kind: "captured", bytes }), + ); + }, + close: () => Promise.resolve(ok(null)), + }); + const ghidra = provider(installationHost(), factory); + const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) }; + const profile = await ghidra.resolveAnalysisProfile(target); + if (!profile.ok || profile.value.profile === null) + throw new Error("Expected admitted PE profile"); + const client = ghidra.createClient(target, profile.value.profile); + await expect( + client.execute("inspect_native_load_image", {}), + ).resolves.toMatchObject({ + ok: false, + error: { _tag: "ProviderAdapterError" }, + }); + await expect( + client.execute("inspect_native_load_image", {}), + ).resolves.toMatchObject({ + ok: true, + value: { + result: { + observations: { + metadata_recovery: [{ status: "complete" }], + }, + }, + }, + }); + expect(snapshotReads).toBe(2); + await client.close(); +}); + +it("rejects all waiters when the loaded image base changes during derivation", async () => { + const bytes = nativeAotPeFixture(); + const observation = fixtureDosLoadImage().observation; + let calls = 0; + let resolveSnapshot: + | ((value: Result) => void) + | undefined; + let markSnapshotStarted: (() => void) | undefined; + const snapshotStarted = new Promise((resolve) => { + markSnapshotStarted = resolve; + }); + const snapshot = new Promise< + Result + >((resolve) => { + resolveSnapshot = resolve; + }); + const factory: GhidraProviderClientFactory = () => ({ + start: () => Promise.resolve(ok(sessionInfo())), + callTool: () => { + calls += 1; + return Promise.resolve( + ok( + jsonValueSchema.parse({ + ...observation, + image_base: calls === 1 ? "0x140000000" : "0x150000000", + }), + ), + ); + }, + readTargetSnapshot: () => { + markSnapshotStarted?.(); + return snapshot; + }, + close: () => Promise.resolve(ok(null)), + }); + const ghidra = provider(installationHost(), factory); + const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) }; + const profile = await ghidra.resolveAnalysisProfile(target); + if (!profile.ok || profile.value.profile === null) + throw new Error("Expected admitted PE profile"); + const client = ghidra.createClient(target, profile.value.profile); + const originalBase = client.execute("inspect_native_load_image", {}); + await snapshotStarted; + const changedBase = await client.execute("inspect_native_load_image", {}); + expect(changedBase).toMatchObject({ + ok: false, + error: { + _tag: "ProviderAdapterError", + diagnostics: { + observed_image_base: "0x150000000", + measured_observation: { image_base: "0x150000000" }, + }, + }, + }); + resolveSnapshot?.(ok({ kind: "captured", bytes })); + await expect(originalBase).resolves.toMatchObject({ + ok: false, + error: { + _tag: "ProviderAdapterError", + diagnostics: { + reason: expect.stringContaining("conflicting loaded image bases"), + measured_observation: { image_base: "0x140000000" }, + conflicting_observation: { image_base: "0x150000000" }, + }, + }, + }); + expect(calls).toBe(2); + await client.close(); +}); + +it("cancels an in-flight NativeAOT waiter when its owning session closes", async () => { + const bytes = nativeAotPeFixture(); + const observations = fixtureDosLoadImage().observation; + observations.image_base = "0x140000000"; + let resolveSnapshot: + | ((value: Result) => void) + | undefined; + let snapshotSignal: AbortSignal | undefined; + let markSnapshotStarted: (() => void) | undefined; + const snapshotStarted = new Promise((resolve) => { + markSnapshotStarted = resolve; + }); + const snapshot = new Promise< + Result + >((resolve) => { + resolveSnapshot = resolve; + }); + const factory: GhidraProviderClientFactory = () => ({ + start: () => Promise.resolve(ok(sessionInfo())), + callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))), + readTargetSnapshot: (_maximumBytes, signal) => { + snapshotSignal = signal; + markSnapshotStarted?.(); + return snapshot; + }, + close: () => Promise.resolve(ok(null)), + }); + const ghidra = provider(installationHost(), factory); + const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) }; + const profile = await ghidra.resolveAnalysisProfile(target); + if (!profile.ok || profile.value.profile === null) + throw new Error("Expected admitted PE profile"); + const client = ghidra.createClient(target, profile.value.profile); + const pending = client.execute("inspect_native_load_image", {}); + await snapshotStarted; + await client.close(); + expect(snapshotSignal?.aborted).toBe(true); + resolveSnapshot?.(ok({ kind: "captured", bytes })); + await expect(pending).resolves.toMatchObject({ + ok: false, + error: { _tag: "AnalysisCancelledError" }, + }); +}); + +describe("Ghidra NativeAOT snapshot budget", () => { + it("reports snapshot-capacity omissions inline through the load-image result", async () => { + const observations = fixtureDosLoadImage().observation; + const target = peTarget("x86_64"); + const factory: GhidraProviderClientFactory = () => ({ + start: () => Promise.resolve(ok(sessionInfo())), + callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))), + readTargetSnapshot: () => + Promise.resolve( + ok({ + kind: "over-capacity", + sourceBytesAtLeast: 128 * 1024 * 1024 + 1, + maximumBytes: 128 * 1024 * 1024, + }), + ), + close: () => Promise.resolve(ok(null)), + }); + const ghidra = provider(installationHost(), factory); + const profile = await ghidra.resolveAnalysisProfile(target); + if (!profile.ok || profile.value.profile === null) + throw new Error("Expected admitted PE profile"); + const client = ghidra.createClient(target, profile.value.profile); + const image = await client.execute("inspect_native_load_image", {}); + expect(image).toMatchObject({ + ok: true, + value: { + result: { + observations: { + metadata_recovery: [ + { + status: "partial", + truncated: true, + coverage: { truncation_reason: "source-byte-budget" }, + }, + ], + }, + }, + }, + }); + await client.close(); + }); +}); + const sessionInfo = () => ({ name: "REA Ghidra bridge" as const, run_id: "11111111-1111-4111-8111-111111111111", diff --git a/src/ghidra/GhidraProviderClient.ts b/src/ghidra/GhidraProviderClient.ts index 49f66a10d..f024ab422 100644 --- a/src/ghidra/GhidraProviderClient.ts +++ b/src/ghidra/GhidraProviderClient.ts @@ -8,7 +8,6 @@ import { type AnalysisOperation, } from "../application/AnalysisProvider.js"; import type { AppConfig } from "../config/types.js"; -import type { JsonValue } from "../domain/jsonValue.js"; import { ghidraSeedCommitmentSchema, ghidraSeedFailure, @@ -30,6 +29,7 @@ import { AnalysisTimeoutError, } from "../domain/analysisErrorCore.js"; import { type AnalysisError } from "../domain/analysisErrorBase.js"; +import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js"; import { ProviderAdapterError } from "../domain/providerAdapterError.js"; import { err, ok, type Result } from "../domain/result.js"; import type { Logger } from "pino"; @@ -52,6 +52,18 @@ import { import { unverifiedGhidraBuildLimitation } from "./GhidraInstallationPolicy.js"; import { GhidraHeadlessLauncher } from "./GhidraLauncher.js"; import { attestGhidraNativeLoadImage } from "./GhidraLoadImageAttest.js"; +import { + NATIVE_AOT_MAX_REPORT_BYTES, + NATIVE_AOT_MAX_SOURCE_BYTES, + nativeAotSnapshotOverCapacity, + parseNativeAotPe, + type NativeAotPeResult, +} from "../domain/native/nativeAotPe.js"; +import { + ABORTED, + waitForAbortable, +} from "../application/binary/AbortablePromise.js"; +import { nativeLoadImageObservationSchema } from "../domain/native/nativeLoadImage.js"; import { GHIDRA_PROVIDER_IDENTITY, healthLimitations, @@ -77,6 +89,250 @@ export type GhidraProviderClientFactory = ( ) => Pick & Partial>; +type GhidraProviderClientInstance = ReturnType; + +type NativeAotSnapshotOwner = { + readonly derive: ( + imageBase: string, + operation: AnalysisOperation, + callerSignal: AbortSignal | undefined, + observations: JsonValue, + ) => Promise>; + readonly readTypeDetail: ( + address: string, + ) => ReturnType; + readonly close: () => void; +}; + +/** Own one session's captured PE snapshot and immutable NativeAOT derivation. */ +const createNativeAotSnapshotOwner = (input: { + readonly client: GhidraProviderClientInstance; + readonly target: BinaryTarget; + readonly startupTimeoutMs: number; +}): NativeAotSnapshotOwner => { + const { client, target, startupTimeoutMs } = input; + const controller = new AbortController(); + let identity: string | undefined; + let result: NativeAotPeResult | undefined; + let pending: + | { + readonly identity: string; + readonly promise: Promise>; + } + | undefined; + let closed = false; + let identityConflict = false; + let identityConflictObservation: + | { readonly imageBase: string; readonly observations: JsonValue } + | undefined; + const identityError = ( + operation: AnalysisOperation, + reason: string, + observedBase: string, + observations: JsonValue, + ): AnalysisError => + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason, + artifact_sha256: target.sha256, + bound_image_base: + identity === undefined + ? null + : `0x${identity.slice(target.sha256.length + 1)}`, + observed_image_base: observedBase, + measured_observation: observations, + conflicting_image_base: identityConflictObservation?.imageBase ?? null, + conflicting_observation: + identityConflictObservation?.observations ?? null, + }, + }); + const identityConflictFailure = ( + operation: AnalysisOperation, + observedBase: string, + observations: JsonValue, + ): Result => { + return err( + identityError( + operation, + "The Ghidra session reported conflicting loaded image bases; reopen the analysis session to establish a new identity.", + observedBase, + observations, + ), + ); + }; + const derive = async ( + imageBase: string, + operation: AnalysisOperation, + callerSignal: AbortSignal | undefined, + observations: JsonValue, + ): Promise> => { + if (callerSignal?.aborted || closed) + return err(new AnalysisCancelledError(operation)); + if (identityConflict) + return identityConflictFailure(operation, imageBase, observations); + if (controller.signal.aborted) + return err(new AnalysisCancelledError(operation)); + let normalizedBase: string; + try { + normalizedBase = BigInt(imageBase).toString(16); + } catch (cause: unknown) { + return err( + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason: + "Ghidra returned an invalid loaded image base for NativeAOT identity binding.", + artifact_sha256: target.sha256, + observed_image_base: imageBase, + measured_observation: observations, + }, + cause, + }), + ); + } + const key = `${target.sha256}:${normalizedBase}`; + if (identity !== undefined && identity !== key) { + identityConflict = true; + identityConflictObservation = { imageBase, observations }; + result = undefined; + controller.abort(); + return err( + identityError( + operation, + "The Ghidra session reported a different loaded image base after NativeAOT metadata was bound to its captured snapshot; reopen the analysis session to establish a new identity.", + imageBase, + observations, + ), + ); + } + identity ??= key; + if (result !== undefined) { + const settled = await waitForAbortable( + Promise.resolve(ok(result)), + callerSignal, + ); + if (closed) return err(new AnalysisCancelledError(operation)); + if (identityConflict) + return identityConflictFailure(operation, imageBase, observations); + if (controller.signal.aborted) + return err(new AnalysisCancelledError(operation)); + return settled === ABORTED + ? err(new AnalysisCancelledError(operation)) + : settled; + } + if (pending !== undefined && pending.identity !== key) + return err( + identityError( + operation, + "A NativeAOT snapshot derivation is already bound to a different loaded image base.", + imageBase, + observations, + ), + ); + if (pending === undefined) { + const promise: Promise> = + (async () => { + try { + if (client.readTargetSnapshot === undefined) + return err( + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason: + "The Ghidra client does not expose its immutable target snapshot for NativeAOT metadata inspection.", + }, + }), + ); + const snapshot = await client.readTargetSnapshot( + NATIVE_AOT_MAX_SOURCE_BYTES, + controller.signal, + ); + if (!snapshot.ok) + return err( + projectSessionError( + operation, + snapshot.error, + startupTimeoutMs, + ), + ); + if (closed) return err(new AnalysisCancelledError(operation)); + if (identityConflict) + return identityConflictFailure( + operation, + imageBase, + observations, + ); + if (controller.signal.aborted) + return err(new AnalysisCancelledError(operation)); + const parsed = + snapshot.value.kind === "over-capacity" + ? { + summary: nativeAotSnapshotOverCapacity( + target.sha256, + snapshot.value.sourceBytesAtLeast, + NATIVE_AOT_MAX_REPORT_BYTES, + ), + readTypeDetail: () => undefined, + } + : parseNativeAotPe( + snapshot.value.bytes, + target.sha256, + NATIVE_AOT_MAX_REPORT_BYTES, + imageBase, + ); + if (closed) return err(new AnalysisCancelledError(operation)); + if (identityConflict) + return identityConflictFailure( + operation, + imageBase, + observations, + ); + if (controller.signal.aborted) + return err(new AnalysisCancelledError(operation)); + return ok(parsed); + } catch (cause: unknown) { + return err( + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason: + cause instanceof Error + ? cause.message + : "NativeAOT metadata could not be derived from the captured target snapshot.", + }, + cause, + }), + ); + } + })(); + const current = { identity: key, promise }; + pending = current; + void promise.then((settled) => { + if (pending !== current) return; + pending = undefined; + if (settled.ok && !closed && !identityConflict) result = settled.value; + }); + } + const settled = await waitForAbortable(pending.promise, callerSignal); + if (closed) return err(new AnalysisCancelledError(operation)); + if (identityConflict) + return identityConflictFailure(operation, imageBase, observations); + if (controller.signal.aborted) + return err(new AnalysisCancelledError(operation)); + return settled === ABORTED + ? err(new AnalysisCancelledError(operation)) + : settled; + }; + return { + derive, + readTypeDetail: (address) => + result?.readTypeDetail(address, NATIVE_AOT_MAX_REPORT_BYTES), + close: () => { + closed = true; + controller.abort(); + result = undefined; + pending = undefined; + }, + }; +}; + /** Build one AnalysisClient for an admitted Ghidra target and profile. */ export const createGhidraProviderClient = (input: { readonly config: AppConfig; @@ -258,6 +514,14 @@ export const createGhidraProviderClient = (input: { ...seedLimitations, ...(releaseLimitation === undefined ? [] : [releaseLimitation]), ]; + const nativeAotOwner = + target.format === "pe" + ? createNativeAotSnapshotOwner({ + client, + target, + startupTimeoutMs: config.ghidraStartupTimeoutMs, + }) + : undefined; return { execute: async (operation, parameters, options) => { if (extensionFailure !== undefined) return err(extensionFailure); @@ -334,19 +598,83 @@ export const createGhidraProviderClient = (input: { if (!result.ok) return result; normalized = result.value; } + if (operation === "inspect_native_data_type") { + const address = input.value.address; + if (typeof address === "string") { + if (target.format === "pe" && nativeAotOwner !== undefined) { + const started = await client.start(options?.signal); + if (!started.ok) + return err( + projectSessionError( + operation, + started.error, + config.ghidraStartupTimeoutMs, + ), + ); + const derived = await nativeAotOwner.derive( + started.value.target.image_base, + operation, + options?.signal, + normalized, + ); + if (!derived.ok) return derived; + } + let metadata: + | ReturnType + | undefined; + metadata = nativeAotOwner?.readTypeDetail(address); + if ( + metadata !== undefined && + normalized !== null && + typeof normalized === "object" && + !Array.isArray(normalized) + ) { + const nativeResult = normalized as Record; + if (nativeResult.metadata_recovery === undefined) + normalized = jsonValueSchema.parse({ + ...nativeResult, + metadata_recovery: metadata, + }); + } + } + } if (operation === "inspect_native_load_image") { - const attested = await attestGhidraNativeLoadImage( + let derived: NativeAotPeResult | undefined; + if (target.format === "pe") { + const observations = + nativeLoadImageObservationSchema.parse(normalized); + if (nativeAotOwner === undefined) + return err( + new ProviderAdapterError("ghidra", operation, { + diagnostics: { + reason: + "The admitted PE session has no read-only NativeAOT snapshot owner.", + }, + }), + ); + const result = await nativeAotOwner.derive( + observations.image_base, + operation, + options?.signal, + normalized, + ); + if (!result.ok) return result; + derived = result.value; + } + const attested = await attestGhidraNativeLoadImage({ target, operation, - normalized, + measured: normalized, client, - (failure) => + mapSessionError: (failure) => projectSessionError( operation, failure, config.ghidraStartupTimeoutMs, ), - ); + nativeAotResult: derived, + signal: options?.signal, + }); if (!attested.ok) return attested; normalized = attested.value; } @@ -368,7 +696,10 @@ export const createGhidraProviderClient = (input: { ? [] : [{ provider: committedProfile.provider, observation }]; }, - close: () => client.close(), + close: async () => { + nativeAotOwner?.close(); + return client.close(); + }, }; }; diff --git a/src/process/BoundedFileBytes.ts b/src/process/BoundedFileBytes.ts index 90b6a41a6..a9d6f33a5 100644 --- a/src/process/BoundedFileBytes.ts +++ b/src/process/BoundedFileBytes.ts @@ -1,27 +1,53 @@ import type { FileHandle } from "node:fs/promises"; -/** Read through EOF, returning undefined on overflow; consume at most maxBytes + 1 bytes without closing the handle. */ +/** Read through EOF, returning undefined on overflow; leave the handle open. */ export const readBoundedFileBytes = async ( handle: FileHandle, maxBytes: number, signal?: AbortSignal, + expectedSize?: number, ): Promise => { if (!Number.isSafeInteger(maxBytes) || maxBytes < 0) throw new RangeError("File byte limit must be a nonnegative safe integer"); - const chunks: Buffer[] = []; - const buffer = Buffer.allocUnsafe(Math.min(64 * 1_024, maxBytes + 1)); + if ( + expectedSize !== undefined && + (!Number.isSafeInteger(expectedSize) || + expectedSize < 0 || + expectedSize > maxBytes) + ) + throw new RangeError("Expected file size must fit the byte limit"); + if (expectedSize === undefined) { + const chunks: Buffer[] = []; + const buffer = Buffer.allocUnsafe(Math.min(64 * 1_024, maxBytes + 1)); + let total = 0; + for (;;) { + signal?.throwIfAborted(); + const { bytesRead } = await handle.read( + buffer, + 0, + Math.min(buffer.length, maxBytes - total + 1), + null, + ); + if (bytesRead === 0) return Buffer.concat(chunks, total); + total += bytesRead; + if (total > maxBytes) return undefined; + chunks.push(Buffer.from(buffer.subarray(0, bytesRead))); + } + } + const buffer = Buffer.allocUnsafe(expectedSize); + if (expectedSize === 0) return buffer; let total = 0; for (;;) { signal?.throwIfAborted(); const { bytesRead } = await handle.read( buffer, - 0, - Math.min(buffer.length, maxBytes - total + 1), + total, + expectedSize - total, null, ); - if (bytesRead === 0) return Buffer.concat(chunks, total); + if (bytesRead === 0) return buffer.subarray(0, total); total += bytesRead; if (total > maxBytes) return undefined; - chunks.push(Buffer.from(buffer.subarray(0, bytesRead))); + if (total === expectedSize) return buffer; } }; diff --git a/tests/boundary/providers/ghidra/ghidraClient.test.ts b/tests/boundary/providers/ghidra/ghidraClient.test.ts index bbc851c2b..0e84f1769 100644 --- a/tests/boundary/providers/ghidra/ghidraClient.test.ts +++ b/tests/boundary/providers/ghidra/ghidraClient.test.ts @@ -146,7 +146,7 @@ afterEach(async () => { await Promise.all(clients.splice(0).map((client) => client.close())); }); -describe("GhidraClient", () => { +describe("GhidraClient captured source snapshot", () => { it("exposes the immutable import snapshot only during the live session", async () => { const client = clientFor(new FixtureLauncher()); await expect(client.readTargetSnapshot()).resolves.toMatchObject({ @@ -154,16 +154,43 @@ describe("GhidraClient", () => { error: { kind: "protocol" }, }); await expect(client.start()).resolves.toMatchObject({ ok: true }); + const abortedRead = new AbortController(); + abortedRead.abort(); + await expect( + client.readTargetSnapshot(1024, abortedRead.signal), + ).resolves.toMatchObject({ ok: false, error: { kind: "cancelled" } }); const snapshot = await client.readTargetSnapshot(); expect(snapshot.ok).toBe(true); - if (snapshot.ok) expect(snapshot.value).toEqual(readFileSync(fixturePath)); + const expected = readFileSync(fixturePath); + if (snapshot.ok) + expect(snapshot.value).toEqual({ + kind: "captured", + bytes: expected, + }); + await expect( + client.readTargetSnapshot(expected.length), + ).resolves.toMatchObject({ + ok: true, + value: { kind: "captured", bytes: expected }, + }); + const bounded = await client.readTargetSnapshot(0); + expect(bounded).toMatchObject({ + ok: true, + value: { + kind: "over-capacity", + maximumBytes: 0, + sourceBytesAtLeast: readFileSync(fixturePath).length, + }, + }); await client.close(); await expect(client.readTargetSnapshot()).resolves.toMatchObject({ ok: false, error: { kind: "protocol" }, }); }); +}); +describe("GhidraClient", () => { it("completes an exact, fragmented post-analysis handshake", async () => { const launcher = new FixtureLauncher("fragmented"); const client = clientFor(launcher); diff --git a/tests/fixtures/nativeaotPe.ts b/tests/fixtures/nativeaotPe.ts new file mode 100644 index 000000000..6c6e94ed6 --- /dev/null +++ b/tests/fixtures/nativeaotPe.ts @@ -0,0 +1,91 @@ +import { createHash } from "node:crypto"; + +const IMAGE_BASE = 0x140000000n; +const address = (rva: number): bigint => IMAGE_BASE + BigInt(rva); + +export const nativeAotPeDigest = (bytes: Buffer): string => + createHash("sha256").update(bytes).digest("hex"); + +export const nativeAotPeFixture = (major = 9, minor = 1): Buffer => { + const bytes = Buffer.alloc(0xa00); + bytes.write("MZ", 0, "ascii"); + bytes.writeUInt32LE(0x80, 0x3c); + bytes.write("PE\0\0", 0x80, "ascii"); + bytes.writeUInt16LE(0x8664, 0x84); + bytes.writeUInt16LE(2, 0x86); + bytes.writeUInt16LE(0xf0, 0x94); + const optional = 0x98; + bytes.writeUInt16LE(0x20b, optional); + bytes.writeBigUInt64LE(IMAGE_BASE, optional + 24); + bytes.writeUInt32LE(0x3000, optional + 56); + bytes.writeUInt32LE(0x400, optional + 60); + const sectionTable = optional + 0xf0; + bytes.write(".text", sectionTable, "ascii"); + bytes.writeUInt32LE(0x200, sectionTable + 8); + bytes.writeUInt32LE(0x1000, sectionTable + 12); + bytes.writeUInt32LE(0x200, sectionTable + 16); + bytes.writeUInt32LE(0x400, sectionTable + 20); + bytes.writeUInt32LE(0x60000020, sectionTable + 36); + const data = sectionTable + 40; + bytes.write(".rdata", data, "ascii"); + bytes.writeUInt32LE(0x400, data + 8); + bytes.writeUInt32LE(0x2000, data + 12); + bytes.writeUInt32LE(0x400, data + 16); + bytes.writeUInt32LE(0x600, data + 20); + bytes.writeUInt32LE(0x40000040, data + 36); + + bytes.writeUInt8(0xc3, 0x410); + bytes.writeUInt8(0xc3, 0x420); + bytes.writeUInt8(0xc3, 0x430); + const header = 0x600; + bytes.writeUInt32LE(0x00525452, header); + bytes.writeUInt16LE(major, header + 4); + bytes.writeUInt16LE(minor, header + 6); + bytes.writeUInt16LE(2, header + 12); + bytes.writeUInt8(24, header + 14); + bytes.writeUInt8(1, header + 15); + bytes.writeUInt32LE(207, header + 16); + bytes.writeUInt32LE(1, header + 20); + bytes.writeBigUInt64LE(address(0x2100), header + 24); + bytes.writeBigUInt64LE(address(0x2105), header + 32); + bytes.writeUInt32LE(206, header + 40); + bytes.writeBigUInt64LE(address(0x2300), header + 48); + bytes.writeBigUInt64LE(address(0x2380), header + 56); + bytes.writeInt32LE(0x280, 0x700); + bytes.writeUInt8(0x09, 0x704); + + const writeMethodTable = ( + rva: number, + flags: number, + baseSize: number, + related: bigint, + slots: readonly bigint[], + ) => { + const offset = 0x600 + rva - 0x2000; + bytes.writeUInt32LE(flags, offset); + bytes.writeUInt32LE(baseSize, offset + 4); + bytes.writeBigUInt64LE(related, offset + 8); + bytes.writeUInt16LE(slots.length, offset + 16); + bytes.writeUInt16LE(0, offset + 18); + bytes.writeUInt32LE(0x1234, offset + 20); + slots.forEach((slot, index) => + bytes.writeBigUInt64LE(slot, offset + 24 + index * 8), + ); + }; + const object = address(0x2200); + writeMethodTable(0x2200, 0x50000000, 0x18, 0n, [ + address(0x1010), + address(0x1020), + address(0x1030), + ]); + writeMethodTable(0x2280, 0x50000000, 0x20, object, [address(0x1040)]); + writeMethodTable(0x22c0, 0x50000000, 0x16, object, []); + + const frozen = 0x600 + 0x2300 - 0x2000; + bytes.writeBigUInt64LE(address(0x22c0), frozen); + const value = Buffer.from("REA_NATIVEAOT_FROZEN", "utf16le"); + bytes.writeUInt32LE(value.length / 2, frozen + 8); + value.copy(bytes, frozen + 12); + bytes.writeUInt16LE(0, frozen + 12 + value.length); + return bytes; +};