feat(ghidra): inspect NativeAOT PE metadata read-only

This commit is contained in:
morluto
2026-10-11 10:31:49 +08:00
parent fe574c0c9f
commit 76d8f68cb9
18 changed files with 3484 additions and 81 deletions
+64 -35
View File
@@ -1,16 +1,25 @@
# Optional NativeAOT metadata recovery
REA integrates [Washi1337/ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot)
through an optional headless adapter. The original source is an opt-in submodule,
pinned to `effeb734fc570c32650f88b159608979dc7b423e`; its MIT license and existing
source notices are retained. No upstream source or algorithms are rewritten.
REA has a built-in read-only snapshot parser for x64 PE targets with .NET 8 RTR
9.1 metadata. It parses the exact admitted Ghidra session snapshot and uses an
in-memory overlay for rehydrated metadata; it does not define Ghidra DataTypes,
labels, functions, or memory bytes. The hydration semantics are adapted from
[Washi1337/ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot) at
`effeb734fc570c32650f88b159608979dc7b423e` (MIT), cross-checked against
.NET's `MethodTable.h` at runtime commit
`a3b2d40328be0be3f8dd43f0e5cc925b8827b044` (MIT). The required notice ships in
[`licenses/nativeaot-parser-MIT.txt`](../licenses/nativeaot-parser-MIT.txt).
The initial tested layout is .NET runtime **8.0.22**, NativeAOT RTR **9.1**,
x86-64 ELF and native Windows PE **targets analyzed on Linux x64**, using
Ghidra **12.1.4** and JDK **21**. The build accepts any Ghidra 12.1.x release and
javac 21 or newer, and it compiles with `--release 21`. Real verification of
this layout remains the Ghidra 12.1.4 and JDK 21 pair. Other NativeAOT layouts,
architectures, and hosts are unsupported.
The optional headless Ghidra adapter remains available for database annotations
and broader recovery. It uses the same pinned upstream commit as an opt-in
submodule and is a separate, more capable workflow.
The parser currently handles x64 PE32+ and RTR 9.1, including bounded
DEHYDRATED_DATA processing, MethodTable relationships, virtual slots, and
frozen strings. Other RTR versions and PE architectures are reported as
unsupported or partial. The matching .NET 8.0.22 fixture is source-built and
the parser's Windows-host Ghidra workflow has not yet had a real Windows-host
run. This does not establish Windows Ghidra import or decompiler behavior.
Windows Ghidra P0 has no database mutation authority. A PE/CLI or ReadyToRun
assembly should use `inspect_managed_artifact`; it is not NativeAOT.
@@ -43,42 +52,62 @@ arbitrary caller-supplied bytes. The bounded loader accepts regular JARs up to
## Use existing tools
Open the executable with provider `ghidra`. `inspect_native_load_image` returns
optional `observations.metadata_recovery` inline, including the detected header,
layout, recovery status, type category paths/addresses, diagnostics, annotation
coverage and derived-memory SHA. The enclosing `unsupported` load-image status
for PE/ELF concerns independent DOS loader verification; metadata recovery has
its own status. It is not a failed import.
Open the executable with provider `ghidra`. For an x64 PE, `inspect_native_load_image`
returns `observations.metadata_recovery` inline from the captured source snapshot,
including the RTR header, MethodTable addresses, frozen strings, derived overlay
identity, and examined/recovered counts. Its enclosing `unsupported` load-image
status means independent PE load-image verification was not performed; it does
not mean the metadata parser failed or the executable failed to load.
Pass one returned type path or its typed address to `inspect_native_data_type`.
Its optional `metadata_recovery` includes related/base type, implemented
interfaces and virtual slots with targets and known procedure names. Follow
these with `analyze_function`, string searches and ordinary cross-references.
For CLI use:
Pass a returned MethodTable address to `inspect_native_data_type`. A direct PE
query also derives the snapshot metadata from the authenticated Ghidra session
handshake, so it does not require a preceding load-image call. In a persistent
MCP session, load-image followed by type inspection reuses that same derivation.
The database result remains `unavailable` when Ghidra has no DataType at that
address; its `metadata_recovery.source` is `read-only-derived-overlay` and
carries related type, interface, slot, and target observations. Generated
address identities do not claim original managed names. The parser does not
merge frozen literals into Ghidra's database string inventory. Follow slot
targets with `analyze_function` and ordinary cross-references.
The built-in CLI path uses a MethodTable address returned by the parser or
another observation:
```sh
rea inspect-native-load-image ./NativeAotFixture --provider ghidra --json
rea inspect-native-data-type ./NativeAotFixture --type /NativeAOT/MethodTables/Class_ADDRESS_MT --provider ghidra --json
rea inspect-native-data-type ./NativeAotFixture --address 0xMETHODTABLE_ADDRESS --provider ghidra --json
rea function ./NativeAotFixture 0xADDRESS --provider ghidra --json
```
No new MCP workflow or approval flag is required. Configuration opts into the
adapter; without it ordinary Ghidra profiles/results remain unchanged.
The generated `/NativeAOT/...` category path belongs to the optional headless
adapter. Use `--type /NativeAOT/MethodTables/Class_ADDRESS_MT` only when that
adapter is configured and has created the corresponding Ghidra DataType.
Built-in PE analysis includes its parser contract and resource policy in the
Ghidra analysis profile; non-PE profiles do not include that contract.
## Evidence boundaries
Discovery prefers an imported RTR symbol, otherwise the upstream initialized,
non-executable-memory signature heuristic. REA rejects ambiguous candidates,
unknown layout versions, malformed directory rows, unsupported flags and
unmapped ranges before recovery. It requires the verified dehydrated layout.
The built-in parser uses an initialized non-executable PE signature scan and
requires a unique RTR 9.1 header and exactly one DEHYDRATED_DATA row. Rehydrated
bytes live only in a parser-owned overlay, identified by virtual address, size,
and SHA-256; the caller's source Buffer remains unchanged. Limits derive from
captured section extents and format field bounds. Separate resource policies
admit up to 128 MiB of captured PE bytes, 64 MiB-equivalent parser work, 64 MiB
of exact parser-owned overlay/index Buffers, and 4 MiB of serialized NativeAOT
report JSON. These are local parser limits, independent of MCP delivery; the
buffer counter does not claim to cap V8 heap or total process memory. Rows that
do not fit the report budget are omitted with explicit partial status and
coverage. Coverage counts describe the pointer scan, recovered MethodTables,
and frozen-string candidates examined, not every possible runtime type or
object. Cancellation is honored during snapshot I/O and checked immediately
before and after parsing; the synchronous parser call itself is bounded but
cannot be interrupted mid-call. Parser output never claims that a Ghidra
database type was defined.
The adapter performs rehydration, method-table recovery and frozen-object
annotation in one ephemeral Program transaction. Fatal failure/cancellation
rolls it back. Recovery may change analysis-memory bytes, never the original
executable file. Derived ranges have a digest and **no original file offset**;
they are not captured runtime state. Candidate/committed instance annotation
counts distinguish partial frozen-object coverage. Coverage concerns the
recovered candidate set, not every possible object or every runtime type.
The optional adapter performs its annotations in an ephemeral Program
transaction. That workflow may change analysis-memory bytes, never the original
executable file. It is unavailable on Windows P0, which has no database
mutation authority.
REA preserves pre-recovery calling conventions and uses the loaded compiler
specification default for newly created methods. The upstream universal
+3 -3
View File
@@ -8,9 +8,9 @@ Executable bytes remain unchanged.
Use [installation](installation.md#ghidra) for supported hosts, prerequisites,
configuration, startup deadlines, resource controls, and cleanup recovery.
[Windows P0](windows-ghidra-p0.md) has its own admission and lifecycle boundary;
[NativeAOT recovery](ghidra-nativeaot.md) is an optional, separately verified
extension. Check the [release boundary](installation.md#released-package-and-main)
when using npm.
[NativeAOT observations](ghidra-nativeaot.md) include a built-in read-only PE
snapshot parser and a separate optional annotation extension. Check the
[release boundary](installation.md#released-package-and-main) when using npm.
See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds)
to override the header-detected language or seed analysis.
+7
View File
@@ -672,6 +672,13 @@ boundaries and budgets.
## Optional NativeAOT Ghidra analysis
The source parser lane is independent of the optional annotation JAR. Its
synthetic RTR 9.1 PE golden checks captured-byte digest admission, rehydration
into an immutable parser overlay, MethodTable/slot relationships, and frozen
literal extraction; provider tests exercise `inspect_native_load_image` and
address-based `inspect_native_data_type` with the database result still
unavailable. These tests do not establish Windows Ghidra host behavior.
This lane is separate from the default native lane. `build:fixtures:nativeaot`
requires an existing .NET SDK 8.0.416, the platform NativeAOT compiler/linker and
runtime pack 8.0.22. It builds benign sources into ignored `_reference/` and
+6 -3
View File
@@ -21,9 +21,12 @@ additional permission flag or degraded mode is required.
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
PE load-image inspection returns measurements with an explicit unsupported
attestation status; independent format-specific verification currently supports
DOS MZ only.
PE load-image inspection returns an explicit unsupported attestation status;
independent load-image verification currently supports DOS MZ only. For x64 PE
NativeAOT RTR 9.1, the read-only snapshot parser can report metadata through
`inspect_native_load_image` and `inspect_native_data_type` without a JAR or
database mutation. A real Windows-host NativeAOT Ghidra run is still required
to establish the complete host workflow.
This boundary does not establish general Windows feature parity. Ghidra GUI
state, annotations, other target architectures and formats, and Hopper-only
+28
View File
@@ -0,0 +1,28 @@
The MIT License (MIT)
=====================
Copyright © 2023-2025 Washi
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
The read-only NativeAOT parser adapts format and hydration semantics from
Washi1337/ghidra-nativeaot commit
effeb734fc570c32650f88b159608979dc7b423e (MIT). MethodTable field layout was
cross-checked against .NET runtime src/coreclr/nativeaot/Runtime/inc/MethodTable.h
at dotnet/runtime commit a3b2d40328be0be3f8dd43f0e5cc925b8827b044 (MIT).
+2 -1
View File
@@ -81,7 +81,8 @@
"bridge/pwndbg/launch_impl.py",
"bridge/pwndbg/core_context.py",
"third_party/pwndbg/README.md",
"third_party/pwndbg/LICENSE.md"
"third_party/pwndbg/LICENSE.md",
"licenses/nativeaot-parser-MIT.txt"
],
"scripts": {
"release:notes": "node scripts/release-notes.mjs",
@@ -0,0 +1,176 @@
import { describe, expect, it } from "vitest";
import type { NativeAotParserLimits } from "./nativeAotPe.js";
import {
NATIVE_AOT_MAX_REPORT_BYTES,
parseNativeAotPe,
} from "./nativeAotPe.js";
import {
nativeAotPeDigest,
nativeAotPeFixture,
} from "../../../tests/fixtures/nativeaotPe.js";
const REPORT_BUDGET = NATIVE_AOT_MAX_REPORT_BYTES;
describe("NativeAOT parser resource budgets", () => {
it("reports source-cap rejection without hashing or inventing omitted-byte coverage", () => {
const bytes = nativeAotPeFixture();
const sha = nativeAotPeDigest(bytes);
const limits: NativeAotParserLimits = {
sourceBytes: bytes.length - 1,
workUnits: 1024,
workingMemoryBytes: 1024,
};
const report = parseNativeAotPe(
bytes,
sha,
REPORT_BUDGET,
undefined,
limits,
).summary;
expect(report).toMatchObject({
status: "partial",
analysis_artifact_sha256: sha,
header_address: null,
coverage: {
source_bytes_examined: 0,
source_byte_budget_bytes: bytes.length - 1,
truncation_reason: "source-byte-budget",
},
});
expect(report.coverage.source_bytes_omitted).toBeUndefined();
});
});
describe("NativeAOT parser report admission", () => {
it("never returns an over-budget report from early exits", () => {
const bytes = nativeAotPeFixture();
const sha = nativeAotPeDigest(bytes);
expect(() =>
parseNativeAotPe(bytes, sha, 0, undefined, {
sourceBytes: bytes.length - 1,
workUnits: 1024,
workingMemoryBytes: 1024,
}),
).toThrow(/minimum truthful report/iu);
const nonPe = Buffer.from("not a PE image");
expect(() => parseNativeAotPe(nonPe, nativeAotPeDigest(nonPe), 0)).toThrow(
/minimum truthful report/iu,
);
});
it("does not claim a missing RTR header when signature coverage hits the work limit", () => {
const bytes = nativeAotPeFixture();
const limits: NativeAotParserLimits = {
sourceBytes: bytes.length,
workUnits: 8,
workingMemoryBytes: 1024,
};
const report = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
REPORT_BUDGET,
undefined,
limits,
).summary;
expect(report).toMatchObject({
status: "partial",
header_address: null,
truncated: true,
coverage: {
rtr_signature_scan_bytes_examined: 8,
rtr_signature_scan_bytes_omitted: expect.any(Number),
truncation_reason: "work-unit-budget",
},
});
expect(report.reason).not.toMatch(/No supported ReadyToRun directory/iu);
});
it("does not claim a missing root when table discovery is incomplete", () => {
const bytes = nativeAotPeFixture();
const report = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
REPORT_BUDGET,
undefined,
{
sourceBytes: bytes.length,
workUnits: 10_000,
workingMemoryBytes: 1024,
},
).summary;
expect(report.truncated).toBe(true);
expect(report.reason).toMatch(/before determining whether.*root exists/iu);
expect(report.reason).not.toMatch(
/no unique System\.Object.*fully examined/iu,
);
});
});
describe("NativeAOT detail and summary byte accounting", () => {
it("admits serialized MethodTable memory descriptors before emitting them", () => {
const bytes = nativeAotPeFixture();
const sha = nativeAotPeDigest(bytes);
const detail = (budget: number) =>
parseNativeAotPe(bytes, sha, REPORT_BUDGET).readTypeDetail(
"0x140002280",
budget,
);
const complete = detail(REPORT_BUDGET);
expect(complete?.derived_memory).not.toBeNull();
let low = 0;
let high = Buffer.byteLength(JSON.stringify(complete));
while (low < high) {
const middle = Math.floor((low + high) / 2);
if (detail(middle)?.derived_memory != null) high = middle;
else low = middle + 1;
}
const atBoundary = detail(low);
const belowBoundary = detail(low - 1);
expect(atBoundary?.derived_memory).not.toBeNull();
expect(Buffer.byteLength(JSON.stringify(atBoundary))).toBeLessThanOrEqual(
low,
);
expect(belowBoundary?.derived_memory ?? null).toBeNull();
expect(belowBoundary?.diagnostics.join(" ")).toContain(
"omitted because the final serialized report exceeded its byte budget",
);
expect(
belowBoundary === undefined
? 0
: Buffer.byteLength(JSON.stringify(belowBoundary)),
).toBeLessThanOrEqual(low - 1);
});
it("truncates report rows before exceeding its JSON byte budget", () => {
const bytes = nativeAotPeFixture();
const sha = nativeAotPeDigest(bytes);
const complete = parseNativeAotPe(bytes, sha, REPORT_BUDGET);
const fullBytes = Buffer.byteLength(JSON.stringify(complete.summary));
let budget = fullBytes - 1;
let summary: ReturnType<typeof parseNativeAotPe>["summary"] | undefined;
while (budget > 0) {
try {
const candidate = parseNativeAotPe(bytes, sha, budget);
if (candidate.summary.truncated) {
summary = candidate.summary;
break;
}
} catch {
// Budgets below the fixed report header cannot carry an honest summary.
}
budget--;
}
if (summary === undefined)
throw new Error("No partial summary budget found");
expect(summary.truncated).toBe(true);
expect(
(summary.types_omitted ?? 0) +
(summary.frozen_strings_omitted ?? 0) +
(summary.coverage.scan_bytes_omitted ?? 0) +
(summary.coverage.frozen_scan_bytes_omitted ?? 0),
).toBeGreaterThan(0);
expect(Buffer.byteLength(JSON.stringify(summary))).toBeLessThanOrEqual(
budget,
);
});
});
+105
View File
@@ -0,0 +1,105 @@
import { describe, expect, it } from "vitest";
import { nativeMetadataRecoverySummarySchema } from "./nativeMetadataRecovery.js";
import {
NATIVE_AOT_MAX_REPORT_BYTES,
parseNativeAotPe,
} from "./nativeAotPe.js";
import {
nativeAotPeDigest,
nativeAotPeFixture,
} from "../../../tests/fixtures/nativeaotPe.js";
const SUMMARY_BUDGET = NATIVE_AOT_MAX_REPORT_BYTES;
describe("read-only NativeAOT PE parser", () => {
it("recovers a bounded RTR9.1 type graph and frozen literal from captured bytes", () => {
const bytes = nativeAotPeFixture();
const before = Buffer.from(bytes);
const parsed = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
SUMMARY_BUDGET,
"0x140000000",
);
expect(bytes).toEqual(before);
expect(parsed.summary).toMatchObject({
format: "dotnet-nativeaot",
status: "complete",
analysis_mode: "read-only-derived-overlay",
format_major: 9,
format_minor: 1,
method_tables: 3,
frozen_strings: [
{ address: "0x140002300", value: "REA_NATIVEAOT_FROZEN" },
],
derived_overlay: { address: "0x140002380", size_bytes: 1 },
coverage: {
basis: "raw-image-and-rehydrated-pointer-scan",
method_tables_recovered: 3,
frozen_string_candidates: 1,
},
});
expect(parsed.readTypeDetail("0x140002280", SUMMARY_BUDGET)).toMatchObject({
source: "read-only-derived-overlay",
method_table_address: "0x140002280",
related_type: { address: "0x140002200", type: null },
virtual_slots: [{ slot: 0, target_address: "0x140001040" }],
});
expect(nativeMetadataRecoverySummarySchema.parse(parsed.summary)).toEqual(
parsed.summary,
);
});
it("fails closed on snapshot identity mismatch", () => {
const bytes = nativeAotPeFixture();
expect(() =>
parseNativeAotPe(bytes, "0".repeat(64), SUMMARY_BUDGET),
).toThrow(/digest/iu);
});
it("reports a detected but unsupported RTR version as partial without guessed types", () => {
const bytes = nativeAotPeFixture(10, 0);
const parsed = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
SUMMARY_BUDGET,
);
expect(parsed.summary).toMatchObject({
status: "partial",
format_major: 10,
format_minor: 0,
method_tables: 0,
types: [],
});
});
it("does not claim an image mapping when Ghidra rebased the image", () => {
const bytes = nativeAotPeFixture();
expect(
parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
SUMMARY_BUDGET,
"0x180000000",
).summary.reason,
).toMatch(/preferred image base/iu);
});
it("reports an unsupported hydration opcode as partial without mutating the snapshot", () => {
const bytes = nativeAotPeFixture();
bytes.writeUInt8(6, 0x704);
const before = Buffer.from(bytes);
const parsed = parseNativeAotPe(
bytes,
nativeAotPeDigest(bytes),
SUMMARY_BUDGET,
);
expect(bytes).toEqual(before);
expect(parsed.summary).toMatchObject({
status: "partial",
derived_overlay: null,
diagnostics: [
expect.stringContaining("Unsupported DEHYDRATED_DATA command"),
],
});
});
});
File diff suppressed because it is too large Load Diff
+85 -11
View File
@@ -6,6 +6,8 @@ const relatedType = z.strictObject({ address, type: z.string().nullable() });
/** Runtime-format metadata recovered in an analysis image, with explicit inference boundaries. */
export const nativeMetadataRecoverySchema = z.strictObject({
format: z.literal("dotnet-nativeaot"),
/** Identifies parser-only facts without claiming Ghidra defined a DataType. */
source: z.enum(["analysis-database", "read-only-derived-overlay"]).optional(),
status: z.enum(["complete", "partial"]),
header_address: address,
format_major: z.number().int().nonnegative(),
@@ -25,19 +27,29 @@ export const nativeMetadataRecoverySchema = z.strictObject({
basis: z.literal("method-table-pointer"),
}),
),
derived_memory: z.strictObject({
address,
size_bytes: z.number().int().nonnegative(),
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
file_offset: z.null(),
}),
derived_memory: z
.strictObject({
address,
size_bytes: z.number().int().nonnegative(),
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
file_offset: z.null(),
})
.nullable(),
truncated: z.boolean().optional(),
virtual_slots_omitted: z.number().int().nonnegative().optional(),
interfaces_omitted: z.number().int().nonnegative().optional(),
output_budget_bytes: z.number().int().nonnegative().optional(),
diagnostics: z.array(z.string()),
limitations: z.array(z.string()),
});
export type NativeMetadataRecovery = z.infer<
typeof nativeMetadataRecoverySchema
>;
/** Discover a recovered runtime format and reusable type identities from the loaded image. */
export const nativeMetadataRecoverySummarySchema = z.strictObject({
format: z.literal("dotnet-nativeaot"),
analysis_mode: z.literal("read-only-derived-overlay").optional(),
status: z.enum(["complete", "partial", "not_applicable"]),
reason: z.string().nullable(),
analysis_artifact_sha256: z.string().regex(/^[a-f0-9]{64}$/u),
@@ -47,14 +59,76 @@ export const nativeMetadataRecoverySummarySchema = z.strictObject({
format_minor: z.number().int().nonnegative().nullable(),
method_tables: z.number().int().nonnegative(),
types: z.array(relatedType),
derived_memory: nativeMetadataRecoverySchema.shape.derived_memory.nullable(),
frozen_strings: z
.array(z.strictObject({ address, value: z.string() }))
.optional(),
truncated: z.boolean().optional(),
types_omitted: z.number().int().nonnegative().optional(),
frozen_strings_omitted: z.number().int().nonnegative().optional(),
output_budget_bytes: z.number().int().nonnegative().optional(),
derived_overlay: z
.strictObject({
address,
size_bytes: z.number().int().nonnegative(),
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
})
.nullable()
.optional(),
derived_memory: nativeMetadataRecoverySchema.shape.derived_memory,
coverage: z
.strictObject({
frozen_object_candidates: z.number().int().nonnegative(),
frozen_objects_annotated: z.number().int().nonnegative(),
basis: z.literal(
frozen_object_candidates: z.number().int().nonnegative().optional(),
frozen_objects_annotated: z.number().int().nonnegative().optional(),
pointer_candidates_examined: z.number().int().nonnegative().optional(),
pointer_slots_examined: z.number().int().nonnegative().optional(),
scan_bytes_examined: z.number().int().nonnegative().optional(),
scan_bytes_omitted: z.number().int().nonnegative().optional(),
rtr_signature_scan_bytes_examined: z
.number()
.int()
.nonnegative()
.optional(),
rtr_signature_scan_bytes_omitted: z
.number()
.int()
.nonnegative()
.optional(),
method_table_validation_items_examined: z
.number()
.int()
.nonnegative()
.optional(),
work_units_examined: z.number().int().nonnegative().optional(),
method_tables_recovered: z.number().int().nonnegative().optional(),
frozen_string_candidates: z.number().int().nonnegative().optional(),
frozen_scan_bytes_examined: z.number().int().nonnegative().optional(),
frozen_scan_bytes_omitted: z.number().int().nonnegative().optional(),
frozen_string_data_bytes_examined: z
.number()
.int()
.nonnegative()
.optional(),
source_bytes_examined: z.number().int().nonnegative().optional(),
source_bytes_omitted: z.number().int().nonnegative().optional(),
source_byte_budget_bytes: z.number().int().nonnegative().optional(),
work_budget_units: z.number().int().nonnegative().optional(),
working_memory_budget_bytes: z.number().int().nonnegative().optional(),
working_memory_bytes_peak: z.number().int().nonnegative().optional(),
truncation_reason: z
.enum([
"source-byte-budget",
"work-unit-budget",
"working-memory-budget",
"response-budget",
])
.optional(),
basis: z.enum([
"rehydrated-pointer-candidates-and-committed-instance-types",
),
"raw-image-pointer-scan",
"raw-image-and-rehydrated-pointer-scan",
"unsupported-layout",
"no-rtr-header",
]),
})
.nullable(),
diagnostics: z.array(z.string()),
+14
View File
@@ -17,6 +17,13 @@ import {
ghidraMipsProfileParameters,
ghidraMipsUnsupportedReason,
} from "./GhidraMipsProfile.js";
import {
NATIVE_AOT_MAX_REPORT_BYTES,
NATIVE_AOT_MAX_SOURCE_BYTES,
NATIVE_AOT_MAX_WORKING_MEMORY_BYTES,
NATIVE_AOT_MAX_WORK_UNITS,
NATIVE_AOT_PROFILE_CONTRACT_REVISION,
} from "../domain/native/nativeAotPe.js";
/** Resolve version-bound, deterministic semantics before Ghidra imports a target. */
export const resolveGhidraAnalysisProfile = (
@@ -69,6 +76,13 @@ export const resolveGhidraAnalysisProfile = (
? {
executable_role: target.executableRole ?? null,
managed: target.managed ?? null,
native_aot_metadata: {
contract_revision: NATIVE_AOT_PROFILE_CONTRACT_REVISION,
source_bytes: NATIVE_AOT_MAX_SOURCE_BYTES,
work_units: NATIVE_AOT_MAX_WORK_UNITS,
working_memory_bytes: NATIVE_AOT_MAX_WORKING_MEMORY_BYTES,
report_bytes: NATIVE_AOT_MAX_REPORT_BYTES,
},
}
: {}),
architecture: target.architecture ?? null,
+95 -3
View File
@@ -43,6 +43,21 @@ import {
} from "./GhidraSessionValues.js";
import { createGhidraTargetSnapshot } from "./GhidraTargetSnapshot.js";
import { readFile } from "node:fs/promises";
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
import {
openRegularFile,
RegularFileChangedError,
sameRegularFileState,
} from "../filesystem/RegularFile.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
export type GhidraTargetSnapshot =
| { readonly kind: "captured"; readonly bytes: Buffer }
| {
readonly kind: "over-capacity";
readonly sourceBytesAtLeast: number;
readonly maximumBytes: number;
};
import { connectGhidraSocket } from "./GhidraSocketConnection.js";
import {
createGhidraEndpoint,
@@ -79,6 +94,7 @@ export class GhidraClient {
#socketRoot: PrivateRuntimeRoot | undefined;
#endpointPath: string | undefined;
#snapshotPath: string | undefined;
readonly #snapshotOwners = new Set<OwnedFileHandle>();
#targetAdmission: JsonValue | undefined;
#token: string | undefined;
// Retain authentication identities for diagnostics from late request settlement.
@@ -227,7 +243,10 @@ export class GhidraClient {
}
/** Read the private immutable source for independent import verification. */
async readTargetSnapshot(): Promise<Result<Buffer, GhidraSessionError>> {
async readTargetSnapshot(
maximumBytes?: number,
signal?: AbortSignal,
): Promise<Result<GhidraTargetSnapshot, GhidraSessionError>> {
if (this.#snapshotPath === undefined)
return err(
this.#failure(
@@ -236,11 +255,80 @@ export class GhidraClient {
),
);
try {
return ok(await readFile(this.#snapshotPath));
if (maximumBytes !== undefined) {
signal?.throwIfAborted();
if (!Number.isSafeInteger(maximumBytes) || maximumBytes < 0)
throw new RangeError("Snapshot byte limit must be a safe integer");
const handle = await openRegularFile(this.#snapshotPath, {
symlinks: "reject",
});
const owner = new OwnedFileHandle(handle);
this.#snapshotOwners.add(owner);
let result: GhidraTargetSnapshot;
let readFailure: unknown;
try {
const metadata = await handle.stat();
signal?.throwIfAborted();
if (metadata.size > maximumBytes)
result = {
kind: "over-capacity",
sourceBytesAtLeast: metadata.size,
maximumBytes,
};
else {
const bytes = await readBoundedFileBytes(
handle,
maximumBytes,
signal,
metadata.size,
);
signal?.throwIfAborted();
const after = await handle.stat();
if (
bytes !== undefined &&
bytes.length === metadata.size &&
sameRegularFileState(metadata, after)
) {
result = { kind: "captured", bytes };
} else if (
bytes === undefined &&
after.isFile() &&
after.size > maximumBytes
) {
result = {
kind: "over-capacity",
sourceBytesAtLeast: after.size,
maximumBytes,
};
} else {
throw new RegularFileChangedError(this.#snapshotPath);
}
}
} catch (cause: unknown) {
readFailure = cause;
}
try {
await owner.close();
this.#snapshotOwners.delete(owner);
} catch (closeFailure: unknown) {
throw new AggregateError(
readFailure === undefined
? [closeFailure]
: [readFailure, closeFailure],
"Ghidra target snapshot read or descriptor cleanup failed",
);
}
if (readFailure !== undefined) throw readFailure;
return ok(result!);
}
return ok({
kind: "captured",
bytes: await readFile(this.#snapshotPath),
});
} catch (cause: unknown) {
return err(
this.#failure(
"protocol",
signal?.aborted === true ? "cancelled" : "protocol",
"Ghidra target snapshot could not be read for load-image verification",
cause,
),
@@ -602,6 +690,10 @@ export class GhidraClient {
this.#lastDiagnostics = this.#diagnostics();
this.#process = undefined;
this.#launch = undefined;
for (const owner of this.#snapshotOwners) {
await owner.close();
this.#snapshotOwners.delete(owner);
}
try {
await this.#removeRuntimeRoots();
} catch (cause: unknown) {
+87 -9
View File
@@ -1,4 +1,5 @@
import type { AnalysisOperation } from "../application/AnalysisProvider.js";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
@@ -12,23 +13,91 @@ import {
attestGhidraDosComLoadImage,
attestGhidraDosLoadImage,
} from "./GhidraLoadImageValues.js";
import { type NativeAotPeResult } from "../domain/native/nativeAotPe.js";
import type { GhidraTargetSnapshot } from "./GhidraClient.js";
import type { GhidraSessionError } from "./GhidraSessionError.js";
type LoadImageSnapshotClient = {
readonly readTargetSnapshot?: () => Promise<
Result<Buffer, GhidraSessionError>
>;
readonly readTargetSnapshot?: (
maximumBytes?: number,
signal?: AbortSignal,
) => Promise<Result<GhidraTargetSnapshot, GhidraSessionError>>;
};
type NativeLoadImageAttestationInput = {
readonly target: BinaryTarget;
readonly operation: AnalysisOperation;
readonly measured: JsonValue;
readonly client: LoadImageSnapshotClient;
readonly mapSessionError: (failure: GhidraSessionError) => AnalysisError;
readonly nativeAotResult?: NativeAotPeResult | undefined;
readonly signal?: AbortSignal | undefined;
};
const isSignalAborted = (signal: AbortSignal | undefined): boolean =>
signal?.aborted === true;
/** Independently verify measured load-image observations for admitted DOS targets. */
export const attestGhidraNativeLoadImage = async (
target: BinaryTarget,
operation: AnalysisOperation,
measured: JsonValue,
client: LoadImageSnapshotClient,
mapSessionError: (failure: GhidraSessionError) => AnalysisError,
input: NativeLoadImageAttestationInput,
): Promise<Result<JsonValue, AnalysisError>> => {
const {
target,
operation,
measured,
client,
mapSessionError,
nativeAotResult,
signal,
} = input;
if (isSignalAborted(signal))
return err(new AnalysisCancelledError(operation));
const observations = nativeLoadImageObservationSchema.parse(measured);
if (target.format === "pe") {
if (nativeAotResult === undefined)
return err(
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason:
"The Ghidra provider did not produce read-only NativeAOT metadata for this admitted PE snapshot.",
},
}),
);
if (isSignalAborted(signal))
return err(new AnalysisCancelledError(operation));
try {
return ok(
jsonValueSchema.parse(
nativeLoadImageSchema.parse({
status: "unsupported",
reason:
"Independent load-image verification currently supports DOS MZ and explicit DOS COM targets only.",
observations: {
...observations,
metadata_recovery: [
...(observations.metadata_recovery ?? []),
nativeAotResult.summary,
],
},
limitations: [
"Load-image verification is not performed for PE targets; NativeAOT metadata observations are derived separately from the exact admitted snapshot.",
],
}),
),
);
} catch (cause: unknown) {
return err(
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason:
cause instanceof Error
? cause.message
: "NativeAOT metadata could not be represented in the load-image result",
},
}),
);
}
}
if (target.format !== "dos-mz" && target.format !== "dos-com") {
return ok(
jsonValueSchema.parse(
@@ -55,11 +124,20 @@ export const attestGhidraNativeLoadImage = async (
);
const snapshot = await client.readTargetSnapshot();
if (!snapshot.ok) return err(mapSessionError(snapshot.error));
if (snapshot.value.kind !== "captured")
return err(
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason:
"The uncapped DOS load-image snapshot was unexpectedly not captured.",
},
}),
);
const attested = (
target.format === "dos-com"
? attestGhidraDosComLoadImage
: attestGhidraDosLoadImage
)(snapshot.value, target.sha256 ?? "", observations);
)(snapshot.value.bytes, target.sha256 ?? "", observations);
if (!attested.ok)
return err(
new ProviderAdapterError("ghidra", operation, {
+522 -1
View File
@@ -10,16 +10,22 @@ import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
import { fixtureDosLoadImage } from "./GhidraLoadImage.fixture.js";
import {
nativeAotPeDigest,
nativeAotPeFixture,
} from "../../tests/fixtures/nativeaotPe.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { parseConfig } from "../config/parseConfig.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import { parseExecutableHeader } from "../domain/binaryTarget.js";
import { GhidraProvider } from "./GhidraProvider.js";
import type { GhidraTargetSnapshot } from "./GhidraClient.js";
import type { GhidraProviderClientFactory } from "./GhidraProviderClient.js";
import type { GhidraInstallationHost } from "./GhidraInstallation.js";
import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js";
import { err, ok } from "../domain/result.js";
import type { Result } from "../domain/result.js";
import { GhidraSessionError } from "./GhidraSessionError.js";
import { silentLogger } from "../logger.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
@@ -219,10 +225,25 @@ describe("Ghidra platform support", () => {
expect(profile.parameters).toMatchObject({
executable_role: target.executableRole,
managed: target.managed,
native_aot_metadata: {
contract_revision: "rtr-9.1-x64-pe-read-only-v1",
source_bytes: 128 * 1024 * 1024,
work_units: 64 * 1024 * 1024,
working_memory_bytes: 64 * 1024 * 1024,
report_bytes: 4 * 1024 * 1024,
},
});
digests.add(profile.digest);
}
expect(digests.size).toBe(3);
const nonPe = await ghidra.resolveAnalysisProfile(
executableTarget("elf", "x86_64"),
);
if (!nonPe.ok || nonPe.value.profile === null)
throw new Error("Expected a non-PE analysis profile");
expect(nonPe.value.profile.parameters).not.toHaveProperty(
"native_aot_metadata",
);
});
it("keeps Windows annotation mutation unavailable independently of native controls", () => {
@@ -717,7 +738,8 @@ describe("Ghidra measured load-image projection", () => {
...(state === "missing-snapshot"
? {}
: {
readTargetSnapshot: () => Promise.resolve(ok(fixture.bytes)),
readTargetSnapshot: () =>
Promise.resolve(ok({ kind: "captured", bytes: fixture.bytes })),
}),
});
const ghidra = provider(installationHost(), factory);
@@ -757,6 +779,505 @@ describe("Ghidra measured load-image projection", () => {
);
});
it("returns read-only NativeAOT metadata inline and drills down by derived MethodTable address", async () => {
const bytes = nativeAotPeFixture();
const observations = fixtureDosLoadImage().observation;
observations.image_base = "0x140000000";
const factory: GhidraProviderClientFactory = () => ({
start: () =>
Promise.resolve(
ok({
...sessionInfo(),
target: { ...sessionInfo().target, image_base: "0x140000000" },
}),
),
callTool: (operation) =>
Promise.resolve(
ok(
jsonValueSchema.parse(
operation === "inspect_native_load_image"
? observations
: {
status: "unavailable",
reason: "No Ghidra DataType is defined at this address.",
id: null,
name: null,
kind: "unavailable",
source: "analysis-database",
source_archive: null,
address: null,
size_bytes: null,
alignment_bytes: null,
packing_enabled: null,
referenced_type: null,
array_count: null,
array_stride_bytes: null,
fields: [],
members: [],
total_fields: 0,
truncated: false,
limitations: [],
},
),
),
),
readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })),
close: () => Promise.resolve(ok(null)),
});
const ghidra = provider(installationHost(), factory);
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
const profile = await ghidra.resolveAnalysisProfile(target);
if (!profile.ok || profile.value.profile === null)
throw new Error("Expected admitted PE profile");
const client = ghidra.createClient(target, profile.value.profile);
const image = await client.execute("inspect_native_load_image", {});
expect(image).toMatchObject({
ok: true,
value: {
result: {
status: "unsupported",
observations: {
metadata_recovery: [
{
status: "complete",
analysis_mode: "read-only-derived-overlay",
method_tables: 3,
frozen_strings: [{ value: "REA_NATIVEAOT_FROZEN" }],
},
],
},
},
rawResult: observations,
},
});
const detail = await client.execute("inspect_native_data_type", {
address: "0x140002280",
});
expect(detail).toMatchObject({
ok: true,
value: {
result: {
status: "unavailable",
source: "analysis-database",
metadata_recovery: {
source: "read-only-derived-overlay",
method_table_address: "0x140002280",
related_type: { address: "0x140002200", type: null },
},
},
},
});
await client.close();
});
it("derives PE metadata for a direct type-address query without a load-image call", async () => {
const bytes = nativeAotPeFixture();
const operations: string[] = [];
const factory: GhidraProviderClientFactory = () => ({
start: () =>
Promise.resolve(
ok({
...sessionInfo(),
target: { ...sessionInfo().target, image_base: "0x140000000" },
}),
),
callTool: (operation) => {
operations.push(operation);
return Promise.resolve(
ok(
jsonValueSchema.parse({
status: "unavailable",
reason: "No Ghidra DataType is defined at this address.",
id: null,
name: null,
kind: "unavailable",
source: "analysis-database",
source_archive: null,
address: null,
size_bytes: null,
alignment_bytes: null,
packing_enabled: null,
referenced_type: null,
array_count: null,
array_stride_bytes: null,
fields: [],
members: [],
total_fields: 0,
truncated: false,
limitations: [],
}),
),
);
},
readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })),
close: () => Promise.resolve(ok(null)),
});
const ghidra = provider(installationHost(), factory);
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
const profile = await ghidra.resolveAnalysisProfile(target);
if (!profile.ok || profile.value.profile === null)
throw new Error("Expected admitted PE profile");
const client = ghidra.createClient(target, profile.value.profile);
const detail = await client.execute("inspect_native_data_type", {
address: "0x140002280",
});
expect(detail).toMatchObject({
ok: true,
value: {
result: {
metadata_recovery: {
source: "read-only-derived-overlay",
method_table_address: "0x140002280",
related_type: { address: "0x140002200", type: null },
},
},
},
});
expect(operations).toEqual(["inspect_native_data_type"]);
await client.close();
});
it("shares one session snapshot across callers while isolating cancellation and rejecting a changed base", async () => {
const bytes = nativeAotPeFixture();
const baseObservation = fixtureDosLoadImage().observation;
baseObservation.image_base = "0x140000000";
let calls = 0;
let snapshotReads = 0;
let snapshotSignal: AbortSignal | undefined;
let releaseSnapshot:
| ((value: Result<GhidraTargetSnapshot, GhidraSessionError>) => void)
| undefined;
let markSnapshotStarted: (() => void) | undefined;
const snapshotStarted = new Promise<void>((resolve) => {
markSnapshotStarted = resolve;
});
const snapshotResult = new Promise<
Result<GhidraTargetSnapshot, GhidraSessionError>
>((resolve) => {
releaseSnapshot = resolve;
});
const factory: GhidraProviderClientFactory = () => ({
start: () => Promise.resolve(ok(sessionInfo())),
callTool: (operation) => {
calls += 1;
const observation = {
...baseObservation,
source_files: [
{ ...baseObservation.source_files[0]!, name: `caller-${calls}` },
],
};
return Promise.resolve(
ok(
jsonValueSchema.parse(
operation === "inspect_native_load_image"
? observation
: {
status: "unavailable",
reason: "No Ghidra DataType is defined at this address.",
id: null,
name: null,
kind: "unavailable",
source: "analysis-database",
source_archive: null,
address: null,
size_bytes: null,
alignment_bytes: null,
packing_enabled: null,
referenced_type: null,
array_count: null,
array_stride_bytes: null,
fields: [],
members: [],
total_fields: 0,
truncated: false,
limitations: [],
},
),
),
);
},
readTargetSnapshot: (_maximumBytes, signal) => {
snapshotReads += 1;
snapshotSignal = signal;
markSnapshotStarted?.();
return snapshotResult;
},
close: () => Promise.resolve(ok(null)),
});
const ghidra = provider(installationHost(), factory);
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
const profile = await ghidra.resolveAnalysisProfile(target);
if (!profile.ok || profile.value.profile === null)
throw new Error("Expected admitted PE profile");
const client = ghidra.createClient(target, profile.value.profile);
const firstCaller = new AbortController();
const secondCaller = new AbortController();
const first = client.execute(
"inspect_native_load_image",
{},
{
signal: firstCaller.signal,
},
);
const second = client.execute(
"inspect_native_load_image",
{},
{
signal: secondCaller.signal,
},
);
await snapshotStarted;
expect(snapshotReads).toBe(1);
expect(snapshotSignal).toBeDefined();
expect(snapshotSignal).not.toBe(firstCaller.signal);
expect(snapshotSignal).not.toBe(secondCaller.signal);
firstCaller.abort();
await expect(first).resolves.toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
releaseSnapshot?.(ok({ kind: "captured", bytes }));
const secondResult = await second;
expect(secondResult).toMatchObject({
ok: true,
value: {
rawResult: {
source_files: [{ name: "caller-2" }],
},
result: {
observations: {
source_files: [{ name: "caller-2" }],
},
},
},
});
expect(snapshotReads).toBe(1);
expect(calls).toBe(2);
baseObservation.image_base = "0x150000000";
const conflictingBase = await client.execute("inspect_native_load_image", {});
expect(conflictingBase).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
diagnostics: {
reason: expect.stringContaining("different loaded image base"),
},
},
});
expect(snapshotReads).toBe(1);
await client.close();
expect(snapshotSignal?.aborted).toBe(true);
});
it("retries a failed NativeAOT snapshot read without retaining a failed derivation", async () => {
const bytes = nativeAotPeFixture();
const observations = fixtureDosLoadImage().observation;
observations.image_base = "0x140000000";
let snapshotReads = 0;
const factory: GhidraProviderClientFactory = () => ({
start: () => Promise.resolve(ok(sessionInfo())),
callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))),
readTargetSnapshot: () => {
snapshotReads += 1;
return Promise.resolve(
snapshotReads === 1
? err(
new GhidraSessionError(
"process",
"temporary snapshot read failure",
),
)
: ok({ kind: "captured", bytes }),
);
},
close: () => Promise.resolve(ok(null)),
});
const ghidra = provider(installationHost(), factory);
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
const profile = await ghidra.resolveAnalysisProfile(target);
if (!profile.ok || profile.value.profile === null)
throw new Error("Expected admitted PE profile");
const client = ghidra.createClient(target, profile.value.profile);
await expect(
client.execute("inspect_native_load_image", {}),
).resolves.toMatchObject({
ok: false,
error: { _tag: "ProviderAdapterError" },
});
await expect(
client.execute("inspect_native_load_image", {}),
).resolves.toMatchObject({
ok: true,
value: {
result: {
observations: {
metadata_recovery: [{ status: "complete" }],
},
},
},
});
expect(snapshotReads).toBe(2);
await client.close();
});
it("rejects all waiters when the loaded image base changes during derivation", async () => {
const bytes = nativeAotPeFixture();
const observation = fixtureDosLoadImage().observation;
let calls = 0;
let resolveSnapshot:
| ((value: Result<GhidraTargetSnapshot, GhidraSessionError>) => void)
| undefined;
let markSnapshotStarted: (() => void) | undefined;
const snapshotStarted = new Promise<void>((resolve) => {
markSnapshotStarted = resolve;
});
const snapshot = new Promise<
Result<GhidraTargetSnapshot, GhidraSessionError>
>((resolve) => {
resolveSnapshot = resolve;
});
const factory: GhidraProviderClientFactory = () => ({
start: () => Promise.resolve(ok(sessionInfo())),
callTool: () => {
calls += 1;
return Promise.resolve(
ok(
jsonValueSchema.parse({
...observation,
image_base: calls === 1 ? "0x140000000" : "0x150000000",
}),
),
);
},
readTargetSnapshot: () => {
markSnapshotStarted?.();
return snapshot;
},
close: () => Promise.resolve(ok(null)),
});
const ghidra = provider(installationHost(), factory);
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
const profile = await ghidra.resolveAnalysisProfile(target);
if (!profile.ok || profile.value.profile === null)
throw new Error("Expected admitted PE profile");
const client = ghidra.createClient(target, profile.value.profile);
const originalBase = client.execute("inspect_native_load_image", {});
await snapshotStarted;
const changedBase = await client.execute("inspect_native_load_image", {});
expect(changedBase).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
diagnostics: {
observed_image_base: "0x150000000",
measured_observation: { image_base: "0x150000000" },
},
},
});
resolveSnapshot?.(ok({ kind: "captured", bytes }));
await expect(originalBase).resolves.toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
diagnostics: {
reason: expect.stringContaining("conflicting loaded image bases"),
measured_observation: { image_base: "0x140000000" },
conflicting_observation: { image_base: "0x150000000" },
},
},
});
expect(calls).toBe(2);
await client.close();
});
it("cancels an in-flight NativeAOT waiter when its owning session closes", async () => {
const bytes = nativeAotPeFixture();
const observations = fixtureDosLoadImage().observation;
observations.image_base = "0x140000000";
let resolveSnapshot:
| ((value: Result<GhidraTargetSnapshot, GhidraSessionError>) => void)
| undefined;
let snapshotSignal: AbortSignal | undefined;
let markSnapshotStarted: (() => void) | undefined;
const snapshotStarted = new Promise<void>((resolve) => {
markSnapshotStarted = resolve;
});
const snapshot = new Promise<
Result<GhidraTargetSnapshot, GhidraSessionError>
>((resolve) => {
resolveSnapshot = resolve;
});
const factory: GhidraProviderClientFactory = () => ({
start: () => Promise.resolve(ok(sessionInfo())),
callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))),
readTargetSnapshot: (_maximumBytes, signal) => {
snapshotSignal = signal;
markSnapshotStarted?.();
return snapshot;
},
close: () => Promise.resolve(ok(null)),
});
const ghidra = provider(installationHost(), factory);
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
const profile = await ghidra.resolveAnalysisProfile(target);
if (!profile.ok || profile.value.profile === null)
throw new Error("Expected admitted PE profile");
const client = ghidra.createClient(target, profile.value.profile);
const pending = client.execute("inspect_native_load_image", {});
await snapshotStarted;
await client.close();
expect(snapshotSignal?.aborted).toBe(true);
resolveSnapshot?.(ok({ kind: "captured", bytes }));
await expect(pending).resolves.toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
});
describe("Ghidra NativeAOT snapshot budget", () => {
it("reports snapshot-capacity omissions inline through the load-image result", async () => {
const observations = fixtureDosLoadImage().observation;
const target = peTarget("x86_64");
const factory: GhidraProviderClientFactory = () => ({
start: () => Promise.resolve(ok(sessionInfo())),
callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))),
readTargetSnapshot: () =>
Promise.resolve(
ok({
kind: "over-capacity",
sourceBytesAtLeast: 128 * 1024 * 1024 + 1,
maximumBytes: 128 * 1024 * 1024,
}),
),
close: () => Promise.resolve(ok(null)),
});
const ghidra = provider(installationHost(), factory);
const profile = await ghidra.resolveAnalysisProfile(target);
if (!profile.ok || profile.value.profile === null)
throw new Error("Expected admitted PE profile");
const client = ghidra.createClient(target, profile.value.profile);
const image = await client.execute("inspect_native_load_image", {});
expect(image).toMatchObject({
ok: true,
value: {
result: {
observations: {
metadata_recovery: [
{
status: "partial",
truncated: true,
coverage: { truncation_reason: "source-byte-budget" },
},
],
},
},
},
});
await client.close();
});
});
const sessionInfo = () => ({
name: "REA Ghidra bridge" as const,
run_id: "11111111-1111-4111-8111-111111111111",
+337 -6
View File
@@ -8,7 +8,6 @@ import {
type AnalysisOperation,
} from "../application/AnalysisProvider.js";
import type { AppConfig } from "../config/types.js";
import type { JsonValue } from "../domain/jsonValue.js";
import {
ghidraSeedCommitmentSchema,
ghidraSeedFailure,
@@ -30,6 +29,7 @@ import {
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { type AnalysisError } from "../domain/analysisErrorBase.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { err, ok, type Result } from "../domain/result.js";
import type { Logger } from "pino";
@@ -52,6 +52,18 @@ import {
import { unverifiedGhidraBuildLimitation } from "./GhidraInstallationPolicy.js";
import { GhidraHeadlessLauncher } from "./GhidraLauncher.js";
import { attestGhidraNativeLoadImage } from "./GhidraLoadImageAttest.js";
import {
NATIVE_AOT_MAX_REPORT_BYTES,
NATIVE_AOT_MAX_SOURCE_BYTES,
nativeAotSnapshotOverCapacity,
parseNativeAotPe,
type NativeAotPeResult,
} from "../domain/native/nativeAotPe.js";
import {
ABORTED,
waitForAbortable,
} from "../application/binary/AbortablePromise.js";
import { nativeLoadImageObservationSchema } from "../domain/native/nativeLoadImage.js";
import {
GHIDRA_PROVIDER_IDENTITY,
healthLimitations,
@@ -77,6 +89,250 @@ export type GhidraProviderClientFactory = (
) => Pick<GhidraClient, "start" | "callTool" | "close"> &
Partial<Pick<GhidraClient, "runtimeLineage" | "readTargetSnapshot">>;
type GhidraProviderClientInstance = ReturnType<GhidraProviderClientFactory>;
type NativeAotSnapshotOwner = {
readonly derive: (
imageBase: string,
operation: AnalysisOperation,
callerSignal: AbortSignal | undefined,
observations: JsonValue,
) => Promise<Result<NativeAotPeResult, AnalysisError>>;
readonly readTypeDetail: (
address: string,
) => ReturnType<NativeAotPeResult["readTypeDetail"]>;
readonly close: () => void;
};
/** Own one session's captured PE snapshot and immutable NativeAOT derivation. */
const createNativeAotSnapshotOwner = (input: {
readonly client: GhidraProviderClientInstance;
readonly target: BinaryTarget;
readonly startupTimeoutMs: number;
}): NativeAotSnapshotOwner => {
const { client, target, startupTimeoutMs } = input;
const controller = new AbortController();
let identity: string | undefined;
let result: NativeAotPeResult | undefined;
let pending:
| {
readonly identity: string;
readonly promise: Promise<Result<NativeAotPeResult, AnalysisError>>;
}
| undefined;
let closed = false;
let identityConflict = false;
let identityConflictObservation:
| { readonly imageBase: string; readonly observations: JsonValue }
| undefined;
const identityError = (
operation: AnalysisOperation,
reason: string,
observedBase: string,
observations: JsonValue,
): AnalysisError =>
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason,
artifact_sha256: target.sha256,
bound_image_base:
identity === undefined
? null
: `0x${identity.slice(target.sha256.length + 1)}`,
observed_image_base: observedBase,
measured_observation: observations,
conflicting_image_base: identityConflictObservation?.imageBase ?? null,
conflicting_observation:
identityConflictObservation?.observations ?? null,
},
});
const identityConflictFailure = (
operation: AnalysisOperation,
observedBase: string,
observations: JsonValue,
): Result<NativeAotPeResult, AnalysisError> => {
return err(
identityError(
operation,
"The Ghidra session reported conflicting loaded image bases; reopen the analysis session to establish a new identity.",
observedBase,
observations,
),
);
};
const derive = async (
imageBase: string,
operation: AnalysisOperation,
callerSignal: AbortSignal | undefined,
observations: JsonValue,
): Promise<Result<NativeAotPeResult, AnalysisError>> => {
if (callerSignal?.aborted || closed)
return err(new AnalysisCancelledError(operation));
if (identityConflict)
return identityConflictFailure(operation, imageBase, observations);
if (controller.signal.aborted)
return err(new AnalysisCancelledError(operation));
let normalizedBase: string;
try {
normalizedBase = BigInt(imageBase).toString(16);
} catch (cause: unknown) {
return err(
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason:
"Ghidra returned an invalid loaded image base for NativeAOT identity binding.",
artifact_sha256: target.sha256,
observed_image_base: imageBase,
measured_observation: observations,
},
cause,
}),
);
}
const key = `${target.sha256}:${normalizedBase}`;
if (identity !== undefined && identity !== key) {
identityConflict = true;
identityConflictObservation = { imageBase, observations };
result = undefined;
controller.abort();
return err(
identityError(
operation,
"The Ghidra session reported a different loaded image base after NativeAOT metadata was bound to its captured snapshot; reopen the analysis session to establish a new identity.",
imageBase,
observations,
),
);
}
identity ??= key;
if (result !== undefined) {
const settled = await waitForAbortable(
Promise.resolve(ok(result)),
callerSignal,
);
if (closed) return err(new AnalysisCancelledError(operation));
if (identityConflict)
return identityConflictFailure(operation, imageBase, observations);
if (controller.signal.aborted)
return err(new AnalysisCancelledError(operation));
return settled === ABORTED
? err(new AnalysisCancelledError(operation))
: settled;
}
if (pending !== undefined && pending.identity !== key)
return err(
identityError(
operation,
"A NativeAOT snapshot derivation is already bound to a different loaded image base.",
imageBase,
observations,
),
);
if (pending === undefined) {
const promise: Promise<Result<NativeAotPeResult, AnalysisError>> =
(async () => {
try {
if (client.readTargetSnapshot === undefined)
return err(
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason:
"The Ghidra client does not expose its immutable target snapshot for NativeAOT metadata inspection.",
},
}),
);
const snapshot = await client.readTargetSnapshot(
NATIVE_AOT_MAX_SOURCE_BYTES,
controller.signal,
);
if (!snapshot.ok)
return err(
projectSessionError(
operation,
snapshot.error,
startupTimeoutMs,
),
);
if (closed) return err(new AnalysisCancelledError(operation));
if (identityConflict)
return identityConflictFailure(
operation,
imageBase,
observations,
);
if (controller.signal.aborted)
return err(new AnalysisCancelledError(operation));
const parsed =
snapshot.value.kind === "over-capacity"
? {
summary: nativeAotSnapshotOverCapacity(
target.sha256,
snapshot.value.sourceBytesAtLeast,
NATIVE_AOT_MAX_REPORT_BYTES,
),
readTypeDetail: () => undefined,
}
: parseNativeAotPe(
snapshot.value.bytes,
target.sha256,
NATIVE_AOT_MAX_REPORT_BYTES,
imageBase,
);
if (closed) return err(new AnalysisCancelledError(operation));
if (identityConflict)
return identityConflictFailure(
operation,
imageBase,
observations,
);
if (controller.signal.aborted)
return err(new AnalysisCancelledError(operation));
return ok(parsed);
} catch (cause: unknown) {
return err(
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason:
cause instanceof Error
? cause.message
: "NativeAOT metadata could not be derived from the captured target snapshot.",
},
cause,
}),
);
}
})();
const current = { identity: key, promise };
pending = current;
void promise.then((settled) => {
if (pending !== current) return;
pending = undefined;
if (settled.ok && !closed && !identityConflict) result = settled.value;
});
}
const settled = await waitForAbortable(pending.promise, callerSignal);
if (closed) return err(new AnalysisCancelledError(operation));
if (identityConflict)
return identityConflictFailure(operation, imageBase, observations);
if (controller.signal.aborted)
return err(new AnalysisCancelledError(operation));
return settled === ABORTED
? err(new AnalysisCancelledError(operation))
: settled;
};
return {
derive,
readTypeDetail: (address) =>
result?.readTypeDetail(address, NATIVE_AOT_MAX_REPORT_BYTES),
close: () => {
closed = true;
controller.abort();
result = undefined;
pending = undefined;
},
};
};
/** Build one AnalysisClient for an admitted Ghidra target and profile. */
export const createGhidraProviderClient = (input: {
readonly config: AppConfig;
@@ -258,6 +514,14 @@ export const createGhidraProviderClient = (input: {
...seedLimitations,
...(releaseLimitation === undefined ? [] : [releaseLimitation]),
];
const nativeAotOwner =
target.format === "pe"
? createNativeAotSnapshotOwner({
client,
target,
startupTimeoutMs: config.ghidraStartupTimeoutMs,
})
: undefined;
return {
execute: async (operation, parameters, options) => {
if (extensionFailure !== undefined) return err(extensionFailure);
@@ -334,19 +598,83 @@ export const createGhidraProviderClient = (input: {
if (!result.ok) return result;
normalized = result.value;
}
if (operation === "inspect_native_data_type") {
const address = input.value.address;
if (typeof address === "string") {
if (target.format === "pe" && nativeAotOwner !== undefined) {
const started = await client.start(options?.signal);
if (!started.ok)
return err(
projectSessionError(
operation,
started.error,
config.ghidraStartupTimeoutMs,
),
);
const derived = await nativeAotOwner.derive(
started.value.target.image_base,
operation,
options?.signal,
normalized,
);
if (!derived.ok) return derived;
}
let metadata:
| ReturnType<NativeAotPeResult["readTypeDetail"]>
| undefined;
metadata = nativeAotOwner?.readTypeDetail(address);
if (
metadata !== undefined &&
normalized !== null &&
typeof normalized === "object" &&
!Array.isArray(normalized)
) {
const nativeResult = normalized as Record<string, JsonValue>;
if (nativeResult.metadata_recovery === undefined)
normalized = jsonValueSchema.parse({
...nativeResult,
metadata_recovery: metadata,
});
}
}
}
if (operation === "inspect_native_load_image") {
const attested = await attestGhidraNativeLoadImage(
let derived: NativeAotPeResult | undefined;
if (target.format === "pe") {
const observations =
nativeLoadImageObservationSchema.parse(normalized);
if (nativeAotOwner === undefined)
return err(
new ProviderAdapterError("ghidra", operation, {
diagnostics: {
reason:
"The admitted PE session has no read-only NativeAOT snapshot owner.",
},
}),
);
const result = await nativeAotOwner.derive(
observations.image_base,
operation,
options?.signal,
normalized,
);
if (!result.ok) return result;
derived = result.value;
}
const attested = await attestGhidraNativeLoadImage({
target,
operation,
normalized,
measured: normalized,
client,
(failure) =>
mapSessionError: (failure) =>
projectSessionError(
operation,
failure,
config.ghidraStartupTimeoutMs,
),
);
nativeAotResult: derived,
signal: options?.signal,
});
if (!attested.ok) return attested;
normalized = attested.value;
}
@@ -368,7 +696,10 @@ export const createGhidraProviderClient = (input: {
? []
: [{ provider: committedProfile.provider, observation }];
},
close: () => client.close(),
close: async () => {
nativeAotOwner?.close();
return client.close();
},
};
};
+33 -7
View File
@@ -1,27 +1,53 @@
import type { FileHandle } from "node:fs/promises";
/** Read through EOF, returning undefined on overflow; consume at most maxBytes + 1 bytes without closing the handle. */
/** Read through EOF, returning undefined on overflow; leave the handle open. */
export const readBoundedFileBytes = async (
handle: FileHandle,
maxBytes: number,
signal?: AbortSignal,
expectedSize?: number,
): Promise<Buffer | undefined> => {
if (!Number.isSafeInteger(maxBytes) || maxBytes < 0)
throw new RangeError("File byte limit must be a nonnegative safe integer");
const chunks: Buffer[] = [];
const buffer = Buffer.allocUnsafe(Math.min(64 * 1_024, maxBytes + 1));
if (
expectedSize !== undefined &&
(!Number.isSafeInteger(expectedSize) ||
expectedSize < 0 ||
expectedSize > maxBytes)
)
throw new RangeError("Expected file size must fit the byte limit");
if (expectedSize === undefined) {
const chunks: Buffer[] = [];
const buffer = Buffer.allocUnsafe(Math.min(64 * 1_024, maxBytes + 1));
let total = 0;
for (;;) {
signal?.throwIfAborted();
const { bytesRead } = await handle.read(
buffer,
0,
Math.min(buffer.length, maxBytes - total + 1),
null,
);
if (bytesRead === 0) return Buffer.concat(chunks, total);
total += bytesRead;
if (total > maxBytes) return undefined;
chunks.push(Buffer.from(buffer.subarray(0, bytesRead)));
}
}
const buffer = Buffer.allocUnsafe(expectedSize);
if (expectedSize === 0) return buffer;
let total = 0;
for (;;) {
signal?.throwIfAborted();
const { bytesRead } = await handle.read(
buffer,
0,
Math.min(buffer.length, maxBytes - total + 1),
total,
expectedSize - total,
null,
);
if (bytesRead === 0) return Buffer.concat(chunks, total);
if (bytesRead === 0) return buffer.subarray(0, total);
total += bytesRead;
if (total > maxBytes) return undefined;
chunks.push(Buffer.from(buffer.subarray(0, bytesRead)));
if (total === expectedSize) return buffer;
}
};
@@ -146,7 +146,7 @@ afterEach(async () => {
await Promise.all(clients.splice(0).map((client) => client.close()));
});
describe("GhidraClient", () => {
describe("GhidraClient captured source snapshot", () => {
it("exposes the immutable import snapshot only during the live session", async () => {
const client = clientFor(new FixtureLauncher());
await expect(client.readTargetSnapshot()).resolves.toMatchObject({
@@ -154,16 +154,43 @@ describe("GhidraClient", () => {
error: { kind: "protocol" },
});
await expect(client.start()).resolves.toMatchObject({ ok: true });
const abortedRead = new AbortController();
abortedRead.abort();
await expect(
client.readTargetSnapshot(1024, abortedRead.signal),
).resolves.toMatchObject({ ok: false, error: { kind: "cancelled" } });
const snapshot = await client.readTargetSnapshot();
expect(snapshot.ok).toBe(true);
if (snapshot.ok) expect(snapshot.value).toEqual(readFileSync(fixturePath));
const expected = readFileSync(fixturePath);
if (snapshot.ok)
expect(snapshot.value).toEqual({
kind: "captured",
bytes: expected,
});
await expect(
client.readTargetSnapshot(expected.length),
).resolves.toMatchObject({
ok: true,
value: { kind: "captured", bytes: expected },
});
const bounded = await client.readTargetSnapshot(0);
expect(bounded).toMatchObject({
ok: true,
value: {
kind: "over-capacity",
maximumBytes: 0,
sourceBytesAtLeast: readFileSync(fixturePath).length,
},
});
await client.close();
await expect(client.readTargetSnapshot()).resolves.toMatchObject({
ok: false,
error: { kind: "protocol" },
});
});
});
describe("GhidraClient", () => {
it("completes an exact, fragmented post-analysis handshake", async () => {
const launcher = new FixtureLauncher("fragmented");
const client = clientFor(launcher);
+91
View File
@@ -0,0 +1,91 @@
import { createHash } from "node:crypto";
const IMAGE_BASE = 0x140000000n;
const address = (rva: number): bigint => IMAGE_BASE + BigInt(rva);
export const nativeAotPeDigest = (bytes: Buffer): string =>
createHash("sha256").update(bytes).digest("hex");
export const nativeAotPeFixture = (major = 9, minor = 1): Buffer => {
const bytes = Buffer.alloc(0xa00);
bytes.write("MZ", 0, "ascii");
bytes.writeUInt32LE(0x80, 0x3c);
bytes.write("PE\0\0", 0x80, "ascii");
bytes.writeUInt16LE(0x8664, 0x84);
bytes.writeUInt16LE(2, 0x86);
bytes.writeUInt16LE(0xf0, 0x94);
const optional = 0x98;
bytes.writeUInt16LE(0x20b, optional);
bytes.writeBigUInt64LE(IMAGE_BASE, optional + 24);
bytes.writeUInt32LE(0x3000, optional + 56);
bytes.writeUInt32LE(0x400, optional + 60);
const sectionTable = optional + 0xf0;
bytes.write(".text", sectionTable, "ascii");
bytes.writeUInt32LE(0x200, sectionTable + 8);
bytes.writeUInt32LE(0x1000, sectionTable + 12);
bytes.writeUInt32LE(0x200, sectionTable + 16);
bytes.writeUInt32LE(0x400, sectionTable + 20);
bytes.writeUInt32LE(0x60000020, sectionTable + 36);
const data = sectionTable + 40;
bytes.write(".rdata", data, "ascii");
bytes.writeUInt32LE(0x400, data + 8);
bytes.writeUInt32LE(0x2000, data + 12);
bytes.writeUInt32LE(0x400, data + 16);
bytes.writeUInt32LE(0x600, data + 20);
bytes.writeUInt32LE(0x40000040, data + 36);
bytes.writeUInt8(0xc3, 0x410);
bytes.writeUInt8(0xc3, 0x420);
bytes.writeUInt8(0xc3, 0x430);
const header = 0x600;
bytes.writeUInt32LE(0x00525452, header);
bytes.writeUInt16LE(major, header + 4);
bytes.writeUInt16LE(minor, header + 6);
bytes.writeUInt16LE(2, header + 12);
bytes.writeUInt8(24, header + 14);
bytes.writeUInt8(1, header + 15);
bytes.writeUInt32LE(207, header + 16);
bytes.writeUInt32LE(1, header + 20);
bytes.writeBigUInt64LE(address(0x2100), header + 24);
bytes.writeBigUInt64LE(address(0x2105), header + 32);
bytes.writeUInt32LE(206, header + 40);
bytes.writeBigUInt64LE(address(0x2300), header + 48);
bytes.writeBigUInt64LE(address(0x2380), header + 56);
bytes.writeInt32LE(0x280, 0x700);
bytes.writeUInt8(0x09, 0x704);
const writeMethodTable = (
rva: number,
flags: number,
baseSize: number,
related: bigint,
slots: readonly bigint[],
) => {
const offset = 0x600 + rva - 0x2000;
bytes.writeUInt32LE(flags, offset);
bytes.writeUInt32LE(baseSize, offset + 4);
bytes.writeBigUInt64LE(related, offset + 8);
bytes.writeUInt16LE(slots.length, offset + 16);
bytes.writeUInt16LE(0, offset + 18);
bytes.writeUInt32LE(0x1234, offset + 20);
slots.forEach((slot, index) =>
bytes.writeBigUInt64LE(slot, offset + 24 + index * 8),
);
};
const object = address(0x2200);
writeMethodTable(0x2200, 0x50000000, 0x18, 0n, [
address(0x1010),
address(0x1020),
address(0x1030),
]);
writeMethodTable(0x2280, 0x50000000, 0x20, object, [address(0x1040)]);
writeMethodTable(0x22c0, 0x50000000, 0x16, object, []);
const frozen = 0x600 + 0x2300 - 0x2000;
bytes.writeBigUInt64LE(address(0x22c0), frozen);
const value = Buffer.from("REA_NATIVEAOT_FROZEN", "utf16le");
bytes.writeUInt32LE(value.length / 2, frozen + 8);
value.copy(bytes, frozen + 12);
bytes.writeUInt16LE(0, frozen + 12 + value.length);
return bytes;
};