mirror of
https://github.com/morluto/rea.git
synced 2026-10-12 02:19:24 +00:00
feat(javascript): recover code-split bundles from multiple inputs (#1686)
* feat(javascript): recover code-split bundles from multiple inputs recover_javascript_sources accepted a single path, and validateModuleSet bound every module's provenance to that one snapshot. A code-split app (runtime + chunks, or an Electron main/preload/renderer set) had to be recovered one file per call, and modules recovered alone could not resolve imports across chunks: cross-chunk require(id) references stayed as numeric ids with property reads. Add an optional additional_paths input. The listed files are concatenated, in order after path, into the single engine snapshot, so one Wakaru run sees every module and cross-chunk imports resolve. validateModuleSet's single-snapshot binding is unchanged. Module provenance is attributed back to the input file its first extraction range starts in, with byte ranges rebased to that file's own offsets; a module whose ranges cross an input boundary keeps the concatenated snapshot identity and offsets, recorded as an explicit limitation. Single-path behavior is byte-for-byte unchanged (one segment, offset 0). Fixes #1605 * fix(javascript): preserve source identity in multi-input recovery --------- Co-authored-by: JingHao-Leon <[email protected]> Co-authored-by: morluto <[email protected]>
This commit is contained in:
co-authored by
JingHao-Leon
morluto
parent
ce845ba428
commit
12c416005f
@@ -1,5 +1,13 @@
|
||||
# JavaScript and Electron application artifacts
|
||||
|
||||
For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`recover_javascript_sources` with an absolute `path` and absent
|
||||
`output_directory`. Supply related runtime/shared/lazy files in
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When size is unknown or large and the connected schema advertises
|
||||
`detail`, request `detail: "summary"`. This returns an `inspect_analysis_view`
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# JavaScript source recovery
|
||||
|
||||
`recover_javascript_sources` / `recover-javascript-sources` derives readable
|
||||
modules from one explicitly selected UTF-8 JavaScript file or bundle. Use it on
|
||||
modules from explicitly selected UTF-8 JavaScript files or bundles. Use it on
|
||||
files returned by `export_web_scripts` or other local scripts. The operation
|
||||
never executes the selected or recovered application code.
|
||||
|
||||
@@ -52,35 +52,58 @@ Both give the same binary (SHA-256
|
||||
```sh
|
||||
rea recover-javascript-sources /tmp/web-scripts/app.js /tmp/recovered-app --json
|
||||
rea analyze-javascript-application /tmp/recovered-app/modules --json
|
||||
|
||||
# Recover related code-split chunks together; repeat --additional-path.
|
||||
rea recover-javascript-sources /tmp/web-scripts/main.js /tmp/recovered-chunks \
|
||||
--additional-path /tmp/web-scripts/runtime.js \
|
||||
--additional-path /tmp/web-scripts/shared.js \
|
||||
--additional-path /tmp/web-scripts/checkout.chunk.js --json
|
||||
```
|
||||
|
||||
MCP input:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/tmp/web-scripts/app.js",
|
||||
"output_directory": "/tmp/recovered-app"
|
||||
"path": "/tmp/web-scripts/main.js",
|
||||
"additional_paths": [
|
||||
"/tmp/web-scripts/runtime.js",
|
||||
"/tmp/web-scripts/shared.js",
|
||||
"/tmp/web-scripts/checkout.chunk.js"
|
||||
],
|
||||
"output_directory": "/tmp/recovered-chunks"
|
||||
}
|
||||
```
|
||||
|
||||
The output directory must be absent, absolute, and have an existing parent.
|
||||
The source is snapshotted in a private workspace, and the engine writes only to
|
||||
Each source is snapshotted separately in a private workspace. Multiple inputs
|
||||
retain their relative directory layout and are passed together to one Wakaru
|
||||
invocation, allowing cross-chunk module IDs to resolve. Inputs must be distinct;
|
||||
`additional_paths` accepts up to 16 additional files. The engine writes only to
|
||||
its own staging directory. Reported module paths, input identities, UTF-8 byte
|
||||
ranges, regular files and emitted map representations are validated before
|
||||
publication. REA writes verified bytes to an exclusively owned output tree and
|
||||
rolls it back on failure. It also checks that the original source and engine
|
||||
bytes remain unchanged before committing. The existing output is never overwritten.
|
||||
|
||||
Multi-input relative filenames must be representable unchanged as artifact paths:
|
||||
NFC Unicode spelling and no literal backslashes. Unsupported layouts fail before
|
||||
invoking the engine rather than silently renaming inputs referenced by source maps.
|
||||
|
||||
Published contents:
|
||||
|
||||
- `inputs/bundle.js`: exact original snapshot; its digest binds the Evidence.
|
||||
- `inputs/bundle.js`: exact original snapshot for a single input. Multiple inputs
|
||||
retain paths relative to their common parent under `inputs/`.
|
||||
- `modules/`: derived modules and any upstream-emitted source maps.
|
||||
- `reports/stdout.json` and `reports/provenance.json`: exact producer reports.
|
||||
- `manifest.json`: identities, modules, transform choices, warnings and limitations.
|
||||
|
||||
The inline result includes complete file paths/digests/sizes, extraction ranges,
|
||||
warnings and `analysis_input`, directly usable as the arguments to
|
||||
`analyze_javascript_application`. Raw Evidence retains selected command arguments,
|
||||
`analyze_javascript_application`. `source` identifies the primary input and binds
|
||||
the Evidence subject; `additional_sources` records the other snapshots and their
|
||||
published copies when selected. Each module names its owning snapshot, original
|
||||
path and original digest; all extraction and context ranges refer to that input.
|
||||
Raw Evidence retains selected command arguments,
|
||||
reported input paths, exit status, stderr and reports; temporary paths are recorded
|
||||
as historical execution coordinates after their workspace is removed.
|
||||
|
||||
@@ -107,7 +130,8 @@ equivalence remain unknown. No network fetching occurs.
|
||||
## Resource policy and verification
|
||||
|
||||
Each operation uses one worker, a 1 GiB address-space ceiling and a 120-second
|
||||
deadline; it accepts up to 64 MiB of input and 128 MiB/10000 entries of staged
|
||||
deadline; it accepts up to 64 MiB of total input across all selected files and
|
||||
128 MiB/10000 entries of staged
|
||||
output, with 8 MiB per diagnostic/report stream. Exceeding a limit is an explicit
|
||||
failure with rollback. These limits apply per operation and do not claim aggregate
|
||||
host containment across concurrent callers.
|
||||
|
||||
+4
-1
@@ -1095,7 +1095,10 @@ and `REA_JAVASCRIPT_FIXTURE_TOOLS` pointing to an isolated npm prefix containing
|
||||
esbuild 0.25.10 and webpack 5.101.3. No global installation is required.
|
||||
The lane compiles source-owned fixtures, exercises CLI and stdio MCP, verifies
|
||||
published bytes and provenance, feeds recovered modules into existing analysis,
|
||||
and compares a finite set of known fixture results. The provider also accepts
|
||||
and compares a finite set of known fixture results. A webpack fixture with separate
|
||||
runtime, shared, entry and lazy chunks verifies multi-input recovery through both
|
||||
surfaces, per-file provenance and resolved cross-chunk imports in the downstream
|
||||
application graph. The provider also accepts
|
||||
other ^1.13.0 releases and reports them as unverified; this lane proves the
|
||||
audited release. It does not establish
|
||||
arbitrary recovered-application equivalence. CI installs these prerequisites only
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { createRequire } from "node:module";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { mkdir, readFile, readdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
@@ -20,23 +20,56 @@ export async function buildRecoveryFixtures(root, prefix) {
|
||||
const source = fileURLToPath(new URL("./entry.mjs", import.meta.url));
|
||||
const webpackOutput = join(root, "webpack");
|
||||
await mkdir(webpackOutput);
|
||||
await new Promise((resolve, reject) => {
|
||||
const compiler = webpack({
|
||||
mode: "production",
|
||||
entry: source,
|
||||
parallelism: 1,
|
||||
optimization: { concatenateModules: false },
|
||||
output: { path: webpackOutput, filename: "bundle.js" },
|
||||
});
|
||||
compiler.run((error, stats) => {
|
||||
compiler.close((closeError) => {
|
||||
if (error || closeError) return reject(error ?? closeError);
|
||||
if (!stats || stats.hasErrors())
|
||||
return reject(new Error(stats?.toString() ?? "No webpack stats"));
|
||||
resolve();
|
||||
const compileWebpack = async (options) =>
|
||||
new Promise((resolve, reject) => {
|
||||
const compiler = webpack(options);
|
||||
compiler.run((error, stats) => {
|
||||
compiler.close((closeError) => {
|
||||
if (error || closeError) return reject(error ?? closeError);
|
||||
if (!stats || stats.hasErrors())
|
||||
return reject(new Error(stats?.toString() ?? "No webpack stats"));
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
});
|
||||
await compileWebpack({
|
||||
mode: "production",
|
||||
entry: source,
|
||||
parallelism: 1,
|
||||
optimization: { concatenateModules: false },
|
||||
output: { path: webpackOutput, filename: "bundle.js" },
|
||||
});
|
||||
const splitOutput = join(root, "webpack-code-split");
|
||||
await mkdir(splitOutput);
|
||||
const splitSource = fileURLToPath(
|
||||
new URL("./code-split/entry.mjs", import.meta.url),
|
||||
);
|
||||
await compileWebpack({
|
||||
mode: "production",
|
||||
entry: splitSource,
|
||||
parallelism: 1,
|
||||
optimization: {
|
||||
concatenateModules: false,
|
||||
runtimeChunk: "single",
|
||||
splitChunks: {
|
||||
chunks: "all",
|
||||
minSize: 0,
|
||||
cacheGroups: {
|
||||
shared: { test: /shared\.mjs$/u, name: "shared", enforce: true },
|
||||
},
|
||||
},
|
||||
},
|
||||
output: {
|
||||
path: splitOutput,
|
||||
filename: "[name].js",
|
||||
chunkFilename: "[id].chunk.js",
|
||||
},
|
||||
});
|
||||
const splitFiles = (await readdir(splitOutput))
|
||||
.filter((name) => name !== "main.js")
|
||||
.sort();
|
||||
if (splitFiles.length !== 3)
|
||||
throw new Error("Expected runtime, shared and lazy chunks");
|
||||
const esbuildPath = join(root, "esbuild.js");
|
||||
await esbuild.build({
|
||||
entryPoints: [source],
|
||||
@@ -95,6 +128,13 @@ export async function buildRecoveryFixtures(root, prefix) {
|
||||
options: { extraction_mode: "inspection", rewrite_level: "minimal" },
|
||||
},
|
||||
],
|
||||
codeSplit: {
|
||||
name: "webpack-code-split",
|
||||
path: join(splitOutput, "main.js"),
|
||||
additional_paths: splitFiles.map((name) => join(splitOutput, name)),
|
||||
expectedFormat: "webpack5",
|
||||
expectedModules: 4,
|
||||
},
|
||||
toolchains: { esbuild: esbuild.version, webpack: webpackVersion },
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
import { Cart, formatPrice, TAX_RATE } from "./shared.mjs";
|
||||
export function checkout(prices) {
|
||||
const cart = new Cart();
|
||||
prices.forEach((price) => cart.add(price));
|
||||
return formatPrice(cart.total() * (1 + TAX_RATE));
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
import { formatPrice } from "./shared.mjs";
|
||||
console.log(formatPrice(1));
|
||||
import("./checkout.mjs").then(({ checkout }) => console.log(checkout([1, 2])));
|
||||
@@ -0,0 +1,14 @@
|
||||
export class Cart {
|
||||
constructor() {
|
||||
this.items = [];
|
||||
}
|
||||
add(price) {
|
||||
this.items.push(price);
|
||||
return this;
|
||||
}
|
||||
total() {
|
||||
return this.items.reduce((sum, price) => sum + price, 0);
|
||||
}
|
||||
}
|
||||
export const formatPrice = (value) => `$${value.toFixed(2)}`;
|
||||
export const TAX_RATE = 0.08;
|
||||
@@ -56,7 +56,7 @@ try {
|
||||
process.env.REA_JAVASCRIPT_FIXTURE_TOOLS,
|
||||
);
|
||||
await client.connect(transport);
|
||||
for (const fixture of fixtures.files) {
|
||||
for (const fixture of [...fixtures.files, fixtures.codeSplit]) {
|
||||
for (const surface of ["cli", "stdio_mcp"]) {
|
||||
const output = join(root, `${fixture.name}-${surface}`);
|
||||
const flags =
|
||||
@@ -75,11 +75,18 @@ try {
|
||||
fixture.path,
|
||||
output,
|
||||
...flags,
|
||||
...(fixture.additional_paths ?? []).flatMap((path) => [
|
||||
"--additional-path",
|
||||
path,
|
||||
]),
|
||||
)
|
||||
: await call("recover_javascript_sources", {
|
||||
path: fixture.path,
|
||||
output_directory: output,
|
||||
...fixture.options,
|
||||
...(fixture.additional_paths
|
||||
? { additional_paths: fixture.additional_paths }
|
||||
: {}),
|
||||
});
|
||||
const result = javascriptRecoveryResultSchema.parse(
|
||||
evidence.normalized_result,
|
||||
@@ -97,6 +104,25 @@ try {
|
||||
analysis.normalized_result.statistics.parsed_javascript_files,
|
||||
result.modules.length,
|
||||
);
|
||||
const imports = analysis.normalized_result.graph.edges.filter(
|
||||
(edge) =>
|
||||
edge.relation === "imports" &&
|
||||
edge.properties.resolution_status === "resolved",
|
||||
);
|
||||
if (fixture.additional_paths) {
|
||||
assert.ok(
|
||||
imports.length >= 4,
|
||||
"Expected cross-chunk imports in the recovered application graph",
|
||||
);
|
||||
const recoveredPaths = new Set(
|
||||
result.modules.map((module) => module.reported_filename),
|
||||
);
|
||||
for (const edge of imports)
|
||||
assert.ok(
|
||||
recoveredPaths.has(edge.properties.resolved_path),
|
||||
`Expected an emitted import target: ${String(edge.properties.resolved_path)}`,
|
||||
);
|
||||
}
|
||||
const recoveredEntry =
|
||||
result.modules.find(
|
||||
(module) => module.reported_filename === "entry.js",
|
||||
@@ -104,7 +130,8 @@ try {
|
||||
assert.ok(recoveredEntry);
|
||||
const oracle =
|
||||
'await import((await import("node:url")).pathToFileURL(process.argv[1])); const f=globalThis.recoveryFixture; console.log(JSON.stringify([f.greet(null),f.greet("REA"),f.total([1,2,3])]));';
|
||||
for (const selected of fixture.options?.extraction_mode === "inspection"
|
||||
for (const selected of fixture.additional_paths ||
|
||||
fixture.options?.extraction_mode === "inspection"
|
||||
? []
|
||||
: [fixture.path, recoveredEntry.artifact.path]) {
|
||||
const execution = await run(
|
||||
@@ -125,7 +152,9 @@ try {
|
||||
maps: result.modules.filter((module) => module.source_map !== null)
|
||||
.length,
|
||||
finite_fixture_oracle:
|
||||
!fixture.additional_paths &&
|
||||
fixture.options?.extraction_mode !== "inspection",
|
||||
cross_chunk_imports: fixture.additional_paths ? imports.length : null,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -157,9 +186,15 @@ async function assertRecoveredArtifacts(evidence, result, fixture) {
|
||||
assert.equal(result.status, "complete");
|
||||
assert.equal(result.reported_format, fixture.expectedFormat);
|
||||
assert.equal(result.modules.length, fixture.expectedModules);
|
||||
assert.equal(result.source.sha256, fingerprint(await readFile(fixture.path)));
|
||||
const sources = [result.source, ...(result.additional_sources ?? [])];
|
||||
assert.deepEqual(
|
||||
sources.map((source) => source.path),
|
||||
[fixture.path, ...(fixture.additional_paths ?? [])],
|
||||
);
|
||||
for (const source of sources)
|
||||
assert.equal(source.sha256, fingerprint(await readFile(source.path)));
|
||||
for (const file of [
|
||||
result.source.published_copy,
|
||||
...sources.map((source) => source.published_copy),
|
||||
result.report,
|
||||
result.provenance,
|
||||
result.manifest,
|
||||
@@ -173,16 +208,27 @@ async function assertRecoveredArtifacts(evidence, result, fixture) {
|
||||
assert.equal(fingerprint(bytes), file.sha256);
|
||||
}
|
||||
for (const module of result.modules) {
|
||||
assert.equal(module.provenance.original_sha256, result.source.sha256);
|
||||
assert.equal(module.provenance.reported_input, result.source.snapshot_path);
|
||||
for (const range of module.provenance.byte_ranges)
|
||||
assert.ok(range.start <= range.end && range.end <= result.source.bytes);
|
||||
const source = sources.find(
|
||||
(source) => source.snapshot_path === module.provenance.reported_input,
|
||||
);
|
||||
assert.ok(source, "Module must name a selected input snapshot");
|
||||
assert.equal(module.provenance.original_path, source.path);
|
||||
assert.equal(module.provenance.original_sha256, source.sha256);
|
||||
for (const range of [
|
||||
...module.provenance.byte_ranges,
|
||||
...module.provenance.context_byte_ranges,
|
||||
])
|
||||
assert.ok(range.start <= range.end && range.end <= source.bytes);
|
||||
if (module.source_map) {
|
||||
const map = JSON.parse(await readFile(module.source_map.path, "utf8"));
|
||||
for (const source of map.sources)
|
||||
assert.equal(
|
||||
resolve(dirname(module.source_map.path), source),
|
||||
result.source.published_copy.path,
|
||||
assert.ok(
|
||||
sources.some(
|
||||
(selected) =>
|
||||
selected.published_copy.path ===
|
||||
resolve(dirname(module.source_map.path), source),
|
||||
),
|
||||
"Map source must name a published input snapshot",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -129,4 +129,50 @@ describe("JavaScript recovery workflow", () => {
|
||||
if (cancelled.ok) throw new Error("expected cancellation");
|
||||
expect(cancelled.error._tag).toBe("AnalysisCancelledError");
|
||||
});
|
||||
it("rejects additional_paths repeating path or each other", async () => {
|
||||
const service = new JavaScriptRecoveryService({
|
||||
recover: async () => {
|
||||
throw new Error("port must not execute");
|
||||
},
|
||||
});
|
||||
for (const additional_paths of [
|
||||
[artifact.path],
|
||||
["/tmp/dir/../bundle.js", "/tmp/bundle.js"],
|
||||
["/tmp/shared.js", "/tmp/shared.js"],
|
||||
]) {
|
||||
const result = await service.recover({
|
||||
...input,
|
||||
additional_paths,
|
||||
});
|
||||
if (result.ok) throw new Error("expected duplicate rejection");
|
||||
expect(result.error._tag).toBe("AnalysisInputError");
|
||||
expect(result.error).toMatchObject({
|
||||
issues: [{ path: ["additional_paths"], reason: "invalid_format" }],
|
||||
});
|
||||
}
|
||||
});
|
||||
it("rejects relative additional_paths before acquiring the port", async () => {
|
||||
let invoked = false;
|
||||
const service = new JavaScriptRecoveryService({
|
||||
recover: async () => {
|
||||
invoked = true;
|
||||
return err(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
"fixture",
|
||||
"recover_javascript_sources",
|
||||
"engine missing",
|
||||
),
|
||||
);
|
||||
},
|
||||
});
|
||||
const result = await service.recover({
|
||||
...input,
|
||||
additional_paths: ["relative-chunk.js"],
|
||||
});
|
||||
expect(invoked).toBe(false);
|
||||
if (result.ok) throw new Error("expected invalid path");
|
||||
expect(result.error).toMatchObject({
|
||||
issues: [{ path: ["additional_paths"], reason: "invalid_format" }],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { isAbsolute } from "node:path";
|
||||
import { isAbsolute, resolve } from "node:path";
|
||||
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
@@ -11,7 +11,7 @@ import {
|
||||
javascriptRecoveryInputSchema,
|
||||
javascriptRecoveryResultSchema,
|
||||
} from "../../domain/javascript/javascriptRecovery.js";
|
||||
import { jsonObjectSchema } from "../../domain/jsonValue.js";
|
||||
import { jsonObjectSchema, jsonValueSchema } from "../../domain/jsonValue.js";
|
||||
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
@@ -44,17 +44,38 @@ export class JavaScriptRecoveryService {
|
||||
const invalidPaths = (["path", "output_directory"] as const).filter(
|
||||
(field) => !isAbsolute(parsed.data[field]),
|
||||
);
|
||||
if (invalidPaths.length > 0)
|
||||
const invalidAdditional = (parsed.data.additional_paths ?? []).filter(
|
||||
(additional) => !isAbsolute(additional),
|
||||
);
|
||||
if (invalidPaths.length > 0 || invalidAdditional.length > 0)
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
OPERATION,
|
||||
undefined,
|
||||
invalidPaths.map((field) => ({
|
||||
new AnalysisInputError(OPERATION, undefined, [
|
||||
...invalidPaths.map((field) => ({
|
||||
path: [field],
|
||||
reason: "invalid_format",
|
||||
reason: "invalid_format" as const,
|
||||
message: `${field} must be an absolute filesystem path on this host.`,
|
||||
})),
|
||||
),
|
||||
...invalidAdditional.map((additional) => ({
|
||||
path: ["additional_paths"],
|
||||
reason: "invalid_format" as const,
|
||||
message: `additional_paths entries must be absolute filesystem paths on this host: ${additional}`,
|
||||
})),
|
||||
]),
|
||||
);
|
||||
const selectedInputs = new Set(
|
||||
[parsed.data.path, ...(parsed.data.additional_paths ?? [])].map((path) =>
|
||||
resolve(path),
|
||||
),
|
||||
);
|
||||
if (selectedInputs.size !== 1 + (parsed.data.additional_paths ?? []).length)
|
||||
return err(
|
||||
new AnalysisInputError(OPERATION, undefined, [
|
||||
{
|
||||
path: ["additional_paths"],
|
||||
reason: "invalid_format",
|
||||
message: "additional_paths must not repeat path or each other.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
const execution = await this.provider.recover(parsed.data, options);
|
||||
if (!execution.ok) return execution;
|
||||
@@ -70,6 +91,11 @@ export class JavaScriptRecoveryService {
|
||||
);
|
||||
if (
|
||||
result.data.source.path !== parsed.data.path ||
|
||||
(result.data.additional_sources ?? []).length !==
|
||||
(parsed.data.additional_paths ?? []).length ||
|
||||
(parsed.data.additional_paths ?? []).some(
|
||||
(path, index) => result.data.additional_sources?.[index]?.path !== path,
|
||||
) ||
|
||||
execution.value.subject.path !== result.data.source.path ||
|
||||
execution.value.subject.sha256 !== result.data.source.sha256 ||
|
||||
execution.value.provider.id !== result.data.engine.id ||
|
||||
@@ -85,7 +111,7 @@ export class JavaScriptRecoveryService {
|
||||
createEvidence(execution.value.subject, execution.value.provider, {
|
||||
operation: OPERATION,
|
||||
parameters: jsonObjectSchema.parse(parsed.data),
|
||||
result: result.data,
|
||||
result: jsonValueSchema.parse(result.data),
|
||||
rawResult: execution.value.rawResult,
|
||||
locations: execution.value.locations,
|
||||
limitations: execution.value.limitations,
|
||||
|
||||
@@ -27,6 +27,12 @@ export const registerJavaScriptRecoveryCommands = (
|
||||
outputDirectory: z.string().describe("Absolute absent output directory"),
|
||||
}),
|
||||
options: z.object({
|
||||
additionalPath: z
|
||||
.array(z.string())
|
||||
.optional()
|
||||
.describe(
|
||||
"Related absolute UTF-8 JavaScript file; repeat for multiple chunks",
|
||||
),
|
||||
extractionMode: z
|
||||
.enum(["structural", "heuristic", "inspection"])
|
||||
.optional()
|
||||
@@ -51,6 +57,9 @@ export const registerJavaScriptRecoveryCommands = (
|
||||
{
|
||||
path: args.path,
|
||||
output_directory: args.outputDirectory,
|
||||
...(options.additionalPath === undefined
|
||||
? {}
|
||||
: { additional_paths: options.additionalPath }),
|
||||
...(options.extractionMode === undefined
|
||||
? {}
|
||||
: { extraction_mode: options.extractionMode }),
|
||||
|
||||
@@ -13,7 +13,7 @@ export const JAVASCRIPT_RECOVERY_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("recover_javascript_sources"),
|
||||
kind: "application",
|
||||
description:
|
||||
"Recover readable modules from one local UTF-8 JavaScript script/bundle into an absent directory, using caller-supplied Wakaru ^1.13.0 (verified with 1.14.0) on Linux x64. Returns original/derived digests, extraction byte ranges, emitted source maps, warnings and analysis_input inline. Structural splitting may fall back to one module; original names and runtime equivalence remain unknown. No application code execution. Requires util-linux prlimit; one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB input, 128 MiB output, 10000 entries and 8 MiB per diagnostic/report stream.",
|
||||
"Recover readable modules from local UTF-8 JavaScript scripts/bundles into an absent directory, using caller-supplied Wakaru ^1.13.0 (verified with 1.14.0) on Linux x64. Use additional_paths for related chunks; each input retains its own snapshot and provenance. Returns original/derived digests, extraction byte ranges, emitted source maps, warnings and analysis_input inline. Structural splitting may fall back to one module; original names and runtime equivalence remain unknown. No application code execution. Requires util-linux prlimit; one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB total input, 128 MiB output, 10000 entries and 8 MiB per diagnostic/report stream.",
|
||||
inputSchema: javascriptRecoveryInputSchema,
|
||||
outputSchema: evidenceResultOf(javascriptRecoveryResultSchema),
|
||||
examples: [
|
||||
@@ -24,6 +24,18 @@ export const JAVASCRIPT_RECOVERY_TOOL_CONTRACTS = [
|
||||
output_directory: "/tmp/recovered-app",
|
||||
},
|
||||
},
|
||||
{
|
||||
title: "Recover a code-split bundle together",
|
||||
input: {
|
||||
path: "/tmp/web-scripts/main.js",
|
||||
additional_paths: [
|
||||
"/tmp/web-scripts/runtime.js",
|
||||
"/tmp/web-scripts/shared.js",
|
||||
"/tmp/web-scripts/checkout.chunk.js",
|
||||
],
|
||||
output_directory: "/tmp/recovered-chunks",
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
] as const satisfies readonly ToolContract[];
|
||||
|
||||
@@ -3,11 +3,18 @@ import { digestSchema } from "../digests.js";
|
||||
import { analyzeJavaScriptApplicationInputSchema } from "./javascriptApplicationAnalysis.js";
|
||||
import { localPathStringSchema } from "../localPath.js";
|
||||
|
||||
/** Recover derived readable modules from one explicitly selected local script. */
|
||||
/** Recover derived readable modules from explicitly selected local scripts. */
|
||||
export const javascriptRecoveryInputSchema = z.strictObject({
|
||||
path: localPathStringSchema.describe(
|
||||
"Absolute path to a UTF-8 JavaScript file; never executed",
|
||||
),
|
||||
additional_paths: z
|
||||
.array(localPathStringSchema)
|
||||
.max(16)
|
||||
.optional()
|
||||
.describe(
|
||||
"Additional absolute UTF-8 JavaScript files recovered together with path; each is snapshotted separately so imports can resolve across related chunks. All inputs share a 64 MiB total budget",
|
||||
),
|
||||
output_directory: localPathStringSchema.describe(
|
||||
"Absolute absent output directory; parent must exist",
|
||||
),
|
||||
@@ -36,12 +43,15 @@ const rangeSchema = z.strictObject({
|
||||
end: z.number().int().nonnegative(),
|
||||
});
|
||||
|
||||
const sourceSchema = fileSchema.extend({
|
||||
snapshot_path: z.string().min(1),
|
||||
published_copy: fileSchema,
|
||||
});
|
||||
|
||||
/** Complete published artifacts and exact producer provenance for source recovery. */
|
||||
export const javascriptRecoveryResultSchema = z.strictObject({
|
||||
source: fileSchema.extend({
|
||||
snapshot_path: z.string().min(1),
|
||||
published_copy: fileSchema,
|
||||
}),
|
||||
source: sourceSchema,
|
||||
additional_sources: z.array(sourceSchema).optional(),
|
||||
engine: z.strictObject({
|
||||
id: z.string().min(1),
|
||||
version: z.string().min(1),
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { constants } from "node:fs";
|
||||
import { lstat, open, opendir, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { lstat, mkdir, open, opendir, writeFile } from "node:fs/promises";
|
||||
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { TextDecoder } from "node:util";
|
||||
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
@@ -89,12 +90,22 @@ export const readRecoveryFile = async (
|
||||
return Buffer.concat(chunks, result.bytes);
|
||||
};
|
||||
|
||||
/** Snapshot one caller-selected UTF-8 script before invoking the engine. */
|
||||
export const snapshotRecoveryInput = async (
|
||||
export type RecoveryInputSnapshot = {
|
||||
path: string;
|
||||
snapshot_path: string;
|
||||
relative_path: string;
|
||||
sha256: string;
|
||||
bytes: number;
|
||||
};
|
||||
|
||||
/** Snapshot one script within the remaining operation-wide input budget. */
|
||||
const snapshotRecoveryInput = async (
|
||||
path: string,
|
||||
snapshotPath: string,
|
||||
relativePath: string,
|
||||
maximum: number,
|
||||
signal?: AbortSignal,
|
||||
) => {
|
||||
): Promise<RecoveryInputSnapshot> => {
|
||||
if (!isAbsolute(path))
|
||||
throw recoveryInputFailure(
|
||||
"path",
|
||||
@@ -102,7 +113,7 @@ export const snapshotRecoveryInput = async (
|
||||
);
|
||||
let bytes: Buffer;
|
||||
try {
|
||||
bytes = await readRecoveryFile(path, RECOVERY_LIMITS.inputBytes, signal);
|
||||
bytes = await readRecoveryFile(path, maximum, signal);
|
||||
new TextDecoder("utf-8", { fatal: true }).decode(bytes);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError) throw cause;
|
||||
@@ -112,15 +123,78 @@ export const snapshotRecoveryInput = async (
|
||||
cause,
|
||||
);
|
||||
}
|
||||
await mkdir(dirname(snapshotPath), { recursive: true, mode: 0o700 });
|
||||
await writeFile(snapshotPath, bytes, { flag: "wx", mode: 0o600 });
|
||||
return {
|
||||
path,
|
||||
snapshot_path: snapshotPath,
|
||||
relative_path: relativePath,
|
||||
sha256: recoveryDigest(bytes),
|
||||
bytes: bytes.length,
|
||||
};
|
||||
};
|
||||
|
||||
/** Preserve each selected file and its relative input layout without concatenation. */
|
||||
export const snapshotRecoveryInputs = async (
|
||||
paths: readonly [string, ...string[]],
|
||||
directory: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<readonly [RecoveryInputSnapshot, ...RecoveryInputSnapshot[]]> => {
|
||||
let commonDirectory = dirname(resolve(paths[0]));
|
||||
for (const path of paths) {
|
||||
while (true) {
|
||||
const fromCommon = relative(commonDirectory, resolve(path));
|
||||
if (
|
||||
fromCommon !== ".." &&
|
||||
!fromCommon.startsWith(`..${sep}`) &&
|
||||
!isAbsolute(fromCommon)
|
||||
)
|
||||
break;
|
||||
const parent = dirname(commonDirectory);
|
||||
if (parent === commonDirectory)
|
||||
throw recoveryInputFailure(
|
||||
"additional_paths",
|
||||
"Selected files must share a filesystem root",
|
||||
);
|
||||
commonDirectory = parent;
|
||||
}
|
||||
}
|
||||
const snapshots: RecoveryInputSnapshot[] = [];
|
||||
let remaining = RECOVERY_LIMITS.inputBytes;
|
||||
for (const path of paths) {
|
||||
const relativePath =
|
||||
paths.length === 1
|
||||
? "bundle.js"
|
||||
: relative(commonDirectory, resolve(path)).split(sep).join("/");
|
||||
try {
|
||||
if (normalizeArtifactPath(relativePath) !== relativePath)
|
||||
throw new TypeError(
|
||||
"Input names must already use NFC Unicode spelling",
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
throw recoveryInputFailure(
|
||||
"path",
|
||||
`Cannot preserve selected input path in published artifacts: ${path}: ${recoveryFailureMessage(cause)}`,
|
||||
cause,
|
||||
);
|
||||
}
|
||||
const snapshot = await snapshotRecoveryInput(
|
||||
path,
|
||||
join(directory, relativePath),
|
||||
relativePath,
|
||||
remaining,
|
||||
signal,
|
||||
);
|
||||
snapshots.push(snapshot);
|
||||
remaining -= snapshot.bytes;
|
||||
}
|
||||
// The selected-input tuple guarantees a primary snapshot.
|
||||
const [primary, ...additional] = snapshots;
|
||||
if (primary === undefined)
|
||||
throw recoveryInputFailure("path", "Select a JavaScript input");
|
||||
return [primary, ...additional];
|
||||
};
|
||||
|
||||
/** Inventory staging without following symlinks or publishing special files. */
|
||||
export const inventoryRecoveryFiles = async (
|
||||
root: string,
|
||||
|
||||
@@ -14,8 +14,7 @@ import {
|
||||
fingerprintRecoveryFile,
|
||||
inventoryRecoveryFiles,
|
||||
readRecoveryFile,
|
||||
recoveryDigest,
|
||||
type snapshotRecoveryInput,
|
||||
type snapshotRecoveryInputs,
|
||||
} from "./RecoveryFiles.js";
|
||||
import {
|
||||
runWakaruCommand,
|
||||
@@ -38,7 +37,7 @@ type RecoveryWorkspace = {
|
||||
root: string;
|
||||
tree: SafeOutputTree;
|
||||
input: JavaScriptRecoveryInput;
|
||||
source: Awaited<ReturnType<typeof snapshotRecoveryInput>>;
|
||||
sources: Awaited<ReturnType<typeof snapshotRecoveryInputs>>;
|
||||
engine: Awaited<ReturnType<typeof resolveWakaruCommand>>;
|
||||
environment: Readonly<Record<string, string | undefined>>;
|
||||
launcher: WakaruLauncher | undefined;
|
||||
@@ -48,7 +47,7 @@ type RecoveryWorkspace = {
|
||||
|
||||
/** Consume the pinned producer representation in a private workspace. */
|
||||
const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
const { root, input, source, engine, environment, launcher, options } =
|
||||
const { root, input, sources, engine, environment, launcher, options } =
|
||||
context;
|
||||
const staging = join(root, "modules");
|
||||
const commandContext = {
|
||||
@@ -81,7 +80,7 @@ const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
...commandContext,
|
||||
staging,
|
||||
arguments: [
|
||||
source.snapshot_path,
|
||||
...sources.map((source) => source.snapshot_path),
|
||||
`--unpack=${MODES[input.extraction_mode]}`,
|
||||
`--level=${input.rewrite_level}`,
|
||||
"--provenance",
|
||||
@@ -92,16 +91,20 @@ const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
],
|
||||
});
|
||||
const files = await inventoryRecoveryFiles(staging, options?.signal);
|
||||
const snapshot = await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
options?.signal,
|
||||
);
|
||||
const snapshots = new Map<string, Buffer>();
|
||||
for (const source of sources)
|
||||
snapshots.set(
|
||||
source.snapshot_path,
|
||||
await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
source.bytes,
|
||||
options?.signal,
|
||||
),
|
||||
);
|
||||
const parsed = await parseWakaruReports({
|
||||
execution: run,
|
||||
provenancePath: join(staging, "provenance.json"),
|
||||
snapshotPath: source.snapshot_path,
|
||||
sourceBytes: snapshot,
|
||||
snapshots,
|
||||
files,
|
||||
});
|
||||
return { version, admitted, run, parsed };
|
||||
@@ -109,7 +112,8 @@ const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
|
||||
/** Publish validated recovery and retain exact source and engine bindings. */
|
||||
export const prepareWakaruExecution = async (context: RecoveryWorkspace) => {
|
||||
const { root, tree, input, source, engine, options } = context;
|
||||
const { root, tree, input, sources, engine, options } = context;
|
||||
const [source] = sources;
|
||||
const staging = join(root, "modules");
|
||||
const { version, admitted, run, parsed } =
|
||||
await executeWakaruRecovery(context);
|
||||
@@ -117,26 +121,25 @@ export const prepareWakaruExecution = async (context: RecoveryWorkspace) => {
|
||||
tree,
|
||||
staging,
|
||||
input,
|
||||
source,
|
||||
sources,
|
||||
engine,
|
||||
admitted,
|
||||
parsed,
|
||||
execution: run,
|
||||
...(options?.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
if (
|
||||
recoveryDigest(
|
||||
await readRecoveryFile(
|
||||
input.path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
options?.signal,
|
||||
),
|
||||
) !== source.sha256
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
OPERATION,
|
||||
`Original input changed during recovery: ${input.path}`,
|
||||
for (const selected of sources) {
|
||||
const current = await fingerprintRecoveryFile(
|
||||
selected.path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
options?.signal,
|
||||
);
|
||||
if (current.sha256 !== selected.sha256 || current.bytes !== selected.bytes)
|
||||
throw new AnalysisOutputError(
|
||||
OPERATION,
|
||||
`Original input changed during recovery: ${selected.path}`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
(
|
||||
await fingerprintRecoveryFile(
|
||||
@@ -166,7 +169,10 @@ export const prepareWakaruExecution = async (context: RecoveryWorkspace) => {
|
||||
format: "javascript",
|
||||
},
|
||||
locations: [
|
||||
{ kind: "artifact-path", path: source.path },
|
||||
...sources.map((selected) => ({
|
||||
kind: "artifact-path" as const,
|
||||
path: selected.path,
|
||||
})),
|
||||
...result.modules.map((module) => ({
|
||||
kind: "artifact-path" as const,
|
||||
path: module.artifact.path,
|
||||
|
||||
@@ -22,7 +22,7 @@ import {
|
||||
checkRecoveryDeadline,
|
||||
recoveryFailureMessage,
|
||||
recoveryInputFailure,
|
||||
snapshotRecoveryInput,
|
||||
snapshotRecoveryInputs,
|
||||
} from "./RecoveryFiles.js";
|
||||
import {
|
||||
resolveWakaruCommand,
|
||||
@@ -55,7 +55,7 @@ export class WakaruProvider implements JavaScriptRecoveryPort {
|
||||
readonly launcher?: WakaruLauncher,
|
||||
) {}
|
||||
|
||||
/** Recover one script using validated upstream reports and verified output bytes. */
|
||||
/** Recover selected scripts using validated reports and verified output bytes. */
|
||||
recover(
|
||||
input: JavaScriptRecoveryInput,
|
||||
options?: ExecutionOptions,
|
||||
@@ -109,9 +109,9 @@ export class WakaruProvider implements JavaScriptRecoveryPort {
|
||||
const staging = join(root, "modules");
|
||||
await mkdir(inputs, { mode: 0o700 });
|
||||
await mkdir(staging, { mode: 0o700 });
|
||||
const source = await snapshotRecoveryInput(
|
||||
input.path,
|
||||
join(inputs, "bundle.js"),
|
||||
const sources = await snapshotRecoveryInputs(
|
||||
[input.path, ...(input.additional_paths ?? [])],
|
||||
inputs,
|
||||
options?.signal,
|
||||
);
|
||||
try {
|
||||
@@ -128,7 +128,7 @@ export class WakaruProvider implements JavaScriptRecoveryPort {
|
||||
root,
|
||||
tree,
|
||||
input,
|
||||
source,
|
||||
sources,
|
||||
engine,
|
||||
environment: this.environment,
|
||||
launcher: this.launcher,
|
||||
|
||||
@@ -7,7 +7,11 @@ import {
|
||||
type JavaScriptRecoveryInput,
|
||||
type JavaScriptRecoveryResult,
|
||||
} from "../../domain/javascript/javascriptRecovery.js";
|
||||
import { readRecoveryFile, recoveryDigest } from "./RecoveryFiles.js";
|
||||
import {
|
||||
readRecoveryFile,
|
||||
recoveryDigest,
|
||||
type RecoveryInputSnapshot,
|
||||
} from "./RecoveryFiles.js";
|
||||
import {
|
||||
recoveryRanges,
|
||||
validateRecoverySourceMap,
|
||||
@@ -28,12 +32,7 @@ type PublicationContext = {
|
||||
tree: SafeOutputTree;
|
||||
staging: string;
|
||||
input: JavaScriptRecoveryInput;
|
||||
source: {
|
||||
path: string;
|
||||
snapshot_path: string;
|
||||
sha256: string;
|
||||
bytes: number;
|
||||
};
|
||||
sources: readonly [RecoveryInputSnapshot, ...RecoveryInputSnapshot[]];
|
||||
engine: Awaited<ReturnType<typeof resolveWakaruCommand>>;
|
||||
admitted: Exclude<WakaruVersionAdmission, { status: "unsupported" }>;
|
||||
parsed: Awaited<ReturnType<typeof parseWakaruReports>>;
|
||||
@@ -47,7 +46,7 @@ type RecoveryWriter = (path: string, bytes: Buffer) => Promise<PublishedFile>;
|
||||
export const publishWakaruArtifacts = async (
|
||||
context: PublicationContext,
|
||||
): Promise<JavaScriptRecoveryResult> => {
|
||||
const { tree, source, parsed, staging, signal } = context;
|
||||
const { tree, sources, parsed, staging, signal } = context;
|
||||
const write = async (path: string, bytes: Buffer) => {
|
||||
const sha256 = recoveryDigest(bytes);
|
||||
const file = await tree.write(
|
||||
@@ -62,17 +61,25 @@ export const publishWakaruArtifacts = async (
|
||||
bytes: file.bytesWritten,
|
||||
};
|
||||
};
|
||||
const snapshot = await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
signal,
|
||||
);
|
||||
if (recoveryDigest(snapshot) !== source.sha256)
|
||||
throw new AnalysisOutputError(
|
||||
"recover_javascript_sources",
|
||||
"The input snapshot changed during recovery",
|
||||
const publishedSources: JavaScriptRecoveryResult["source"][] = [];
|
||||
for (const selected of sources) {
|
||||
const { relative_path, ...source } = selected;
|
||||
const snapshot = await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
source.bytes,
|
||||
signal,
|
||||
);
|
||||
const publishedCopy = await write("inputs/bundle.js", snapshot);
|
||||
if (recoveryDigest(snapshot) !== source.sha256)
|
||||
throw new AnalysisOutputError(
|
||||
"recover_javascript_sources",
|
||||
`The input snapshot changed during recovery: ${source.path}`,
|
||||
);
|
||||
publishedSources.push({
|
||||
...source,
|
||||
published_copy: await write(`inputs/${relative_path}`, snapshot),
|
||||
});
|
||||
}
|
||||
const [source, ...additionalSources] = publishedSources;
|
||||
const modules: JavaScriptRecoveryResult["modules"] = [];
|
||||
for (const module of parsed.report.modules) {
|
||||
modules.push(await publishModule(context, module, write));
|
||||
@@ -90,7 +97,10 @@ export const publishWakaruArtifacts = async (
|
||||
),
|
||||
);
|
||||
const details = {
|
||||
source: { ...source, published_copy: publishedCopy },
|
||||
source,
|
||||
...(additionalSources.length === 0
|
||||
? {}
|
||||
: { additional_sources: additionalSources }),
|
||||
engine: {
|
||||
id: "wakaru",
|
||||
version: context.admitted.version,
|
||||
@@ -151,7 +161,7 @@ const publishModule = async (
|
||||
module: PublicationContext["parsed"]["report"]["modules"][number],
|
||||
write: RecoveryWriter,
|
||||
): Promise<JavaScriptRecoveryResult["modules"][number]> => {
|
||||
const { staging, source, parsed, signal } = context;
|
||||
const { staging, sources, parsed, signal } = context;
|
||||
const artifact = await write(
|
||||
`modules/${module.filename}`,
|
||||
await readRecoveryFile(
|
||||
@@ -182,6 +192,14 @@ const publishModule = async (
|
||||
"recover_javascript_sources",
|
||||
`Missing module provenance: ${module.filename}`,
|
||||
);
|
||||
const origin = sources.find(
|
||||
(source) => source.snapshot_path === provenance.input,
|
||||
);
|
||||
if (origin === undefined)
|
||||
throw new AnalysisOutputError(
|
||||
"recover_javascript_sources",
|
||||
`Unselected module provenance input: ${provenance.input}`,
|
||||
);
|
||||
return {
|
||||
reported_filename: module.filename,
|
||||
reported_status: module.status,
|
||||
@@ -189,8 +207,8 @@ const publishModule = async (
|
||||
source_map: sourceMap,
|
||||
provenance: {
|
||||
reported_input: provenance.input,
|
||||
original_path: source.path,
|
||||
original_sha256: source.sha256,
|
||||
original_path: origin.path,
|
||||
original_sha256: origin.sha256,
|
||||
extraction: provenance.extraction,
|
||||
byte_ranges: recoveryRanges(provenance.ranges),
|
||||
context_byte_ranges: recoveryRanges(provenance.context_ranges ?? []),
|
||||
|
||||
@@ -36,5 +36,5 @@ export const RECOVERY_LIMITATIONS = [
|
||||
"Extraction ranges refer to half-open UTF-8 byte offsets in the original input, not rewritten line positions or whole-bundle coverage.",
|
||||
"Emitted source maps are preserved as reported by the engine; their mapping accuracy is not independently established.",
|
||||
"Inspection mode may produce non-executable regions. No input or recovered application code is executed.",
|
||||
"Linux adapter: one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB input, 128 MiB output, 10000 output entries and 8 MiB per diagnostic/report stream. Resource limits are per operation, not aggregate host containment.",
|
||||
"Linux adapter: one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB total input, 128 MiB output, 10000 output entries and 8 MiB per diagnostic/report stream. Resource limits are per operation, not aggregate host containment.",
|
||||
] as const;
|
||||
|
||||
@@ -54,8 +54,7 @@ const OPERATION = "recover_javascript_sources";
|
||||
type ReportContext = {
|
||||
execution: Awaited<ReturnType<typeof runWakaruCommand>>;
|
||||
provenancePath: string;
|
||||
snapshotPath: string;
|
||||
sourceBytes: Buffer;
|
||||
snapshots: ReadonlyMap<string, Buffer>;
|
||||
files: readonly { relativePath: string }[];
|
||||
};
|
||||
|
||||
@@ -121,7 +120,7 @@ const validateModuleSet = (
|
||||
provenance: z.output<typeof provenanceSchema>,
|
||||
context: ReportContext,
|
||||
): void => {
|
||||
const { files, snapshotPath, sourceBytes } = context;
|
||||
const { files, snapshots } = context;
|
||||
const registry = new ArtifactPathRegistry();
|
||||
const expected = new Set(["provenance.json"]);
|
||||
const observed = new Set(files.map((file) => file.relativePath));
|
||||
@@ -135,7 +134,9 @@ const validateModuleSet = (
|
||||
expected.add(path);
|
||||
if (observed.has(`${path}.map`)) expected.add(`${path}.map`);
|
||||
const entry = provenance.modules[path];
|
||||
if (entry === undefined || entry.input !== snapshotPath)
|
||||
const sourceBytes =
|
||||
entry === undefined ? undefined : snapshots.get(entry.input);
|
||||
if (entry === undefined || sourceBytes === undefined)
|
||||
throw new TypeError(
|
||||
`Module provenance does not bind the selected input snapshot: ${path}`,
|
||||
);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { access, mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { access, mkdir, open, readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { SafeOutputTree } from "../../../src/artifacts/SafeOutputTree.js";
|
||||
@@ -6,6 +6,7 @@ import { SafeOutputTreeCreationFailure } from "../../../src/artifacts/SafeOutput
|
||||
import { javascriptRecoveryResultSchema } from "../../../src/domain/javascript/javascriptRecovery.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
|
||||
import { RECOVERY_LIMITS } from "../../../src/javascript/recovery/WakaruRelease.js";
|
||||
import {
|
||||
assertRecoveryCleanup,
|
||||
recoveryFixture,
|
||||
@@ -36,6 +37,7 @@ describe.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
},
|
||||
});
|
||||
expect(value.source.path).toBe(fixture.path);
|
||||
expect(value.additional_sources).toBeUndefined();
|
||||
expect(value.source.published_copy.sha256).toBe(value.source.sha256);
|
||||
expect(value.modules[0]?.provenance.byte_ranges).toEqual([
|
||||
{ start: 0, end: value.source.bytes },
|
||||
@@ -396,3 +398,153 @@ describe.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
describe.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
"Wakaru multi-input ownership",
|
||||
() => {
|
||||
it.each(["cafe\u0301.js", "chunk\\name.js"])(
|
||||
"rejects an unpublishable native filename %s before launching",
|
||||
async (name) => {
|
||||
const fixture = await recoveryFixture();
|
||||
const additional = join(fixture.root, name);
|
||||
await writeFile(additional, "export const value = 1;");
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok)
|
||||
throw new Error("expected unsupported input-layout failure");
|
||||
expect(projectAnalysisError(result.error).code).toBe("invalid_request");
|
||||
expect(JSON.stringify(projectAnalysisError(result.error))).toContain(
|
||||
"Cannot preserve selected input path",
|
||||
);
|
||||
expect(fixture.launches).toHaveLength(0);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
expect(await readFile(additional, "utf8")).toBe(
|
||||
"export const value = 1;",
|
||||
);
|
||||
},
|
||||
);
|
||||
it("keeps separate snapshots, per-file ranges and relative paths for same-named inputs", async () => {
|
||||
const fixture = await recoveryFixture(
|
||||
"normal",
|
||||
"import './nested/selected.js';\n",
|
||||
);
|
||||
const nested = join(fixture.root, "nested");
|
||||
await mkdir(nested);
|
||||
const additional = join(nested, "selected.js");
|
||||
const content = "export const target = '🎯';\n";
|
||||
await writeFile(additional, content);
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const value = javascriptRecoveryResultSchema.parse(
|
||||
result.value.normalized_result,
|
||||
);
|
||||
const sources = [value.source, ...(value.additional_sources ?? [])];
|
||||
expect(sources.map((source) => source.path)).toEqual([
|
||||
fixture.path,
|
||||
additional,
|
||||
]);
|
||||
expect(sources.map((source) => source.published_copy.path)).toEqual([
|
||||
join(fixture.output, "inputs/selected.js"),
|
||||
join(fixture.output, "inputs/nested/selected.js"),
|
||||
]);
|
||||
expect(await readFile(sources[1]!.published_copy.path, "utf8")).toBe(
|
||||
content,
|
||||
);
|
||||
expect(result.value.subject?.digest.sha256).toBe(value.source.sha256);
|
||||
for (const [index, module] of value.modules.entries()) {
|
||||
const source = sources[index]!;
|
||||
expect(source.published_copy.sha256).toBe(source.sha256);
|
||||
expect(module.provenance).toMatchObject({
|
||||
reported_input: source.snapshot_path,
|
||||
original_path: source.path,
|
||||
original_sha256: source.sha256,
|
||||
byte_ranges: [{ start: 0, end: source.bytes }],
|
||||
context_byte_ranges: [{ start: 0, end: source.bytes }],
|
||||
});
|
||||
}
|
||||
const manifest = JSON.parse(await readFile(value.manifest.path, "utf8"));
|
||||
expect(manifest.additional_sources).toEqual(value.additional_sources);
|
||||
await assertRecoveryCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it.each([
|
||||
"secondary-foreign",
|
||||
"secondary-overflow",
|
||||
"secondary-utf8",
|
||||
"secondary-change",
|
||||
"redistribute",
|
||||
])("rejects %s against its own source and rolls back", async (mode) => {
|
||||
const fixture = await recoveryFixture(mode, "a");
|
||||
const additional = join(fixture.root, "chunk.js");
|
||||
await writeFile(additional, mode === "secondary-utf8" ? "🎯" : "bc");
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok) throw new Error("expected per-source validation failure");
|
||||
expect(projectAnalysisError(result.error).code).toMatch(
|
||||
/execution_failure|unreadable_output/u,
|
||||
);
|
||||
const reason =
|
||||
mode === "secondary-foreign"
|
||||
? "does not bind the selected input snapshot"
|
||||
: mode === "redistribute" || mode === "secondary-change"
|
||||
? "Original input changed"
|
||||
: "Invalid half-open UTF-8 byte range";
|
||||
expect(JSON.stringify(projectAnalysisError(result.error))).toContain(
|
||||
reason,
|
||||
);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
await assertRecoveryCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it("rejects aggregate input overflow before launching the engine", async () => {
|
||||
const fixture = await recoveryFixture("normal", "a");
|
||||
const additional = join(fixture.root, "chunk.js");
|
||||
const handle = await open(additional, "wx");
|
||||
try {
|
||||
await handle.truncate(RECOVERY_LIMITS.inputBytes);
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok) throw new Error("expected aggregate capacity failure");
|
||||
expect(projectAnalysisError(result.error).code).toBe("invalid_request");
|
||||
expect(fixture.launches).toHaveLength(0);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects non-UTF-8 input before launching recovery", async () => {
|
||||
const fixture = await recoveryFixture();
|
||||
await writeFile(fixture.path, Buffer.from([0xff, 0xfe]));
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok) throw new Error("expected malformed source failure");
|
||||
expect(projectAnalysisError(result.error).code).toBe("invalid_request");
|
||||
expect(fixture.launches).toHaveLength(0);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import { expect, it, onTestFinished } from "vitest";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
@@ -10,9 +12,11 @@ import {
|
||||
recoveryFixture,
|
||||
} from "../../fixtures/javascript-recovery/provider.js";
|
||||
|
||||
it.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
"advertises and executes recovery through the named SDK contract without a binary target",
|
||||
async () => {
|
||||
it
|
||||
.skipIf(process.platform !== "linux" || process.arch !== "x64")
|
||||
.each([false, true])(
|
||||
"advertises and executes recovery through the named SDK contract (multiple inputs: %s)",
|
||||
async (multiple) => {
|
||||
const fixture = await recoveryFixture();
|
||||
const session = createTestBinarySession(() => {
|
||||
throw new Error("no binary provider should start");
|
||||
@@ -44,7 +48,13 @@ it.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
const outputSchema: Record<string, unknown> = advertised.outputSchema;
|
||||
expect(ajv.validateSchema(inputSchema)).toBe(true);
|
||||
expect(ajv.validateSchema(outputSchema)).toBe(true);
|
||||
const args = { path: fixture.path, output_directory: fixture.output };
|
||||
const additional = join(fixture.root, "chunk.js");
|
||||
if (multiple) await writeFile(additional, "export const chunk = 1;");
|
||||
const args = {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
...(multiple ? { additional_paths: [additional] } : {}),
|
||||
};
|
||||
expect(ajv.validate(inputSchema, args)).toBe(true);
|
||||
const response = await client.callTool({
|
||||
name: "recover_javascript_sources",
|
||||
|
||||
+65
@@ -14,6 +14,71 @@ if (args.includes("--version")) {
|
||||
} else {
|
||||
const input = args[0];
|
||||
const output = args[args.indexOf("-o") + 1];
|
||||
const inputs = args.slice(
|
||||
0,
|
||||
args.findIndex((arg) => arg.startsWith("--")),
|
||||
);
|
||||
if (inputs.length > 1) {
|
||||
const modules = [];
|
||||
const provenance = {
|
||||
format: "unknown",
|
||||
strategy: "structural",
|
||||
modules: {},
|
||||
};
|
||||
for (const [index, selected] of inputs.entries()) {
|
||||
const bytes = await readFile(selected);
|
||||
const filename = `module-${index}.js`;
|
||||
modules.push({ filename, kind: "javascript", status: "decompiled" });
|
||||
await writeFile(join(output, filename), bytes);
|
||||
provenance.modules[filename] = {
|
||||
input:
|
||||
index === 1 && mode === "secondary-foreign"
|
||||
? "/unselected.js"
|
||||
: selected,
|
||||
ranges: [
|
||||
[
|
||||
0,
|
||||
index === 1 && mode === "secondary-overflow"
|
||||
? bytes.length + 1
|
||||
: bytes.length,
|
||||
],
|
||||
],
|
||||
context_ranges: [
|
||||
[0, index === 1 && mode === "secondary-utf8" ? 2 : bytes.length],
|
||||
],
|
||||
extraction: "structural",
|
||||
};
|
||||
}
|
||||
if (mode === "redistribute") {
|
||||
const primary = process.env.REA_RECOVERY_ORIGINAL_PATH;
|
||||
const secondary = process.env.REA_RECOVERY_ADDITIONAL_PATH;
|
||||
const first = await readFile(primary);
|
||||
const second = await readFile(secondary);
|
||||
await writeFile(primary, Buffer.concat([first, second.subarray(0, 1)]));
|
||||
await writeFile(secondary, second.subarray(1));
|
||||
}
|
||||
if (mode === "secondary-change")
|
||||
await writeFile(
|
||||
process.env.REA_RECOVERY_ADDITIONAL_PATH,
|
||||
"changed additional input",
|
||||
);
|
||||
await writeFile(
|
||||
join(output, "provenance.json"),
|
||||
JSON.stringify(provenance),
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
detected_formats: [],
|
||||
safety: "normal",
|
||||
modules,
|
||||
warnings: [],
|
||||
total: modules.length,
|
||||
failed: 0,
|
||||
elapsed_ms: 1,
|
||||
}),
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
const bytes = await readFile(input);
|
||||
await writeFile(join(dirname(output), "recovery-started"), "started");
|
||||
if (mode === "hang") await new Promise(() => setInterval(() => {}, 1000));
|
||||
|
||||
@@ -38,6 +38,7 @@ export const recoveryFixture = async (
|
||||
...options.env,
|
||||
REA_RECOVERY_FIXTURE_MODE: mode,
|
||||
REA_RECOVERY_ORIGINAL_PATH: path,
|
||||
REA_RECOVERY_ADDITIONAL_PATH: join(root, "chunk.js"),
|
||||
REA_RECOVERY_ENGINE_PATH: command,
|
||||
},
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user