test: eliminate fixture races and prune redundant scaffolding (#849)

* test: eliminate fixture races and prune redundant scaffolding

* fix(test): publish launcher capture readiness without renaming
This commit is contained in:
morluto
2026-10-07 12:25:37 +08:00
committed by GitHub
parent 5cbf06c7ad
commit f72931d9dd
22 changed files with 421 additions and 719 deletions
+1 -1
View File
@@ -277,7 +277,7 @@ jobs:
- run: npm run verify:keyed-archive
- run: npm run verify:asset-catalog
- name: Native compiled NIB hierarchy regressions
run: npm run test:focused -- src/application/NibNativeHierarchy.test.ts
run: npm run test:focused -- tests/boundary/filesystem/nibNativeHierarchy.test.ts tests/boundary/filesystem/interfaceBuilderBundle.test.ts
- name: Real binary metadata and Interface Builder integration
run: npm run verify:apple-dispatch && npm run verify:interface-builder
- name: Native plist CLI selection regressions
-169
View File
@@ -1,169 +0,0 @@
# Test suite audit
## Scope and priority
The expanded audit on 2026-10-04 reviewed every one of the 355 Vitest files
present at its start: 82 domain/contracts, 56 application/composition, 147
adapter/configuration/process/filesystem/CLI files, and 70 MCP/acceptance/
conformance/evaluation files. Three review passes checked the proposed removals
against existing stronger coverage and checked the final changes for lost
assertions. The remaining suite has 351 files.
Prefer full real-provider E2E, then integration across schemas and transports,
then real-data goldens. Remove lower-level duplication once the replacement
asserts the same behavior. Keep distinct failure regressions that a successful
E2E cannot exercise. Do not lower coverage thresholds to accommodate pruning.
## Changes made
| Previous test or assertion | Decision | Replacement or retained claim |
| ----------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Asset application tests using fabricated `.car` bytes and injected utility JSON | Remove two plumbing cases | `verify:asset-catalog`: real actool → assetutil → CLI and stdio MCP, including digest, full metadata, pagination and invalid input |
| Keyed archive positive binary/XML objects assembled with the same plist library as the reader | Remove two synthetic success cases | Captured Foundation XML plus reviewed complete object/reference graph golden; real Foundation binary/XML → CLI and stdio MCP |
| Generic contract schema/annotation/example loops and output-map inventory comparisons | Consolidate into one SDK wire test | Assert exported schemas, guidance, effect annotations and valid examples on the actual advertised MCP catalog |
| Ghidra function operation enum membership | Remove | Canonical catalog, provider handshake and actual operation roundtrips establish admission; malformed provider results remain covered |
| Asset page assertion pinning `10_001` | Remove | Invalid-page rejection through real MCP; malformed utility data still covered |
| Suggested prompt tool positions | Remove incidental ordering assertions | Keep authority, evidence, untrusted input and preparation constraints |
| Package “acceptance” assertions pinning CI matrix/shard values and fake launcher command arrays | Remove configuration-shaped cases | Actual package/install verifier remains; retain a compact release verification-before-publish and registry-canary static guard |
| Package failure classifier under acceptance | Move to package evaluation | Preserve all malformed/unrelated-failure rejection cases; do not call them E2E |
| Native value graph checks conditional on success | Strengthen | Require success before pagination/ambiguity assertions; unexpected errors can no longer pass |
| Native UI verifier treating permission denial as a positive pass | Separate outcomes | Default lane requires real capture/action success; opt-in permissions lane explicitly reports no positive E2E proof |
Ten previous test cases were removed or consolidated; three replacement
Vitest cases cover the captured goldens and release guard. The two real artifact
E2E scripts add product-level evidence outside the deterministic Vitest count.
## Exhaustive follow-up
The follow-up removes or consolidates another 31 test declarations, including
three redundant test files. Parameterized declarations and executed cases have
different counts; the figures above count source declarations. It also repairs
assertions that could pass without exercising their stated behavior.
| Area | Cleanup | Stronger retained evidence |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| Configuration | Merge duplicate target/loader tests and preserve custom launcher, explicit provider and actionable error diagnostics | Independent malformed inputs; each capability flag is rejected separately |
| Browser scenarios | Remove fake launch success and expression-selected artifact success | Real Chromium scenarios, actual artifacts, CLI capture and owned/external cleanup in `verify:browser` |
| CLI policy | Remove duplicate confirmation helper cases | Compiled CLI proves project files stay unchanged until `--yes`; distinct interactive choices remain |
| Provider capabilities | Remove numerical catalog freezes; consolidate Hopper byte-read discovery | Exact canonical capability identities, admission, mutation effects and detached descriptors |
| Evidence ledger | Merge reordered-key checks into serialized bundle import; remove one-record quota claims | Atomic import, conflicts, residual revisions, actual large collections and byte regressions |
| MCP batching and concurrency | Replace maximum concurrency and empty indistinguishable responses | Every distinct ordered batch result; concurrently gated requests complete in reverse order with matching payloads |
| MCP inventories and graphs | Remove traversal/inventory call order and redundant catalog/prose checks | Exact complete cyclic graph, full inline inventories, canonical SDK catalog and all-handler roundtrips |
| Authority and lifecycle | Attempt the refused Ghidra mutation; require successful setup, call and switch; close twice | Typed refusal with zero mutation dispatch, restored target calls and exact owned-resource cleanup |
| Process and provider output | Remove incidental environment read and command option order/defaults | PID/ownership validation, exact disabled-policy failure and complete decoded response above 10 MiB |
| Conformance and CI guards | Merge identical replay runs; remove duplicate missing-row fault, trivial hash/self equality and incidental worker/step counts | Distinct faulty evidence, digest order, actual test discovery, file existence and release/runner authority |
| Pure schemas | Remove fabricated comparison-dimension injection and impossible long boundary success | Real artifact comparison and SDK coverage evaluation with returned boundary identity |
The removed long boundary selector test only parsed a 1,000-character request;
workspace boundary identities admit at most 200 characters, so that positive
could never represent a successful evaluation. The actual MCP evaluation stays
covered. One unused ledger serialization helper was also retired after removing
its only test consumer; no CLI or MCP contract changed.
No further immediately safe pruning was identified in the final review. Retained
replacement candidates below are coverage gaps, rather than permission to delete
unique tests. Test-only domain scaffolding also stays pending a separate decision
about the corresponding source behavior; deleting those tests alone would leave
unverified code and fail unused-export checks.
## Stronger replacements from the closing pass
Three more synthetic success cases were removed. Native value composition now
runs through the production CLI and a separate stdio MCP subprocess with real
Ghidra, using a source-built native fixture. It checks argument/parameter/return
relationships, the fixture's global operand, authenticated Evidence, upstream
and workflow analysis profiles, identical CLI/MCP graphs, discovery availability
and successful MCP session closure. Ambiguity, cancellation, target changes,
missing flow and bounded-work regressions remain in the focused suite.
That replacement found missing CLI workflow routing for `trace_native_values`
and missing workflow profile classification. The routing and discovery now use
the composed operation's actual provider prerequisites; workflow Evidence commits
to the upstream Ghidra profile.
The lone hand-built client-document equality test is replaced by a real TOML
filesystem transaction: reversed environment keys leave the original bytes
unchanged and create no backup. A real ZIP entry with an application root longer
than 5,000 characters replaces injected schema-only positive data. Inventory
completeness tests partition actual scans into authenticated partial observations
and require exact assembled results and every source citation, including more
than 100 citations. These are consumer merge tests, not provider pagination
claims. Replay aggregation now covers all four statuses and retains actual runner
diagnostics; it does not claim to execute a real sandbox. Retained pixel-metric
fixtures now write correct PNG chunk checksums.
The all-zero screenshot buffer is replaced by a real Chromium PNG of 9,467,940
bytes, transported intact through CLI and stdio MCP and decoded by the production
pixel comparator. The verifier configures the SDK client's receive buffer because
its 10 MiB default is smaller than the inline JSON response. This does not claim
large comparison-request transport coverage. A real SDK initialize request,
captured during a successful production stdio handshake, replaces the JSON-RPC
decoder's field-count-only oracle with exact field/value expectations.
The closing pass also removed assertions tied to internal call order, defaults,
private cache sizes, duplicated polling and fabricated schema extensions. Unique
malformed-input, security-capacity and cleanup regressions remain.
## Real process regression found
The full suite exposed a renderer ordering race. Publishing a terminal output
journal entry could synchronously trigger a reactive resize before the preceding
output was queued for rendering. Queue that output first. The existing real
source-owned interactive PTY test now also requires the first resized frame to
contain the output that triggered the resize, preserving the original event
order rather than sorting or changing timestamps to conceal it.
## Coverage kept
Malformed archives, broken and cyclic references, unsupported metadata pointers,
relative Objective-C method layouts, ambiguous call candidates, digest mismatch,
cancellation, host permission denial, bounded output and owned-resource cleanup
remain covered. These cases provide evidence beyond a successful real-tool run.
The hand-assembled relative method fixture stays until compiler-produced
relative-list fixtures establish equivalent assertions.
## Remaining priorities
1. Extend the real Ghidra CLI/stdio MCP journey from value tracing to dispatch
and UI handlers before pruning their hand-authored composition success cases.
Native value tracing now establishes full transport parity.
2. Real binary dispatch and nib → full product transports, with normalized
semantic goldens. The compiler-backed readers already run in the new Apple
CI lane; synthetic pointer/layout rejection tests remain useful.
3. An approved interactive macOS runner for native UI capture/actions, including
stdio MCP scenarios. Hosted runners without permissions cannot supply this
proof. No arbitrary application's windows should be opened for this lane.
4. Real macOS/Linux Ghidra CI runners with bring-your-own installations. This
change adds Apple artifact CI without installing Java or Ghidra.
5. Classify fake-runner replay/application acceptance accurately and replace
repeated happy paths only after real sandbox journeys assert the same claims.
The replay workflow's allow-unavailable mode must not imply executed replay.
6. Replace repeated-array schema ceiling tests with real producer/transport
regressions before deleting them. Existing integration tests do not yet
establish their large-input guarantees.
Some static guards protect irreversible release authority rather than product
behavior. Keep these few guards explicit and separate from E2E evidence.
## Verification after the expanded audit
- `npm run check:pr`: 351 files passed; 1,794 tests passed and two skipped.
Typecheck, lint, formatting, unused-code and generated-document gates passed.
- `npm run check:fast`: passed; 70 lint warnings and zero errors.
- `npm run verify:browser`: actual Chromium observation and scenario CLI routes,
retained artifacts, owned/external lifecycle checks and large CLI/stdio MCP PNG
capture passed.
- `npm run verify:ghidra`: macOS ARM64 with Ghidra 12.1.4/JDK 21 passed, including
real native value tracing through CLI and stdio MCP: 171 nodes, 298 edges and
six decompilations.
- `npm run verify:asset-catalog`: real actool/assetutil → CLI and stdio MCP,
complete records, pagination and malformed input checks passed.
- `npm run verify:keyed-archive`: real Foundation binary/XML → CLI and stdio MCP,
full graph golden, shared/cyclic identity and malformed scope checks passed.
- `npm run verify:fixtures`: seven source-built host fixtures, source/artifact
digests and exhaustive inventory checks passed.
No Hopper windows were opened for this audit. The closing pass runs real Ghidra
on macOS ARM64; earlier Hopper evidence remains separate. No new hosted CI success
is claimed.
-3
View File
@@ -106,9 +106,6 @@ It uses an installed browser and accepts an optional installed REA entrypoint.
The complete `verify:browser` lane also exercises passive script export through
both public adapters. See [website script export](website-script-export.md).
The [test suite audit](test-suite-audit.md) records the pruning decisions,
replacement evidence and remaining priorities.
## Real-toolchain verification lanes
Each real-toolchain command must require only the host tools needed to prove
+1 -36
View File
@@ -4,43 +4,8 @@ import {
parseArtifactInventoryEvidence,
tryAssembleInventorySet,
} from "./artifactInventoryEvidence.js";
import { EvidenceReferenceError } from "./evidenceErrors.js";
import type { ReferenceSourceReaderError } from "../reference/ReferenceSourceReaderTypes.js";
/**
* The three validation taxonomies and their disjoint codes:
* - Filesystem reader: cancelled | invalid-root | io | unsupported
* (per-entry: cancelled | changed | io | symlink | unsupported).
* - Session evidence references: missing | wrong_operation |
* wrong_predicate | identity_mismatch.
* - Inventory assembly: TypeError strings, migrating to Result leaves
* starting with the empty-pages code below.
*/
const readerCodes: readonly ReferenceSourceReaderError["code"][] = [
"cancelled",
"invalid-root",
"io",
"unsupported",
];
const referenceReasons = (
[
"missing",
"wrong_operation",
"wrong_predicate",
"identity_mismatch",
] as const
).map(
(reason) =>
new EvidenceReferenceError("ev_test", reason, "expected", null).reason,
);
describe("validation taxonomy mapping", () => {
it("keeps session reference reasons disjoint from reader codes", () => {
for (const reason of referenceReasons)
expect(readerCodes).not.toContain(reason);
});
describe("inventory validation failure channels", () => {
it("returns the empty-pages leaf as Result without throwing", () => {
const result = tryAssembleInventorySet([]);
expect(result).toEqual({
@@ -1,29 +1,23 @@
import { spawn } from "node:child_process";
import { rm } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { describe, expect } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import {
processTest as it,
waitForExit,
} from "../../support/process/processFixture.js";
import {
BrowserStartupError,
waitForBrowserDevtoolsPort,
} from "../../../src/browser/BrowserProcessStartup.js";
const roots: string[] = [];
afterEach(async () => {
await Promise.all(
roots.splice(0).map((root) => rm(root, { recursive: true, force: true })),
);
});
describe("browser process startup", () => {
it("returns a delayed valid DevToolsActivePort", async () => {
it("returns a delayed valid DevToolsActivePort", async ({ processes }) => {
const root = await temporaryRoot();
const portPath = join(root, "DevToolsActivePort");
const child = spawn(process.execPath, [
const child = processes.spawn(process.execPath, [
"-e",
`setTimeout(() => require("node:fs").writeFileSync(${JSON.stringify(portPath)}, "43117\\n/browser"), 20); setTimeout(() => {}, 1000)`,
]);
@@ -40,12 +34,15 @@ describe("browser process startup", () => {
).resolves.toBe(43_117);
} finally {
child.kill("SIGKILL");
expect(await waitForExit(child, 5_000)).toBe(true);
}
});
it("classifies signal termination instead of timing out", async () => {
it("classifies signal termination instead of timing out", async ({
processes,
}) => {
const root = await temporaryRoot();
const child = spawn(process.execPath, [
const child = processes.spawn(process.execPath, [
"-e",
"process.kill(process.pid, 'SIGTERM')",
]);
@@ -68,9 +65,12 @@ describe("browser process startup", () => {
});
});
it("reports bounded timeout diagnostics", async () => {
it("reports bounded timeout diagnostics", async ({ processes }) => {
const root = await temporaryRoot();
const child = spawn(process.execPath, ["-e", "setTimeout(() => {}, 1000)"]);
const child = processes.spawn(process.execPath, [
"-e",
"setTimeout(() => {}, 1000)",
]);
try {
const failure = await waitForBrowserDevtoolsPort({
child,
@@ -90,12 +90,11 @@ describe("browser process startup", () => {
expect(String(failure)).toContain("stderr=<empty>");
} finally {
child.kill("SIGKILL");
expect(await waitForExit(child, 5_000)).toBe(true);
}
});
});
const temporaryRoot = async (): Promise<string> => {
const root = await createTestTempDirectory("rea-browser-startup-");
roots.push(root);
return root;
return createTestTempDirectory("rea-browser-startup-");
};
@@ -391,11 +391,12 @@ describeBrowser("CdpBrowserProvider: lifecycle failures 4", () => {
);
if (!committed.ok) throw committed.error;
expect(committed.value.target.origin).toBe(transitioning.allowedOrigin);
expect(
transitioning.commands.filter(
({ method }) => method === "Page.getFrameTree",
),
).toHaveLength(5);
expect(committed.value.frames).toContainEqual(
expect.objectContaining({
frame_id: "frame-main",
origin: transitioning.allowedOrigin,
}),
);
const navigated = await startFakeCdpBrowser({
attachedFrameUrl: "https://unapproved.example.test/after-attach",
@@ -1,5 +1,5 @@
import { createServer } from "node:http";
import { describe, expect, it } from "vitest";
import { describe, expect, it, onTestFinished } from "vitest";
import { fetchWebSourceMaps } from "../../../src/browser/WebSourceMapFetcher.js";
import { analyzeWebBundleInputSchema } from "../../../src/domain/webBundleAnalysis.js";
@@ -34,6 +34,12 @@ describe("source-map discarded HTTP response ownership", () => {
notifyClosed?.();
});
});
const closeServer = async (): Promise<void> => {
server.closeAllConnections();
if (server.listening)
await new Promise<void>((resolve) => server.close(() => resolve()));
};
onTestFinished(closeServer);
try {
await new Promise<void>((resolve) =>
server.listen(0, "127.0.0.1", resolve),
@@ -62,22 +68,10 @@ describe("source-map discarded HTTP response ownership", () => {
expect(result.items[0]?.status).toBe(
status === 302 ? "included" : "fetch_failed",
);
let timer: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
closed,
new Promise<void>((resolve) => {
timer = setTimeout(resolve, 500);
}),
]);
expect(didClose).toBe(true);
} finally {
clearTimeout(timer);
}
await closed;
expect(didClose).toBe(true);
} finally {
server.closeAllConnections();
if (server.listening)
await new Promise<void>((resolve) => server.close(() => resolve()));
await closeServer();
}
},
);
@@ -1,26 +1,20 @@
import { existsSync } from "node:fs";
import { execFileSync } from "node:child_process";
import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { execFile } from "node:child_process";
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { buildBinary } from "plist";
import { afterEach, describe, expect, it } from "vitest";
import { describe, expect, it } from "vitest";
import { analyzeInterfaceBuilderBundle } from "./InterfaceBuilderAnalysis.js";
import { analyzeInterfaceBuilderBundle } from "../../../src/application/InterfaceBuilderAnalysis.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const roots: string[] = [];
afterEach(async () => {
await Promise.all(
roots.splice(0).map((root) => rm(root, { recursive: true, force: true })),
);
});
const compile = promisify(execFile);
describe("compiled Interface Builder bundle reader", () => {
it("reads nib plist archives and reports provenance", async () => {
const root = await mkdtemp(join(tmpdir(), "rea-ib-test-"));
roots.push(root);
const root = await createTestTempDirectory("rea-ib-test-");
const bundle = join(root, "Example.app");
const nib = join(
bundle,
@@ -78,8 +72,7 @@ describe("compiled Interface Builder bundle reader", () => {
});
it("honors cancellation during directory traversal", async () => {
const root = await mkdtemp(join(tmpdir(), "rea-ib-test-"));
roots.push(root);
const root = await createTestTempDirectory("rea-ib-test-");
const controller = new AbortController();
controller.abort();
await expect(
@@ -94,8 +87,7 @@ describe("compiled Interface Builder bundle reader", () => {
describe("bounded Interface Builder archive decoding", () => {
it("counts malformed archives against the document limit", async () => {
const root = await mkdtemp(join(tmpdir(), "rea-ib-test-"));
roots.push(root);
const root = await createTestTempDirectory("rea-ib-test-");
const bundle = join(root, "Example.app");
const resources = join(bundle, "Contents", "Resources");
await mkdir(resources, { recursive: true });
@@ -122,8 +114,7 @@ describe("bounded Interface Builder archive decoding", () => {
it.skipIf(process.platform !== "darwin" || !existsSync("/usr/bin/ibtool"))(
"decodes an Xcode-compiled storyboard NIB and recovers its UI routes",
async () => {
const root = await mkdtemp(join(tmpdir(), "rea-ib-compiled-test-"));
roots.push(root);
const root = await createTestTempDirectory("rea-ib-compiled-test-");
const bundle = join(root, "Example.app");
const resources = join(bundle, "Contents", "Resources");
const source = join(
@@ -134,7 +125,7 @@ describe("bounded Interface Builder archive decoding", () => {
"MacFixture.storyboard",
);
await mkdir(resources, { recursive: true });
execFileSync("/usr/bin/ibtool", [
await compile("/usr/bin/ibtool", [
"--compile",
join(resources, "MacFixture.storyboardc"),
source,
@@ -1,12 +1,15 @@
import { execFileSync } from "node:child_process";
import { execFile } from "node:child_process";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { decodeNibArchive } from "../artifacts/NibArchive.js";
import { analyzeInterfaceBuilderBundle } from "./InterfaceBuilderAnalysis.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { decodeNibArchive } from "../../../src/artifacts/NibArchive.js";
import { analyzeInterfaceBuilderBundle } from "../../../src/application/InterfaceBuilderAnalysis.js";
const compile = promisify(execFile);
const compileNestedViews = async (depth: number, withWindow = false) => {
const root = await createTestTempDirectory("rea-nib-hierarchy-");
@@ -34,7 +37,7 @@ const compileNestedViews = async (depth: number, withWindow = false) => {
.replace("</view>\n </objects>", "</view></window>\n </objects>");
await writeFile(xib, xibSource);
const nib = join(resources, "Main.nib");
execFileSync("/usr/bin/xcrun", [
await compile("/usr/bin/xcrun", [
"ibtool",
"--errors",
"--warnings",
+22 -18
View File
@@ -11,24 +11,28 @@ import { PRODUCT_IDENTITY } from "../../../src/identity.js";
const pinnedPackage = PRODUCT_IDENTITY.registrationPackageSpecifier;
describe("JSON client configuration transaction", () => {
it("rejects a symlink target not owned by the current user", async () => {
if (typeof process.getuid !== "function") return;
const currentUid = process.getuid();
let statCalls = 0;
expect(
await resolveClientConfigTransactionPath("/config", {
lstat: () => {
statCalls += 1;
return Promise.resolve({
uid: statCalls === 1 ? currentUid : currentUid + 1,
isFile: () => statCalls > 1,
isSymbolicLink: () => statCalls === 1,
});
},
realpath: () => Promise.resolve("/target"),
}),
).toBeUndefined();
});
it.skipIf(typeof process.getuid !== "function")(
"rejects a symlink target not owned by the current user",
async () => {
const currentUid = process.getuid?.();
if (currentUid === undefined)
throw new Error("UID support is required by this test");
let statCalls = 0;
expect(
await resolveClientConfigTransactionPath("/config", {
lstat: () => {
statCalls += 1;
return Promise.resolve({
uid: statCalls === 1 ? currentUid : currentUid + 1,
isFile: () => statCalls > 1,
isSymbolicLink: () => statCalls === 1,
});
},
realpath: () => Promise.resolve("/target"),
}),
).toBeUndefined();
},
);
it("preserves existing keys, creates a backup, and reads back the MCP entry", async () => {
const directory = await createTestTempDirectory("rea-setup-");
+21 -20
View File
@@ -1,7 +1,7 @@
import { readFile, rm } from "node:fs/promises";
import { readFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { afterEach, describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
@@ -11,15 +11,7 @@ import {
} from "../../../src/mcpDoctor.js";
import { CATALOG_IDENTITY } from "../../../src/catalogIdentity.js";
const temporary: string[] = [];
afterEach(async () => {
await Promise.all(
temporary
.splice(0)
.map((path) => rm(path, { recursive: true, force: true })),
);
});
afterEach(() => vi.useRealTimers());
describe("production MCP doctor", () => {
it("connects to the production stdio child and verifies the canonical catalog", async () => {
@@ -78,17 +70,30 @@ describe("production MCP doctor", () => {
it("kills a child that misses the absolute startup deadline", async () => {
const root = await createTestTempDirectory("rea-mcp-doctor-");
temporary.push(root);
const pidPath = join(root, "pid");
const result = await runProductionMcpDoctor({
// Keep filesystem/process I/O real. Advance only the deadline timers,
// after the fixture proves it started and published its PID.
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
const pending = runProductionMcpDoctor({
command: process.execPath,
args: [resolve("tests/fixtures/mcpDoctorHang.mjs")],
cwd: process.cwd(),
environment: { ...process.env, REA_MCP_DOCTOR_PID_PATH: pidPath },
deadlineMs: 250,
deadlineMs: 60_000,
});
const pid = await vi.waitFor(
async () => {
const value = Number.parseInt(await readFile(pidPath, "utf8"), 10);
expect(Number.isSafeInteger(value) && value > 0).toBe(true);
return value;
},
{ timeout: 10_000 },
);
await vi.advanceTimersByTimeAsync(60_000);
const result = await vi.waitFor(() => pending, { timeout: 10_000 });
vi.useRealTimers();
expect(result.healthy).toBe(false);
const pid = Number.parseInt(await readFile(pidPath, "utf8"), 10);
expect(result.checks[0]?.detail).toMatch(/deadline|abort|timed/iu);
await expect(waitForExit(pid)).resolves.toBeUndefined();
}, 10_000);
});
@@ -98,11 +103,7 @@ const waitForExit = async (pid: number): Promise<void> => {
try {
process.kill(pid, 0);
} catch (cause: unknown) {
if (
cause instanceof Error &&
"code" in cause &&
(cause as NodeJS.ErrnoException).code === "ESRCH"
)
if (cause instanceof Error && "code" in cause && cause.code === "ESRCH")
return;
throw cause;
}
@@ -5,7 +5,7 @@ import { access, mkdir } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { afterEach, describe, expect, it } from "vitest";
import { afterEach, describe, expect, it, vi } from "vitest";
import pino from "pino";
import type { Logger } from "../../../../src/logger.js";
@@ -20,6 +20,7 @@ import type {
} from "../../../../src/ghidra/GhidraLauncher.js";
import type { GhidraTransportKind } from "../../../../src/ghidra/GhidraTransport.js";
import { GHIDRA_SESSION_CAPABILITIES } from "../../../../src/ghidra/GhidraSessionValues.js";
import { waitForExit as waitForChildExit } from "../../../support/process/processFixture.js";
const fixturePath = fileURLToPath(
new URL("../../../fixtures/fakeGhidra.mjs", import.meta.url),
@@ -50,10 +51,15 @@ type FixtureMode =
| "exit";
class FixtureLauncher implements GhidraLauncher {
#notifyStarted: ((child: ChildProcess) => void) | undefined;
readonly started = new Promise<ChildProcess>((resolve) => {
this.#notifyStarted = resolve;
});
readonly runtimeRoots: string[] = [];
readonly endpointPaths: string[] = [];
readonly tokens: string[] = [];
readonly processes: ChildProcess[] = [];
readonly requests: string[] = [];
constructor(readonly mode: FixtureMode = "success") {}
@@ -64,7 +70,7 @@ class FixtureLauncher implements GhidraLauncher {
const projectRoot = join(session.runtimeRoot, "project");
await mkdir(projectRoot, { recursive: true });
const process_ = spawn(process.execPath, [fixturePath], {
stdio: ["ignore", "pipe", "pipe"],
stdio: ["ignore", "pipe", "pipe", "ipc"],
env: {
...process.env,
REA_GHIDRA_FIXTURE_CONFIG: JSON.stringify({
@@ -79,7 +85,17 @@ class FixtureLauncher implements GhidraLauncher {
}),
},
});
process_.on("message", (value: unknown) => {
if (
typeof value === "object" &&
value !== null &&
"method" in value &&
typeof value.method === "string"
)
this.requests.push(value.method);
});
this.processes.push(process_);
process_.once("spawn", () => this.#notifyStarted?.(process_));
return ok({
process: process_,
ownsProcessLifetime: true,
@@ -120,6 +136,7 @@ const clientFor = (
};
afterEach(async () => {
vi.useRealTimers();
await Promise.all(clients.splice(0).map((client) => client.close()));
});
@@ -246,10 +263,15 @@ describe("GhidraClient", () => {
describe("GhidraClient startup lifecycle", () => {
it("applies one startup deadline and removes the private runtime", async () => {
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout", "Date"] });
const launcher = new FixtureLauncher("silent");
const client = clientFor(launcher, { startupTimeoutMs: 30 });
const result = await client.start();
const pending = client.start();
const child = await launcher.started;
await vi.advanceTimersByTimeAsync(30);
const result = await vi.waitFor(() => pending, { timeout: 10_000 });
vi.useRealTimers();
expect(result).toMatchObject({
ok: false,
@@ -259,7 +281,7 @@ describe("GhidraClient startup lifecycle", () => {
code: "ENOENT",
});
await client.close();
await expect(waitForExit(launcher.processes[0])).resolves.toBe(true);
await expect(waitForExit(child)).resolves.toBe(true);
});
it("cancels startup promptly and leaves no project or process", async () => {
@@ -267,7 +289,8 @@ describe("GhidraClient startup lifecycle", () => {
const client = clientFor(launcher, { startupTimeoutMs: 10_000 });
const controller = new AbortController();
const pending = client.start(controller.signal);
setTimeout(() => controller.abort(), 10);
const process_ = await launcher.started;
controller.abort();
const result = await pending;
@@ -276,9 +299,7 @@ describe("GhidraClient startup lifecycle", () => {
code: "ENOENT",
});
await client.close();
const process_ = launcher.processes[0];
if (process_ !== undefined)
await expect(waitForExit(process_)).resolves.toBe(true);
await expect(waitForExit(process_)).resolves.toBe(true);
});
});
@@ -305,7 +326,9 @@ describe("GhidraClient established requests", () => {
{ document: null, procedure: "fixture_main" },
{ signal: activeController.signal },
);
await wait(5);
await vi.waitFor(() =>
expect(launcher.requests).toContain("procedure_pseudo_code"),
);
const queuedController = new AbortController();
const queued = client.callTool(
"procedure_info",
@@ -322,7 +345,7 @@ describe("GhidraClient established requests", () => {
document: null,
procedure: "fixture_main",
});
await wait(5);
expect(launcher.requests).not.toContain("procedure_info");
const process_ = launcher.processes[0];
activeController.abort();
await expect(active).resolves.toMatchObject({
@@ -404,15 +427,9 @@ const exited = (process_: ChildProcess | undefined): boolean =>
const waitForExit = async (
process_: ChildProcess | undefined,
): Promise<boolean> => {
for (let attempt = 0; attempt < 1_000 && !exited(process_); attempt += 1) {
await wait(10);
}
return exited(process_);
return process_ !== undefined && waitForChildExit(process_, 10_000);
};
const wait = (milliseconds: number): Promise<void> =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
describe("GhidraClient rejected requests", () => {
it("redacts authentication after failed startup cleanup and retry", async () => {
const fixture = new FixtureLauncher();
@@ -1,8 +1,9 @@
import { access, chmod, readFile, rm, stat } from "node:fs/promises";
import { access, chmod, readFile, stat } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { z } from "zod";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { createGhidraTestRuntime } from "../../../fixtures/ghidraRuntime.js";
@@ -22,9 +23,26 @@ const fixturePath = fileURLToPath(
import.meta.url,
),
);
const roots: string[] = [];
const runtimes: PrivateRuntimeRoot[] = [];
const launchCaptureSchema = z.object({
arguments: z.array(z.string()),
environment: z.record(z.string(), z.string()),
descriptor_mode: z.number().int(),
descriptor_has_token: z.boolean(),
});
const expectOptions = (
arguments_: readonly string[],
options: readonly (readonly [string, string])[],
): void => {
for (const [flag, value] of options) {
const index = arguments_.indexOf(flag);
expect(index, flag).toBeGreaterThanOrEqual(0);
expect(arguments_[index + 1], flag).toBe(value);
}
};
const expectIsolatedEnvironment = (
environment: Record<string, string>,
runtimeRoot: string,
@@ -68,9 +86,6 @@ beforeAll(async () => {
afterEach(async () => {
vi.unstubAllEnvs();
await Promise.all(runtimes.splice(0).map((runtime) => runtime.close()));
await Promise.all(
roots.splice(0).map((path) => rm(path, { recursive: true, force: true })),
);
});
describe("Ghidra COM loader", () => {
@@ -84,15 +99,11 @@ describe("Ghidra COM loader", () => {
scriptLogPath: "/tmp/script.log",
dosCom: true,
});
expect(arguments_.slice(4, 12)).toEqual([
"-loader",
"BinaryLoader",
"-loader-baseAddr",
"1000:0100",
"-processor",
"x86:LE:16:Real Mode",
"-cspec",
"default",
expectOptions(arguments_, [
["-loader", "BinaryLoader"],
["-loader-baseAddr", "1000:0100"],
["-processor", "x86:LE:16:Real Mode"],
["-cspec", "default"],
]);
const prepare = arguments_.indexOf("-preScript");
expect(arguments_[prepare + 1]).toBe(
@@ -115,44 +126,36 @@ describe("Ghidra headless launcher", () => {
scriptLogPath: "/tmp/script.log",
dosMz: true,
});
expect(arguments_.slice(4, 10)).toEqual([
"-loader",
"MzLoader",
"-processor",
"x86:LE:16:Real Mode",
"-cspec",
"default",
expectOptions(arguments_, [
["-loader", "MzLoader"],
["-processor", "x86:LE:16:Real Mode"],
["-cspec", "default"],
]);
expect(arguments_).toContain("-readOnly");
expect(arguments_).toContain("-deleteProject");
});
it("builds a read-only import in deterministic order", () => {
expect(
ghidraHeadlessArguments({
projectRoot: "/tmp/project",
targetPath: "/tmp/target",
bridgeScriptPath: "/package/bridge/ReaGhidraBridge.java",
descriptorPath: "/tmp/session.json",
ghidraLogPath: "/tmp/ghidra.log",
scriptLogPath: "/tmp/script.log",
}),
).toEqual([
"/tmp/project",
"rea-project",
"-import",
"/tmp/target",
"-readOnly",
"-deleteProject",
"-log",
"/tmp/ghidra.log",
"-scriptlog",
"/tmp/script.log",
"-scriptPath",
"/package/bridge",
"-postScript",
"/package/bridge/ReaGhidraBridge.java",
"/tmp/session.json",
it("builds a read-only ephemeral import with its bridge descriptor", () => {
const arguments_ = ghidraHeadlessArguments({
projectRoot: "/tmp/project",
targetPath: "/tmp/target",
bridgeScriptPath: "/package/bridge/ReaGhidraBridge.java",
descriptorPath: "/tmp/session.json",
ghidraLogPath: "/tmp/ghidra.log",
scriptLogPath: "/tmp/script.log",
});
expect(arguments_.slice(0, 2)).toEqual(["/tmp/project", "rea-project"]);
expectOptions(arguments_, [
["-import", "/tmp/target"],
["-log", "/tmp/ghidra.log"],
["-scriptlog", "/tmp/script.log"],
["-scriptPath", "/package/bridge"],
["-postScript", "/package/bridge/ReaGhidraBridge.java"],
]);
expect(arguments_[arguments_.indexOf("-postScript") + 2]).toBe(
"/tmp/session.json",
);
expect(arguments_).toContain("-readOnly");
expect(arguments_).toContain("-deleteProject");
});
it("wraps the Windows batch launcher without enabling a Node shell", () => {
@@ -203,7 +206,6 @@ describe("Ghidra headless launcher", () => {
vi.stubEnv("JDK_JAVA_OPTIONS", "-XX:MaxRAMPercentage=99");
vi.stubEnv("_JAVA_OPTIONS", "-Xmx99G");
const parent = await createTestTempDirectory("rea-launcher-test-");
roots.push(parent);
const runtime = await createGhidraTestRuntime(parent);
runtimes.push(runtime);
const runtimeRoot = runtime.path;
@@ -229,8 +231,10 @@ describe("Ghidra headless launcher", () => {
expect(launched.ok).toBe(true);
if (!launched.ok) return;
const capturePath = join(runtimeRoot, "launch-capture.json");
await waitFor(capturePath);
const capture = JSON.parse(await readFile(capturePath, "utf8"));
await vi.waitFor(() => access(`${capturePath}.ready`), { timeout: 10_000 });
const capture = launchCaptureSchema.parse(
JSON.parse(await readFile(capturePath, "utf8")),
);
const encodedArguments = JSON.stringify(capture.arguments);
const encodedEnvironment = JSON.stringify(capture.environment);
expect(encodedArguments).not.toContain(token);
@@ -255,15 +259,3 @@ describe("Ghidra headless launcher", () => {
await expect(access(join(runtimeRoot, "project"))).resolves.toBeUndefined();
});
});
const waitFor = async (path: string): Promise<void> => {
for (let attempt = 0; attempt < 100; attempt += 1) {
try {
await access(path);
return;
} catch {
await new Promise((resolve) => setTimeout(resolve, 10));
}
}
throw new Error(`Timed out waiting for ${path}`);
};
@@ -1,6 +1,5 @@
import { spawn, type ChildProcess } from "node:child_process";
import { access } from "node:fs/promises";
import { fileURLToPath } from "node:url";
import { afterEach, describe, expect, it } from "vitest";
@@ -14,44 +13,11 @@ import { HopperClient } from "../../../../src/hopper/HopperClient.js";
import { cleanupOwnedProcessGroup } from "../../../../src/process/ProcessOwnership.js";
import { spawnOwnedProviderProcess } from "../../../../src/process/ProviderProcess.js";
const fixturePath = fileURLToPath(
new URL("../../../fixtures/fakeHopper.mjs", import.meta.url),
);
class FixtureLauncher implements BridgeLauncher {
socketPaths: string[] = [];
directories: string[] = [];
runIds: string[] = [];
processes: ChildProcess[] = [];
constructor(readonly tokenOverride?: string) {}
launch(session: BridgeSession) {
this.socketPaths.push(session.socketPath);
this.directories.push(session.directory);
this.runIds.push(session.runId);
const child = spawn(
process.execPath,
[
fixturePath,
session.socketPath,
this.tokenOverride ?? session.token,
session.runId,
],
{
stdio: ["ignore", "ignore", "pipe"],
},
);
this.processes.push(child);
return Promise.resolve(
ok({
process: child,
ownsProcessLifetime: true,
shutdownMode: "bridge-request" as const,
}),
);
}
}
import {
HopperFixtureLauncher as FixtureLauncher,
hopperFixturePath as fixturePath,
startHopperFixtureClient as startClient,
} from "./hopperClient.fixture.js";
class OwnedFixtureLauncher implements BridgeLauncher {
ownership:
@@ -131,18 +97,6 @@ class UnconfirmedOwnedFixtureLauncher implements BridgeLauncher {
}
const clients: HopperClient[] = [];
const startClient = async () => {
const client = new HopperClient({
launcher: new FixtureLauncher(),
startupTimeoutMs: 1_000,
});
clients.push(client);
await expect(client.start()).resolves.toEqual({
ok: true,
value: { name: "REA Hopper bridge", version: "1.0.0" },
});
return client;
};
afterEach(async () => {
await Promise.all(clients.splice(0).map((client) => client.close()));
@@ -150,12 +104,15 @@ afterEach(async () => {
describe("HopperClient cleanup", () => {
it("waits for an operation reply without a fixed request deadline", async () => {
const client = await startClient();
await expect(
client.callTool("echo", { value: "waited", delay: 150 }),
).resolves.toEqual({
const launcher = new FixtureLauncher();
const client = await startClient(launcher);
const pending = client.callTool("echo", { value: "waited", gate: "reply" });
const request = await launcher.waitForRequest("echo", "reply");
expect(client.requestActivity()).toMatchObject({ operation: "echo" });
await launcher.release(request);
await expect(pending).resolves.toEqual({
ok: true,
value: { value: "waited", delay: 150 },
value: { value: "waited", gate: "reply" },
});
});
@@ -0,0 +1,112 @@
import { spawn, type ChildProcess } from "node:child_process";
import { fileURLToPath } from "node:url";
import { onTestFinished, vi } from "vitest";
import { z } from "zod";
import { ok } from "../../../../src/domain/result.js";
import type {
BridgeLauncher,
BridgeSession,
} from "../../../../src/hopper/BridgeLauncher.js";
import { HopperClient } from "../../../../src/hopper/HopperClient.js";
export const hopperFixturePath = fileURLToPath(
new URL("../../../fixtures/fakeHopper.mjs", import.meta.url),
);
const requestSchema = z.object({
id: z.number().int(),
method: z.string(),
params: z.record(z.string(), z.unknown()).optional(),
});
type FixtureRequest = z.infer<typeof requestSchema>;
/** Real socket fixture with request acknowledgements and explicit response gates. */
export class HopperFixtureLauncher implements BridgeLauncher {
readonly socketPaths: string[] = [];
readonly directories: string[] = [];
readonly runIds: string[] = [];
readonly processes: ChildProcess[] = [];
readonly requests: FixtureRequest[] = [];
constructor(readonly tokenOverride?: string) {}
launch(session: BridgeSession) {
this.socketPaths.push(session.socketPath);
this.directories.push(session.directory);
this.runIds.push(session.runId);
const child = spawn(
process.execPath,
[
hopperFixturePath,
session.socketPath,
this.tokenOverride ?? session.token,
session.runId,
],
{ stdio: ["ignore", "ignore", "pipe", "ipc"] },
);
child.on("message", (value: unknown) => {
const request = requestSchema.safeParse(value);
if (request.success) this.requests.push(request.data);
});
this.processes.push(child);
return Promise.resolve(
ok({
process: child,
ownsProcessLifetime: true as const,
shutdownMode: "bridge-request" as const,
}),
);
}
/** Wait for acknowledgement that the fixture actually received a request. */
waitForRequest(method: string, gate?: string): Promise<FixtureRequest> {
return vi.waitFor(
() => {
const request = this.requests.find(
(value) =>
value.method === method &&
(gate === undefined || value.params?.gate === gate),
);
if (request === undefined)
throw new Error(`Waiting for Hopper fixture request ${method}`);
return request;
},
{ timeout: 10_000 },
);
}
/** Release a response held by the fixture's gate parameter. */
release(request: FixtureRequest): Promise<void> {
const child = this.processes.at(-1);
if (child === undefined) throw new Error("Hopper fixture was not launched");
return new Promise((resolve, reject) => {
child.send({ release: request.id }, (error) => {
if (error !== null) reject(error);
else resolve();
});
});
}
}
/** Register owned client cleanup before any startup assertion can fail. */
export const trackHopperClient = (client: HopperClient): HopperClient => {
onTestFinished(() => client.close());
return client;
};
/** Start a real fixture session with test-scoped cleanup. */
export const startHopperFixtureClient = async (
launcher = new HopperFixtureLauncher(),
): Promise<HopperClient> => {
const client = trackHopperClient(
new HopperClient({
launcher,
startupTimeoutMs: 10_000,
}),
);
const started = await client.start();
if (!started.ok) throw started.error;
return client;
};
@@ -1,75 +1,14 @@
import { spawn, type ChildProcess } from "node:child_process";
import { fileURLToPath } from "node:url";
import { describe, expect, it, vi } from "vitest";
import { afterEach, describe, expect, it } from "vitest";
import { ok } from "../../../../src/domain/result.js";
import type {
BridgeLauncher,
BridgeSession,
} from "../../../../src/hopper/BridgeLauncher.js";
import { HopperClient } from "../../../../src/hopper/HopperClient.js";
const fixturePath = fileURLToPath(
new URL("../../../fixtures/fakeHopper.mjs", import.meta.url),
);
class FixtureLauncher implements BridgeLauncher {
socketPaths: string[] = [];
directories: string[] = [];
runIds: string[] = [];
processes: ChildProcess[] = [];
constructor(readonly tokenOverride?: string) {}
launch(session: BridgeSession) {
this.socketPaths.push(session.socketPath);
this.directories.push(session.directory);
this.runIds.push(session.runId);
const child = spawn(
process.execPath,
[
fixturePath,
session.socketPath,
this.tokenOverride ?? session.token,
session.runId,
],
{
stdio: ["ignore", "ignore", "pipe"],
},
);
this.processes.push(child);
return Promise.resolve(
ok({
process: child,
ownsProcessLifetime: true as const,
shutdownMode: "bridge-request" as const,
}),
);
}
}
const clients: HopperClient[] = [];
const startClient = async () => {
const client = new HopperClient({
launcher: new FixtureLauncher(),
startupTimeoutMs: 1_000,
});
clients.push(client);
await expect(client.start()).resolves.toEqual({
ok: true,
value: { name: "REA Hopper bridge", version: "1.0.0" },
});
return client;
};
afterEach(async () => {
await Promise.all(clients.splice(0).map((client) => client.close()));
});
import {
HopperFixtureLauncher,
startHopperFixtureClient as startClient,
} from "./hopperClient.fixture.js";
describe("HopperClient progress", () => {
it("forwards correlated Python progress before the terminal response", async () => {
const client = await startClient();
const launcher = new HopperFixtureLauncher();
const client = await startClient(launcher);
const updates: Array<{
readonly phase: string;
readonly completed: number;
@@ -77,7 +16,7 @@ describe("HopperClient progress", () => {
}> = [];
const pending = client.callTool(
"echo",
{ value: "progress" },
{ value: "progress", gate: "progress" },
{
progress: {
report: (update) => {
@@ -87,9 +26,20 @@ describe("HopperClient progress", () => {
},
},
);
const request = await launcher.waitForRequest("echo", "progress");
await vi.waitFor(() =>
expect(updates).toContainEqual(
expect.objectContaining({ phase: "hopper_bridge", completed: 0 }),
),
);
expect(updates.some(({ terminal }) => terminal === true)).toBe(false);
await launcher.release(request);
const result = await pending;
expect(result).toEqual({ ok: true, value: { value: "progress" } });
expect(result).toEqual({
ok: true,
value: { value: "progress", gate: "progress" },
});
expect(updates).toEqual(
expect.arrayContaining([
expect.objectContaining({
@@ -136,17 +86,22 @@ describe("HopperClient progress", () => {
});
it("cancels and ignores late responses without corrupting the session", async () => {
const client = await startClient();
const launcher = new HopperFixtureLauncher();
const client = await startClient(launcher);
const controller = new AbortController();
const pending = client.callTool("hang", {}, { signal: controller.signal });
const pending = client.callTool(
"echo",
{ gate: "late-reply" },
{ signal: controller.signal },
);
const request = await launcher.waitForRequest("echo", "late-reply");
controller.abort();
const cancelled = await pending;
expect(cancelled.ok).toBe(false);
if (!cancelled.ok)
expect(cancelled.error._tag).toBe("HopperCancelledError");
const waited = await client.callTool("echo", { delay: 30 });
expect(waited).toMatchObject({ ok: true });
await launcher.release(request);
await expect(client.callTool("echo", { value: "alive" })).resolves.toEqual({
ok: true,
value: { value: "alive" },
@@ -154,12 +109,13 @@ describe("HopperClient progress", () => {
});
it("retains cancelled bridge activity until the reply releases the queue", async () => {
const client = await startClient();
const launcher = new HopperFixtureLauncher();
const client = await startClient(launcher);
const progress: string[] = [];
const controller = new AbortController();
const pending = client.callTool(
"echo",
{ delay: 60 },
{ gate: "cancel-active" },
{
signal: controller.signal,
progress: {
@@ -170,7 +126,7 @@ describe("HopperClient progress", () => {
},
},
);
await new Promise((resolve) => setTimeout(resolve, 5));
const request = await launcher.waitForRequest("echo", "cancel-active");
controller.abort();
const result = await pending;
@@ -187,8 +143,8 @@ describe("HopperClient progress", () => {
expect.stringContaining("started on Hopper's serial bridge"),
]),
);
await new Promise((resolve) => setTimeout(resolve, 70));
expect(client.requestActivity()).toBeNull();
await launcher.release(request);
await vi.waitFor(() => expect(client.requestActivity()).toBeNull());
await expect(client.callTool("echo", { alive: true })).resolves.toEqual({
ok: true,
value: { alive: true },
@@ -1,74 +1,14 @@
import { spawn, type ChildProcess } from "node:child_process";
import { fileURLToPath } from "node:url";
import { afterEach, describe, expect, it } from "vitest";
import { describe, expect, it, onTestFinished, vi } from "vitest";
import { HOPPER_PROVIDER_TOOL_CONTRACTS } from "../../../../src/hopper/HopperProvider.js";
import { ok } from "../../../../src/domain/result.js";
import type {
BridgeLauncher,
BridgeSession,
} from "../../../../src/hopper/BridgeLauncher.js";
import { HopperClient } from "../../../../src/hopper/HopperClient.js";
import type { HopperDiagnostic } from "../../../../src/hopper/HopperDiagnostics.js";
import { providerCleanupFailure } from "../../../../src/hopper/HopperDiagnostics.js";
const fixturePath = fileURLToPath(
new URL("../../../fixtures/fakeHopper.mjs", import.meta.url),
);
class FixtureLauncher implements BridgeLauncher {
socketPaths: string[] = [];
directories: string[] = [];
runIds: string[] = [];
processes: ChildProcess[] = [];
constructor(readonly tokenOverride?: string) {}
launch(session: BridgeSession) {
this.socketPaths.push(session.socketPath);
this.directories.push(session.directory);
this.runIds.push(session.runId);
const child = spawn(
process.execPath,
[
fixturePath,
session.socketPath,
this.tokenOverride ?? session.token,
session.runId,
],
{
stdio: ["ignore", "ignore", "pipe"],
},
);
this.processes.push(child);
return Promise.resolve(
ok({
process: child,
ownsProcessLifetime: true as const,
shutdownMode: "bridge-request" as const,
}),
);
}
}
const clients: HopperClient[] = [];
const startClient = async () => {
const client = new HopperClient({
launcher: new FixtureLauncher(),
startupTimeoutMs: 1_000,
});
clients.push(client);
await expect(client.start()).resolves.toEqual({
ok: true,
value: { name: "REA Hopper bridge", version: "1.0.0" },
});
return client;
};
afterEach(async () => {
await Promise.all(clients.splice(0).map((client) => client.close()));
});
import {
HopperFixtureLauncher as FixtureLauncher,
startHopperFixtureClient as startClient,
} from "./hopperClient.fixture.js";
describe("HopperClient protocol", () => {
it("sanitizes unexpected owned-cleanup exceptions", () => {
@@ -89,7 +29,7 @@ describe("HopperClient protocol", () => {
runId: "11111111-1111-4111-8111-111111111111",
startupTimeoutMs: 1_000,
});
clients.push(client);
onTestFinished(() => client.close());
expect((await client.start()).ok).toBe(true);
expect(launcher.runIds).toEqual(["11111111-1111-4111-8111-111111111111"]);
@@ -98,25 +38,36 @@ describe("HopperClient protocol", () => {
it("keeps the native socket path below macOS sockaddr_un limits", async () => {
const launcher = new FixtureLauncher();
const client = new HopperClient({ launcher, startupTimeoutMs: 1_000 });
clients.push(client);
onTestFinished(() => client.close());
const started = await client.start();
expect(started.ok).toBe(true);
expect(Buffer.byteLength(launcher.socketPaths[0] ?? "")).toBeLessThan(104);
});
it("serializes concurrent calls until the active Hopper reply arrives", async () => {
const client = await startClient();
const slow = client.callTool("echo", { label: "slow", delay: 30 });
const fast = client.callTool("echo", { label: "fast", delay: 1 });
const launcher = new FixtureLauncher();
const client = await startClient(launcher);
const slow = client.callTool("echo", { label: "slow", gate: "serial" });
const request = await launcher.waitForRequest("echo", "serial");
const fast = client.callTool("echo", { label: "fast" });
let fastSettled = false;
void fast.then(() => {
fastSettled = true;
});
await new Promise((resolve) => setTimeout(resolve, 10));
await vi.waitFor(() =>
expect(client.requestActivity()).toMatchObject({
operation: "echo",
queuedRequests: 1,
}),
);
expect(fastSettled).toBe(false);
expect(launcher.requests.filter(({ method }) => method === "echo")).toEqual(
[request],
);
await launcher.release(request);
await expect(Promise.all([slow, fast])).resolves.toEqual([
{ ok: true, value: { label: "slow", delay: 30 } },
{ ok: true, value: { label: "fast", delay: 1 } },
{ ok: true, value: { label: "slow", gate: "serial" } },
{ ok: true, value: { label: "fast" } },
]);
});
@@ -136,7 +87,7 @@ describe("HopperClient protocol", () => {
launcher: new FixtureLauncher("wrong-token"),
startupTimeoutMs: 1_000,
});
clients.push(client);
onTestFinished(() => client.close());
const result = await client.start();
expect(result.ok).toBe(false);
if (!result.ok) expect(result.error._tag).toBe("HopperRemoteError");
@@ -150,24 +101,29 @@ describe("HopperClient protocol", () => {
["remote_error", "HopperRemoteError"],
["exit", "HopperProcessError"],
])("projects %s as %s", async (method, expectedTag) => {
const client = await startClient();
const launcher = new FixtureLauncher();
const client = await startClient(launcher);
const result = await client.callTool(method);
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error._tag).toBe(expectedTag);
if (result.error._tag === "HopperRemoteError")
expect(result.error).toMatchObject({ operation: method, requestId: 2 });
expect(result.error).toMatchObject({
operation: method,
requestId: (await launcher.waitForRequest(method)).id,
});
}
});
it("preserves a sanitized bridge exception diagnostic", async () => {
const launcher = new FixtureLauncher();
const diagnostics: HopperDiagnostic[] = [];
const client = new HopperClient({
launcher: new FixtureLauncher(),
launcher,
startupTimeoutMs: 1_000,
onDiagnostic: (event) => diagnostics.push(event),
});
clients.push(client);
onTestFinished(() => client.close());
await expect(client.start()).resolves.toMatchObject({ ok: true });
const result = await client.callTool("remote_error");
expect(result.ok).toBe(false);
@@ -178,7 +134,7 @@ describe("HopperClient protocol", () => {
});
expect(diagnostics).toContainEqual({
type: "bridge-diagnostic",
request_id: 2,
request_id: (await launcher.waitForRequest("remote_error")).id,
code: -32001,
category: "bridge_exception",
message: "safe fake failure",
@@ -1,71 +1,10 @@
import { spawn, type ChildProcess } from "node:child_process";
import { fileURLToPath } from "node:url";
import { afterEach, describe, expect, it } from "vitest";
import { ok } from "../../../../src/domain/result.js";
import type {
BridgeLauncher,
BridgeSession,
} from "../../../../src/hopper/BridgeLauncher.js";
import { describe, expect, it, onTestFinished } from "vitest";
import { HopperClient } from "../../../../src/hopper/HopperClient.js";
const fixturePath = fileURLToPath(
new URL("../../../fixtures/fakeHopper.mjs", import.meta.url),
);
class FixtureLauncher implements BridgeLauncher {
socketPaths: string[] = [];
directories: string[] = [];
runIds: string[] = [];
processes: ChildProcess[] = [];
constructor(readonly tokenOverride?: string) {}
launch(session: BridgeSession) {
this.socketPaths.push(session.socketPath);
this.directories.push(session.directory);
this.runIds.push(session.runId);
const child = spawn(
process.execPath,
[
fixturePath,
session.socketPath,
this.tokenOverride ?? session.token,
session.runId,
],
{
stdio: ["ignore", "ignore", "pipe"],
},
);
this.processes.push(child);
return Promise.resolve(
ok({
process: child,
ownsProcessLifetime: true as const,
shutdownMode: "bridge-request" as const,
}),
);
}
}
const clients: HopperClient[] = [];
const startClient = async () => {
const client = new HopperClient({
launcher: new FixtureLauncher(),
startupTimeoutMs: 1_000,
});
clients.push(client);
await expect(client.start()).resolves.toEqual({
ok: true,
value: { name: "REA Hopper bridge", version: "1.0.0" },
});
return client;
};
afterEach(async () => {
await Promise.all(clients.splice(0).map((client) => client.close()));
});
import {
HopperFixtureLauncher as FixtureLauncher,
startHopperFixtureClient as startClient,
} from "./hopperClient.fixture.js";
describe("HopperClient restart", () => {
it("does not finish startup after an immediate close", async () => {
@@ -73,7 +12,7 @@ describe("HopperClient restart", () => {
launcher: new FixtureLauncher(),
startupTimeoutMs: 1_000,
});
clients.push(client);
onTestFinished(() => client.close());
const starting = client.start();
await client.close();
@@ -1,6 +1,5 @@
import { spawn, type ChildProcess } from "node:child_process";
import { access } from "node:fs/promises";
import { fileURLToPath } from "node:url";
import { afterEach, describe, expect, it } from "vitest";
@@ -14,44 +13,7 @@ import type {
import { HopperClient } from "../../../../src/hopper/HopperClient.js";
import { LINUX_PRIVATE_DISPLAY_DIAGNOSTIC_PREFIX } from "../../../../src/hopper/LinuxPrivateDisplayDiagnostic.js";
const fixturePath = fileURLToPath(
new URL("../../../fixtures/fakeHopper.mjs", import.meta.url),
);
class FixtureLauncher implements BridgeLauncher {
socketPaths: string[] = [];
directories: string[] = [];
runIds: string[] = [];
processes: ChildProcess[] = [];
constructor(readonly tokenOverride?: string) {}
launch(session: BridgeSession) {
this.socketPaths.push(session.socketPath);
this.directories.push(session.directory);
this.runIds.push(session.runId);
const child = spawn(
process.execPath,
[
fixturePath,
session.socketPath,
this.tokenOverride ?? session.token,
session.runId,
],
{
stdio: ["ignore", "ignore", "pipe"],
},
);
this.processes.push(child);
return Promise.resolve(
ok({
process: child,
ownsProcessLifetime: true as const,
shutdownMode: "bridge-request" as const,
}),
);
}
}
import { HopperFixtureLauncher as FixtureLauncher } from "./hopperClient.fixture.js";
class SilentLauncher implements BridgeLauncher {
launch() {
@@ -175,12 +137,10 @@ describe("HopperClient startup failures", () => {
// Cancel after launch: a timer can fire during runtime-directory creation,
// leaving the silent first launch to be consumed by the retry instead.
const firstProcess = await launcher.firstLaunch;
const startedAt = Date.now();
controller.abort();
const result = await pending;
expect(result.ok).toBe(false);
if (!result.ok) expect(result.error._tag).toBe("HopperCancelledError");
expect(Date.now() - startedAt).toBeLessThan(500);
expect(
firstProcess.exitCode !== null || firstProcess.signalCode !== null,
).toBe(true);
+5 -1
View File
@@ -8,8 +8,9 @@ const descriptorPath = arguments_.at(-1);
if (descriptorPath === undefined) process.exit(64);
const descriptor = JSON.parse(await readFile(descriptorPath, "utf8"));
const runtimeRoot = dirname(descriptorPath);
const capturePath = join(runtimeRoot, "launch-capture.json");
await writeFile(
join(runtimeRoot, "launch-capture.json"),
capturePath,
`${JSON.stringify({
arguments: arguments_,
environment: {
@@ -38,4 +39,7 @@ await writeFile(
})}\n`,
{ mode: 0o600 },
);
// A Windows private runtime retains directory handles that restrict renames.
// Publish readiness only after writeFile has closed the complete JSON capture.
await writeFile(`${capturePath}.ready`, "", { flag: "wx", mode: 0o600 });
setInterval(() => undefined, 1_000);
+1
View File
@@ -72,6 +72,7 @@ const onSocketData = (socket, server, chunk, state) => {
state.buffer.value = state.buffer.value.slice(newline + 1);
newline = state.buffer.value.indexOf("\n");
const request = JSON.parse(line);
process.send?.({ method: request.method });
if (request.token !== state.token) {
socket.end(
`${JSON.stringify({
+25 -3
View File
@@ -2,6 +2,14 @@ import { createServer } from "node:net";
const [socketPath, token, runId, shutdownMode = "acknowledge"] =
process.argv.slice(2);
const heldReplies = new Map();
if (process.send !== undefined)
process.on("message", (message) => {
const reply = heldReplies.get(message?.release);
if (reply === undefined) return;
heldReplies.delete(message.release);
reply();
});
const shutdownResult = () => ({
shutdown: true,
analysis_stopped: true,
@@ -115,7 +123,7 @@ const sendEchoWithProgress = (send, request) => {
message: "Hopper bridge started request",
},
});
setTimeout(() => {
const reply = () => {
send({
id: request.id,
event: {
@@ -128,7 +136,10 @@ const sendEchoWithProgress = (send, request) => {
},
});
send({ id: request.id, result: request.params ?? {} });
}, request.params?.delay ?? 0);
};
if (typeof request.params?.gate === "string")
heldReplies.set(request.id, reply);
else setTimeout(reply, request.params?.delay ?? 0);
};
const server = createServer((socket) => {
@@ -147,6 +158,11 @@ const server = createServer((socket) => {
let newline = buffer.indexOf("\n");
while (newline >= 0) {
const request = JSON.parse(buffer.slice(0, newline));
process.send?.({
id: request.id,
method: request.method,
params: request.params,
});
const bridgeEventMessages = bridgeEventFixtureMessages(request);
buffer = buffer.slice(newline + 1);
if (request.token !== token) {
@@ -169,7 +185,13 @@ const server = createServer((socket) => {
) {
send({ id: request.id, result: shutdownResult() });
if (shutdownMode !== "cleanup-required")
setTimeout(() => server.close(), 2);
setTimeout(
() =>
server.close(() => {
if (process.connected) process.disconnect();
}),
2,
);
} else if (request.method === "hang") {
// Deliberately leave the request pending.
} else if (request.method === "exit") {