Compare commits

..
59 changed files with 407 additions and 1872 deletions

No files matched your search

+6 -9
View File
@@ -185,9 +185,8 @@ package that installed it. A Debian package records its name, version,
architecture and the apt sources that list the installed version, with the architecture and the apt sources that list the installed version, with the
origin (`o=`) apt reports for each source in `source_origins`, such as `Ubuntu` origin (`o=`) apt reports for each source in `source_origins`, such as `Ubuntu`
for the Ubuntu archive and its mirrors or `LP-PPA-<name>` for a PPA; an empty for the Ubuntu archive and its mirrors or `LP-PPA-<name>` for a PPA; an empty
source list means apt reports no source for the installed version. source list means the version came from a local `.deb` or the package lists
`package_lists_present` reports whether the lists are present. It is `null` are missing, which `package_lists_present` tells apart. apt's lists folder,
when REA cannot read the lists directory. apt's lists folder,
and the trusted keys below, are where `apt-config` resolves and the trusted keys below, are where `apt-config` resolves
`Dir::State::Lists`, `Dir::Etc::TrustedParts` and `Dir::Etc::Trusted` under `Dir::State::Lists`, `Dir::Etc::TrustedParts` and `Dir::Etc::Trusted` under
`Dir`, so an `APT_CONFIG` file that moves them is followed: on a computer `Dir`, so an `APT_CONFIG` file that moves them is followed: on a computer
@@ -215,9 +214,9 @@ does not bring it. A program link that dpkg does not own counts as owned when
dpkg owns the file it resolves to, as an alternatives link does. An dpkg owns the file it resolves to, as an alternatives link does. An
`--executable` launch, or an entry no package owns, such as `--executable` launch, or an entry no package owns, such as
one in `~/.local/share/applications`, records the executable's package. When one in `~/.local/share/applications`, records the executable's package. When
`dpkg-query` fails for the executable or entry, or the selected path contains `dpkg-query` does not finish or exits with an error for the executable or the
characters `dpkg-query -S` would interpret as a pattern, package ownership is entry, the package is `unknown` and the limitations say which path, since a
`unknown` and the limitations name the path. The sources, vendor failed lookup is not proof that no package owns it. The sources, vendor
repositories and `--package-file` match below are the recorded package's; the repositories and `--package-file` match below are the recorded package's; the
library walk still reads the executable. library walk still reads the executable.
@@ -244,9 +243,7 @@ writes it on Ubuntu 24.04, has that key as its only candidate, as apt trusts
no other for it: REA reads the field's continuation lines as deb822 writes no other for it: REA reads the field's continuation lines as deb822 writes
them, a `.` line standing for an empty one, from the key block's `BEGIN` line them, a `.` line standing for an empty one, from the key block's `BEGIN` line
to its last `END` line, and records the key's path as the entry's sources to its last `END` line, and records the key's path as the entry's sources
file. A fingerprint restriction in `Signed-By` leaves `signing_key` null. file. A key counts when gpgv reports a valid signature by it and no bad,
The reason names the restriction that REA could not verify. A key counts
when gpgv reports a valid signature by it and no bad,
expired, revoked or unverifiable signature by it, whatever gpgv's exit status: expired, revoked or unverifiable signature by it, whatever gpgv's exit status:
a release also signed by a key the host lacks makes gpgv fail. Each candidate a release also signed by a key the host lacks makes gpgv fail. Each candidate
is copied, an armored key decoded to a binary keyring, into a private is copied, an armored key decoded to a binary keyring, into a private
-3
View File
@@ -336,9 +336,6 @@ A withheld file larger than 48 KiB is refused, since its base64 must fit one
environment variable. `--withheld-copy <folder>` copies the withheld files, environment variable. `--withheld-copy <folder>` copies the withheld files,
as captured, into a new or empty folder outside the bundle. The result lists as captured, into a new or empty folder outside the bundle. The result lists
them as `withheld_files`, each with its `setting`, `variable` and `handle`. them as `withheld_files`, each with its `setting`, `variable` and `handle`.
REA resolves both destinations before writing either. A path that cannot be
resolved, including a dangling link, is refused; the copy and bundle cannot
contain one another, including through symbolic links.
`--consent <file>` ships the person's consents, a `consent.json` that `--consent <file>` ships the person's consents, a `consent.json` that
`record_build_consent` records, as the bundle's own `consent.json`. `record_build_consent` records, as the bundle's own `consent.json`.
@@ -14,22 +14,17 @@ import { join } from "node:path";
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js"; import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { AnalysisCleanupLease } from "../../artifacts/AnalysisCleanupLease.js";
import { import {
desktopApplicationCaptureSchema, desktopApplicationCaptureSchema,
desktopCaptureEventSchema, desktopCaptureEventSchema,
} from "../../domain/desktop/desktopCapture.js"; } from "../../domain/desktop/desktopCapture.js";
import type { SessionRequest } from "../../desktop/DesktopApplicationSession.js"; import type { SessionRequest } from "../../desktop/DesktopApplicationSession.js";
import * as desktopSession from "../../desktop/DesktopApplicationSession.js";
import * as linuxInventory from "../../desktop/LinuxApplicationInventory.js";
import { X11WindowObserver } from "../../desktop/X11WindowObserver.js";
import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js"; import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
import { CaptureFile } from "./DesktopCaptureOutput.js"; import { CaptureFile } from "./DesktopCaptureOutput.js";
import { import {
captureDesktopApplication, captureDesktopApplication,
type DesktopCaptureRequest, type DesktopCaptureRequest,
} from "./DesktopApplicationCapture.js"; } from "./DesktopApplicationCapture.js";
import { capture as captureFixture } from "./desktopCapture.fixture.js";
const sha256 = (text: string) => const sha256 = (text: string) =>
createHash("sha256").update(text).digest("hex"); createHash("sha256").update(text).digest("hex");
@@ -175,15 +170,6 @@ describe("captureDesktopApplication refusals", () => {
await captureDesktopApplication(request(output), host(environment)), await captureDesktopApplication(request(output), host(environment)),
), ),
).toContain("is not empty"); ).toContain("is not empty");
const file = join(root, "existing-file");
await writeFile(file, "keep");
expect(
await captureDesktopApplication(request(file), host(environment)),
).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["output"], reason: "invalid_value" }] },
});
expect(await readFile(file, "utf8")).toBe("keep");
}); });
it("refuses a window manager that does not publish its client list", async () => { it("refuses a window manager that does not publish its client list", async () => {
@@ -240,56 +226,6 @@ describe("captureDesktopApplication refusals", () => {
}); });
}); });
it.each([false, true])(
"preserves unresolved session ownership when publication fails (unreleased: %s)",
async (unreleased) => {
const { root, home, environment } = await fixture();
const output = join(root, "out");
const captured = captureFixture();
const lease = new AnalysisCleanupLease(async () => {}, ["run-pending"]);
vi.spyOn(linuxInventory, "inspectLinuxApplication").mockResolvedValue({
ok: true,
value: {
inventory: { ...captured.application, state_directories: [] },
limitations: [],
signingKeys: new Map(),
},
});
vi.spyOn(X11WindowObserver.prototype, "sample").mockResolvedValue({
state: "listed",
windows: [],
});
vi.spyOn(desktopSession, "runDesktopSession").mockResolvedValue({
session: {
...captured.session,
cleanup: { verified: !unreleased, reason: unreleased ? "busy" : null },
},
unreleased: unreleased ? { reason: "busy", leases: [lease] } : undefined,
unreadableWindowSamples: 0,
});
vi.spyOn(CaptureFile.prototype, "publish").mockRejectedValue(
new Error("publication denied"),
);
try {
const result = await captureDesktopApplication(request(output), {
...host(environment),
callerDirectory: home,
});
expect(result).toMatchObject({
code: unreleased ? "cleanup_incomplete" : "execution_failure",
message: expect.stringContaining("publication denied"),
...(unreleased
? { details: { cleanup: "incomplete", resources: ["run-pending"] } }
: {}),
});
expect(await readdir(output)).toEqual([]);
} finally {
vi.restoreAllMocks();
await lease.release();
}
},
);
it.skipIf(process.platform !== "linux")( it.skipIf(process.platform !== "linux")(
"returns the published evidence when cancellation arrives during publication", "returns the published evidence when cancellation arrives during publication",
async () => { async () => {
@@ -19,13 +19,11 @@ import {
import { createEvidence, type Evidence } from "../../domain/evidence.js"; import { createEvidence, type Evidence } from "../../domain/evidence.js";
import { DesktopContentStore } from "../../desktop/DesktopContentStore.js"; import { DesktopContentStore } from "../../desktop/DesktopContentStore.js";
import { import {
DesktopSessionCleanupError,
runDesktopSession, runDesktopSession,
runningApplicationWindows, runningApplicationWindows,
type SessionRequest, type SessionRequest,
} from "../../desktop/DesktopApplicationSession.js"; } from "../../desktop/DesktopApplicationSession.js";
import { inspectLinuxApplication } from "../../desktop/LinuxApplicationInventory.js"; import { inspectLinuxApplication } from "../../desktop/LinuxApplicationInventory.js";
import { canonicalCallerPath } from "../../filesystem/canonicalCallerPath.js";
import { X11WindowObserver } from "../../desktop/X11WindowObserver.js"; import { X11WindowObserver } from "../../desktop/X11WindowObserver.js";
import { import {
discardFailedCapture, discardFailedCapture,
@@ -33,7 +31,11 @@ import {
projectFailure, projectFailure,
publishedCaptureCleanup, publishedCaptureCleanup,
} from "./DesktopCaptureFailure.js"; } from "./DesktopCaptureFailure.js";
import { CaptureFile, prepareOutputDirectory } from "./DesktopCaptureOutput.js"; import {
CaptureFile,
canonicalPath,
prepareOutputDirectory,
} from "./DesktopCaptureOutput.js";
import { import {
DESKTOP_OPERATION, DESKTOP_OPERATION,
DESKTOP_PROVIDER, DESKTOP_PROVIDER,
@@ -230,7 +232,7 @@ const admit = async (
throw outputRefusal(request.output, prepared.refusal); throw outputRefusal(request.output, prepared.refusal);
return { return {
display, display,
home: await canonicalCallerPath(home), home: await canonicalPath(home),
output: prepared.path, output: prepared.path,
inventory, inventory,
limitations, limitations,
@@ -242,7 +244,7 @@ const admit = async (
type Admitted = Awaited<ReturnType<typeof admit>>; type Admitted = Awaited<ReturnType<typeof admit>>;
/** Record the baseline, run the session and assemble its Evidence. */ /** Record the baseline, run the session and publish `capture.json`. */
const recordCapture = async ( const recordCapture = async (
request: DesktopCaptureRequest, request: DesktopCaptureRequest,
host: DesktopCaptureHost, host: DesktopCaptureHost,
@@ -250,6 +252,7 @@ const recordCapture = async (
readonly admitted: Admitted; readonly admitted: Admitted;
readonly scope: CaptureScope; readonly scope: CaptureScope;
readonly store: DesktopContentStore; readonly store: DesktopContentStore;
readonly captureFile: CaptureFile;
}, },
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
) => { ) => {
@@ -338,7 +341,8 @@ const recordCapture = async (
], ],
}; };
const evidence = captureEvidence(request, capture); const evidence = captureEvidence(request, capture);
return { evidence, session, unreleased }; const published = await context.captureFile.publish(evidence);
return { evidence, published, session, unreleased };
}; };
const runCapture = async ( const runCapture = async (
@@ -356,26 +360,22 @@ const runCapture = async (
); );
await store.prepare(); await store.prepare();
const captureFile = new CaptureFile(output); const captureFile = new CaptureFile(output);
let recorded: Awaited<ReturnType<typeof recordCapture>> | undefined; let recorded: Awaited<ReturnType<typeof recordCapture>>;
let published: string;
try { try {
recorded = await recordCapture( recorded = await recordCapture(
request, request,
host, host,
{ admitted, scope, store }, { admitted, scope, store, captureFile },
signal, signal,
); );
published = await captureFile.publish(recorded.evidence);
} catch (cause: unknown) { } catch (cause: unknown) {
// Nothing was published, so a failure or cancellation removes what was written. // Nothing was published, so a failure or cancellation removes what was written.
throw failedCaptureError( throw failedCaptureError(
recorded?.unreleased === undefined cause,
? cause
: new DesktopSessionCleanupError(cause, recorded.unreleased),
await discardFailedCapture(store, captureFile), await discardFailedCapture(store, captureFile),
); );
} }
const { evidence, session, unreleased } = recorded; const { evidence, published, session, unreleased } = recorded;
const cleanup = await publishedCaptureCleanup(published, store, unreleased); const cleanup = await publishedCaptureCleanup(published, store, unreleased);
if (cleanup !== undefined) throw cleanup; if (cleanup !== undefined) throw cleanup;
// A cancellation that arrives during publication must not read as success. // A cancellation that arrives during publication must not read as success.
@@ -9,11 +9,6 @@ import { z } from "zod";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js"; import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { SafeOutputTree } from "../../artifacts/SafeOutputTree.js"; import { SafeOutputTree } from "../../artifacts/SafeOutputTree.js";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import type { AnalysisCleanupLease } from "../../artifacts/AnalysisCleanupLease.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
import { RetainedContent } from "./DesktopRetainedContent.js";
import { import {
VARICAD_FILE, VARICAD_FILE,
VARICAD_URL, VARICAD_URL,
@@ -2222,114 +2217,6 @@ describe("exportDesktopApplication launch document beside its lock", () => {
}); });
}); });
describe("exportDesktopApplication retained input finalization", () => {
it("honors cancellation during retained input finalization before publication", async () => {
const controller = new AbortController();
const reason = new Error("cancel during seed finalization");
let writing = false;
let finalized = false;
const write = unspied("write");
vi.spyOn(SafeOutputTree.prototype, "write").mockImplementation(
async function (this: SafeOutputTree, ...args) {
const written = await write.apply(this, args);
writing = true;
return written;
},
);
const release = RetainedContent.prototype.release;
vi.spyOn(RetainedContent.prototype, "release").mockImplementation(
async function (this: RetainedContent, resources) {
const result = await release.call(this, resources);
if (writing) {
finalized = true;
controller.abort(reason);
}
return result;
},
);
const { result, directory } = await exportCapture(
capture(),
controller.signal,
);
expect(finalized).toBe(true);
expect(result).toMatchObject({ _tag: "AnalysisCancelledError" });
expect((result as Error).cause).toBe(reason);
expect(await exists(directory)).toBe(false);
});
it.each([false, true])(
"finalizes retained seed inputs before publication and retains cleanup retry (rollback fails: %s)",
async (rollbackFails) => {
let writing = false;
let reading = false;
const blocked = new Set<OwnedFileHandle>();
const leases: AnalysisCleanupLease[] = [];
if (rollbackFails)
vi.spyOn(
SafeOutputTree.prototype,
"closeHandles",
).mockRejectedValueOnce(new Error("output close failed"));
const write = unspied("write");
vi.spyOn(SafeOutputTree.prototype, "write").mockImplementation(
async function (this: SafeOutputTree, ...args) {
const written = await write.apply(this, args);
writing = true;
return written;
},
);
const read = RetainedContent.prototype.read;
vi.spyOn(RetainedContent.prototype, "read").mockImplementation(
async function (this: RetainedContent, ...args) {
reading = writing;
try {
return await read.apply(this, args);
} finally {
reading = false;
}
},
);
const close = OwnedFileHandle.prototype.close;
const closing = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockImplementation(function (this: OwnedFileHandle) {
if (reading && blocked.size === 0) blocked.add(this);
return blocked.has(this)
? Promise.reject(new Error("retained seed close failed"))
: close.call(this);
});
const release = ArtifactResourceScope.prototype.release;
vi.spyOn(ArtifactResourceScope.prototype, "release").mockImplementation(
async function (this: ArtifactResourceScope, owner) {
const result = await release.call(this, owner);
if (result.kind === "failed") leases.push(result.lease);
return result;
},
);
try {
const { result, directory } = await exportCapture(capture());
expect(blocked.size).toBe(1);
expect(result).toBeInstanceOf(AnalysisError);
if (!(result instanceof AnalysisError))
throw new Error("expected cleanup failure");
expect(result.cleanupIncomplete).toBe(true);
expect(result.cleanupResources).toHaveLength(rollbackFails ? 2 : 1);
if (rollbackFails) expect(result.cleanupResources).toContain(directory);
expect(await exists(directory)).toBe(rollbackFails);
expect(leases).toHaveLength(rollbackFails ? 2 : 1);
closing.mockRestore();
await leases[0]!.release();
expect(result.cleanupIncomplete).toBe(rollbackFails);
for (const lease of leases.slice(1)) await lease.release();
expect(result.cleanupIncomplete).toBe(false);
expect(await exists(directory)).toBe(false);
} finally {
closing.mockRestore();
for (const lease of leases) await lease.release();
}
},
);
});
describe("exportDesktopApplication bundle writes", () => { describe("exportDesktopApplication bundle writes", () => {
it.each(["cancelled", "out of space"])( it.each(["cancelled", "out of space"])(
"removes the unpublished bundle when its write fails (%s)", "removes the unpublished bundle when its write fails (%s)",
@@ -403,19 +403,19 @@ const runExport = async (
const text = bundleText(plan, admitted, dconf.holdsIdentity, held.consent); const text = bundleText(plan, admitted, dconf.holdsIdentity, held.consent);
const { generated, readme } = text; const { generated, readme } = text;
let scrub = NO_SCRUB; let scrub = NO_SCRUB;
const unclosed = await writeOutputTree( const unclosed = await withRetainedContent(
copy.bundle, request.capture,
(tree) => (content) =>
withRetainedContent( writeOutputTree(
request.capture, request.directory,
async (content) => { async (tree) => {
const bundle = { generated, copies, identity, readme }; const bundle = { generated, copies, identity, readme };
scrub = await writeFiles(tree, content, bundle, signal); scrub = await writeFiles(tree, content, bundle, signal);
}, },
SEED_MAX_BYTES, { operation: OPERATION, field: "directory" },
signal,
), ),
{ operation: OPERATION, field: "directory" }, SEED_MAX_BYTES,
signal,
); );
const limitations = exportLimitations(capture, plan, { const limitations = exportLimitations(capture, plan, {
install: install.limitations, install: install.limitations,
@@ -1,17 +1,10 @@
import { readFile, writeFile } from "node:fs/promises"; import { readFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { DatabaseSync } from "node:sqlite"; import { DatabaseSync } from "node:sqlite";
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it } from "vitest";
import { AnalysisCancelledError } from "../../domain/analysisErrorCore.js"; import { AnalysisCancelledError } from "../../domain/analysisErrorCore.js";
import {
AnalysisError,
primaryAnalysisError,
} from "../../domain/analysisErrorBase.js";
import { EvidenceFileError } from "../../domain/evidenceErrors.js";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
import type { import type {
DesktopApplicationCapture, DesktopApplicationCapture,
DesktopCheckpoint, DesktopCheckpoint,
@@ -19,7 +12,6 @@ import type {
import { desktopApplicationStateSchema } from "../../domain/desktop/desktopState.js"; import { desktopApplicationStateSchema } from "../../domain/desktop/desktopState.js";
import { import {
inspectDesktopApplicationState, inspectDesktopApplicationState,
readDesktopCapture,
type DesktopStateReaders, type DesktopStateReaders,
} from "./DesktopApplicationState.js"; } from "./DesktopApplicationState.js";
import { draftDesktopApplicationTask } from "./DesktopTaskDraft.js"; import { draftDesktopApplicationTask } from "./DesktopTaskDraft.js";
@@ -35,49 +27,6 @@ import {
writeCaptureFolder, writeCaptureFolder,
} from "./desktopCapture.fixture.js"; } from "./desktopCapture.fixture.js";
it("retains invalid capture JSON and the descriptor owner through failed final cleanup", async () => {
const root = await createTestTempDirectory("rea-capture-read-cleanup-");
await writeFile(join(root, "capture.json"), '{"broken":');
await expect(readDesktopCapture(root, undefined)).rejects.toMatchObject({
_tag: "EvidenceFileError",
reason: "invalid-json",
});
const scopes = new Set<ArtifactResourceScope>();
const originalClose = ArtifactResourceScope.prototype.close;
const scopeClose = vi
.spyOn(ArtifactResourceScope.prototype, "close")
.mockImplementation(function (this: ArtifactResourceScope) {
scopes.add(this);
return originalClose.call(this);
});
const fileClose = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockRejectedValue(new Error("descriptor close remains unavailable"));
let failure: AnalysisError | undefined;
try {
const reported = await readDesktopCapture(root, undefined).catch(
(cause: unknown) => cause,
);
expect(reported).toBeInstanceOf(AnalysisError);
if (!(reported instanceof AnalysisError)) throw reported;
failure = reported;
expect(primaryAnalysisError(failure)).toBeInstanceOf(EvidenceFileError);
expect(primaryAnalysisError(failure)).toMatchObject({
reason: "invalid-json",
});
expect(failure.cleanupIncomplete).toBe(true);
expect(failure.cleanupResources).toEqual([join(root, "capture.json")]);
} finally {
fileClose.mockRestore();
scopeClose.mockRestore();
for (const scope of scopes) await scope.close();
}
expect(failure?.cleanupIncomplete).toBe(false);
expect(primaryAnalysisError(failure!)).toMatchObject({
reason: "invalid-json",
});
});
/** The fixture capture with extra files at the baseline and the second checkpoint. */ /** The fixture capture with extra files at the baseline and the second checkpoint. */
const extended = (extra: { const extended = (extra: {
readonly baseline?: DesktopCheckpoint["retained"]; readonly baseline?: DesktopCheckpoint["retained"];
@@ -10,7 +10,6 @@ import { join } from "node:path";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js"; import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js"; import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js"; import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
import { import {
desktopApplicationCaptureSchema, desktopApplicationCaptureSchema,
@@ -93,69 +92,44 @@ export const readDesktopCapture = async (
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
): Promise<{ evidence: Evidence; capture: DesktopApplicationCapture }> => { ): Promise<{ evidence: Evidence; capture: DesktopApplicationCapture }> => {
const resources = new ArtifactResourceScope(); const resources = new ArtifactResourceScope();
const outcome = await resources
.run(async () => {
const loaded = await readJsonFile(
join(folder, "capture.json"),
signal,
resources,
OPERATION,
);
if (!loaded.ok) {
if (loaded.error instanceof AnalysisError) throw loaded.error;
throw inputError(
`${join(folder, "capture.json")} could not be read as JSON.`,
"Pass a folder written by capture-desktop-application.",
);
}
const evidence = safeParseEvidence(loaded.value);
if (
!evidence.ok ||
evidence.value.operation !== DESKTOP_OPERATION ||
evidence.value.predicate_type !== "rea.desktop-application-capture" ||
evidence.value.provider.id !== DESKTOP_PROVIDER.id ||
evidence.value.provider.version !== DESKTOP_PROVIDER.version
)
throw inputError(
"capture.json is not desktop application capture Evidence.",
"Pass a folder written by capture-desktop-application.",
);
const capture = desktopApplicationCaptureSchema.safeParse(
evidence.value.normalized_result,
);
if (!capture.success)
throw inputError(
`The capture result is malformed (${describeValidationFailure(capture.error)}).`,
"Capture the application again.",
);
return { evidence: evidence.value, capture: capture.data };
})
.then(ok, (cause: unknown) => err(cause));
try { try {
const loaded = await readJsonFile(
join(folder, "capture.json"),
signal,
resources,
OPERATION,
);
if (!loaded.ok) {
if (loaded.error instanceof AnalysisError) throw loaded.error;
throw inputError(
`${join(folder, "capture.json")} could not be read as JSON.`,
"Pass a folder written by capture-desktop-application.",
);
}
const evidence = safeParseEvidence(loaded.value);
if (
!evidence.ok ||
evidence.value.operation !== DESKTOP_OPERATION ||
evidence.value.predicate_type !== "rea.desktop-application-capture" ||
evidence.value.provider.id !== DESKTOP_PROVIDER.id ||
evidence.value.provider.version !== DESKTOP_PROVIDER.version
)
throw inputError(
"capture.json is not desktop application capture Evidence.",
"Pass a folder written by capture-desktop-application.",
);
const capture = desktopApplicationCaptureSchema.safeParse(
evidence.value.normalized_result,
);
if (!capture.success)
throw inputError(
`The capture result is malformed (${describeValidationFailure(capture.error)}).`,
"Capture the application again.",
);
return { evidence: evidence.value, capture: capture.data };
} finally {
await resources.close(); await resources.close();
} catch (cause: unknown) {
const cleanup = desktopOperationFailure(
cause,
signal,
OPERATION,
"Desktop capture input",
);
if (outcome.ok)
throw cleanup.retainPartialObservation(outcome.value.evidence);
const primary = desktopOperationFailure(
outcome.error,
signal,
OPERATION,
"Desktop capture input",
);
throw analysisErrorWithCleanupFailure(
primary,
cleanup,
OPERATION,
).bindCleanupDependencies([primary, cleanup]);
} }
if (!outcome.ok) throw outcome.error;
return outcome.value;
}; };
interface ViewContext { interface ViewContext {
@@ -46,7 +46,6 @@ import {
WORK, WORK,
} from "./DesktopBuildCommands.js"; } from "./DesktopBuildCommands.js";
import { import {
planOutputTree,
writeOutputTree, writeOutputTree,
writtenWithOpenHandles, writtenWithOpenHandles,
} from "./DesktopOutputTree.js"; } from "./DesktopOutputTree.js";
@@ -184,7 +183,7 @@ const writeBundle = async (
unchecked: boolean; unchecked: boolean;
}[] = []; }[] = [];
const unclosed = await writeOutputTree( const unclosed = await writeOutputTree(
await planOutputTree(directory, { operation: EXPORT, field: "directory" }), directory,
async (tree) => { async (tree) => {
fetched = await fetchHomeFolder( fetched = await fetchHomeFolder(
computer, computer,
@@ -28,7 +28,6 @@ import {
import { errorText } from "./DesktopBuildCommands.js"; import { errorText } from "./DesktopBuildCommands.js";
import { DESKTOP_PROVIDER } from "./DesktopCaptureRefusals.js"; import { DESKTOP_PROVIDER } from "./DesktopCaptureRefusals.js";
import { import {
planOutputTree,
writeOutputTree, writeOutputTree,
writtenWithOpenHandles, writtenWithOpenHandles,
} from "./DesktopOutputTree.js"; } from "./DesktopOutputTree.js";
@@ -154,7 +153,7 @@ const writeRecordFolder = async (
> => { > => {
try { try {
const unclosed = await writeOutputTree( const unclosed = await writeOutputTree(
await planOutputTree(directory, { operation: CHECK, field: "directory" }), directory,
async (tree) => { async (tree) => {
for (const file of files) for (const file of files)
await tree.write( await tree.write(
@@ -13,7 +13,6 @@ import { DesktopSessionCleanupError } from "../../desktop/DesktopApplicationSess
import { DesktopContentStore } from "../../desktop/DesktopContentStore.js"; import { DesktopContentStore } from "../../desktop/DesktopContentStore.js";
import { walkHomeState } from "../../desktop/HomeStateWalker.js"; import { walkHomeState } from "../../desktop/HomeStateWalker.js";
import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js"; import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
import { captureDesktopApplication } from "./DesktopApplicationCapture.js";
import { CaptureFile } from "./DesktopCaptureOutput.js"; import { CaptureFile } from "./DesktopCaptureOutput.js";
import { import {
discardFailedCapture, discardFailedCapture,
@@ -237,77 +236,6 @@ describe("failedCaptureError", () => {
}); });
describe("projectFailure", () => { describe("projectFailure", () => {
it.each([
{ oversized: false, category: "cancelled" },
{ oversized: true, category: "execution_failure" },
])(
"preserves the physical read outcome beside final cancellation and cleanup (oversized: $oversized)",
async ({ oversized, category }) => {
const root = await createTestTempDirectory("desktop-inventory-cancel-");
const entry = join(root, "notes.desktop");
await writeFile(
entry,
oversized
? "x".repeat(1024 * 1024 + 1)
: "[Desktop Entry]\nType=Application\nName=Notes\nExec=notes\n",
);
const controller = new AbortController();
const reason = new Error("cancelled while closing the desktop entry");
let owner: OwnedFileHandle | undefined;
const close = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockImplementation(function (this: OwnedFileHandle) {
// oxlint-disable-next-line typescript/no-this-alias -- Release the exact denied descriptor after the public call.
owner = this;
controller.abort(reason);
return Promise.reject(new Error("close denied"));
});
try {
const result = await captureDesktopApplication(
{
desktopEntry: entry,
executable: undefined,
arguments: [],
output: join(root, "capture"),
statePaths: [],
excludes: [],
maxDurationSeconds: 1,
maxEntries: 10,
maxFileBytes: 1024,
maxContentBytes: 1024,
packageFile: undefined,
},
{
platform: "linux",
callerDirectory: root,
environment: {
HOME: root,
DISPLAY: ":controlled",
REA_XPROP_COMMAND: process.execPath,
},
nextRequest: () => undefined,
report: async () => {
throw new Error(
"inventory failure must precede target execution",
);
},
},
controller.signal,
);
expect(owner).toBeDefined();
expect(result).toMatchObject({
error: "Desktop capture failed",
code: "cleanup_incomplete",
category,
details: { resources: [entry] },
});
} finally {
close.mockRestore();
await owner?.close();
}
},
);
it("keeps the owner of a resource the inventory could not release", async () => { it("keeps the owner of a resource the inventory could not release", async () => {
const resources = new ArtifactResourceScope(); const resources = new ArtifactResourceScope();
let attempts = 0; let attempts = 0;
@@ -14,23 +14,10 @@ import { afterEach, describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js"; import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import type { Evidence } from "../../domain/evidence.js"; import type { Evidence } from "../../domain/evidence.js";
import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js"; import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
import { CaptureFile, prepareOutputDirectory } from "./DesktopCaptureOutput.js"; import { CaptureFile } from "./DesktopCaptureOutput.js";
const evidence = { id: "evidence" } as unknown as Evidence; const evidence = { id: "evidence" } as unknown as Evidence;
it.each(["", "/child"])(
"refuses an output path blocked by a regular file (%s)",
async (suffix) => {
const root = await createTestTempDirectory("desktop-output-file-");
const file = join(root, "existing");
await writeFile(file, "keep");
await expect(prepareOutputDirectory(`${file}${suffix}`)).resolves.toEqual({
refusal: "not_a_directory",
});
expect(await readFile(file, "utf8")).toBe("keep");
},
);
describe("CaptureFile", () => { describe("CaptureFile", () => {
it("publishes through its partial file and leaves nothing to discard", async () => { it("publishes through its partial file and leaves nothing to discard", async () => {
const output = await createTestTempDirectory("desktop-output-"); const output = await createTestTempDirectory("desktop-output-");
@@ -8,12 +8,31 @@ import {
rename, rename,
rm, rm,
} from "node:fs/promises"; } from "node:fs/promises";
import { join } from "node:path"; import { basename, dirname, join, resolve } from "node:path";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js"; import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import type { Evidence } from "../../domain/evidence.js"; import type { Evidence } from "../../domain/evidence.js";
import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js"; import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
/**
* The physical spelling of a path that may not exist yet: the real path of
* its longest existing ancestor followed by the rest. Walks, exclusions and
* state directories compare in this spelling, so a symbolic link in the
* caller's spelling cannot make one directory look like two.
*/
export const canonicalPath = async (path: string): Promise<string> => {
const absolute = resolve(path);
const rest: string[] = [];
for (let current = absolute; ; current = dirname(current)) {
try {
return join(await realpath(current), ...rest.reverse());
} catch {
if (dirname(current) === current) return absolute;
rest.push(basename(current));
}
}
};
/** Why the output folder cannot hold a private capture. */ /** Why the output folder cannot hold a private capture. */
export type OutputRefusal = "not_a_directory" | "foreign_owner" | "not_empty"; export type OutputRefusal = "not_a_directory" | "foreign_owner" | "not_empty";
@@ -24,14 +43,7 @@ export type OutputRefusal = "not_a_directory" | "foreign_owner" | "not_empty";
export const prepareOutputDirectory = async ( export const prepareOutputDirectory = async (
output: string, output: string,
): Promise<{ readonly path: string } | { readonly refusal: OutputRefusal }> => { ): Promise<{ readonly path: string } | { readonly refusal: OutputRefusal }> => {
try { await mkdir(output, { recursive: true, mode: 0o700 });
await mkdir(output, { recursive: true, mode: 0o700 });
} catch (cause: unknown) {
const code = (cause as NodeJS.ErrnoException).code;
if (code === "EEXIST" || code === "ENOTDIR")
return { refusal: "not_a_directory" };
throw cause;
}
const stats = await lstat(output); const stats = await lstat(output);
if (!stats.isDirectory()) return { refusal: "not_a_directory" }; if (!stats.isDirectory()) return { refusal: "not_a_directory" };
const uid = process.getuid?.(); const uid = process.getuid?.();
@@ -2,10 +2,7 @@ import { readdir } from "node:fs/promises";
import { isAbsolute } from "node:path"; import { isAbsolute } from "node:path";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { import { bindArtifactFileCleanup } from "../../artifacts/ArtifactFileOperation.js";
artifactFileCancellationCause,
bindArtifactFileCleanup,
} from "../../artifacts/ArtifactFileOperation.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js"; import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { import {
AnalysisCancelledError, AnalysisCancelledError,
@@ -196,8 +193,6 @@ export const checkDesktopCaptureHost = async (
); );
const existing = await readdir(request.output).catch((cause: unknown) => { const existing = await readdir(request.output).catch((cause: unknown) => {
if ((cause as NodeJS.ErrnoException).code === "ENOENT") return []; if ((cause as NodeJS.ErrnoException).code === "ENOENT") return [];
if ((cause as NodeJS.ErrnoException).code === "ENOTDIR")
throw outputRefusal(request.output, "not_a_directory");
throw cause; throw cause;
}); });
if (existing.length > 0) throw outputRefusal(request.output, "not_empty"); if (existing.length > 0) throw outputRefusal(request.output, "not_empty");
@@ -237,15 +232,8 @@ export const desktopOperationFailure = (
} }
// A resource that could not be released stays reported, with its owner. // A resource that could not be released stays reported, with its owner.
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) { if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) {
const primary =
cause.reason === "cancelled"
? new AnalysisCancelledError(operation, {
cause: artifactFileCancellationCause(cause),
})
: undefined;
const error = new ProviderAdapterError(DESKTOP_PROVIDER.id, operation, { const error = new ProviderAdapterError(DESKTOP_PROVIDER.id, operation, {
cause, cause,
...(primary === undefined ? {} : { cleanupPrimary: primary }),
cleanup: cause.cleanup, cleanup: cause.cleanup,
diagnostics: { phase: operation }, diagnostics: { phase: operation },
userMessage: `${label} could not release a resource it used: ${describeValidationFailure(cause)}`, userMessage: `${label} could not release a resource it used: ${describeValidationFailure(cause)}`,
@@ -141,36 +141,6 @@ describe("captureScope", () => {
); );
expect(scope.contentRoots).not.toContain(linked); expect(scope.contentRoots).not.toContain(linked);
}); });
it("follows a selected symlink before its parent segment for capture and exclusion", async () => {
const root = await realpath(
await createTestTempDirectory("desktop-scope-parent-"),
);
const home = join(root, "home");
await mkdir(home);
await mkdir(join(root, "physical", "child"), { recursive: true });
await mkdir(join(root, "named"));
await symlink(join(root, "physical", "child"), join(root, "named", "link"));
await writeFile(join(root, "physical", "selected"), "selected");
await writeFile(join(root, "named", "selected"), "decoy");
const selected = `${root}/named/link/../selected`;
const { scope, selectedPaths } = await captureScope(
{ ...request([]), statePaths: [selected], excludes: [selected] },
inventory(),
home,
join(root, "out"),
);
expect(selectedPaths).toEqual([
{
path: selected,
physical: join(root, "physical", "selected"),
kind: "file",
},
]);
expect(scope.contentRoots).toContain(join(root, "physical", "selected"));
expect(scope.excluded).toContain(join(root, "physical", "selected"));
expect(scope.excluded).not.toContain(join(root, "named", "selected"));
});
}); });
describe("applicationWindowMatching", () => { describe("applicationWindowMatching", () => {
+6 -10
View File
@@ -1,12 +1,11 @@
import { lstat } from "node:fs/promises"; import { lstat } from "node:fs/promises";
import { basename, isAbsolute } from "node:path"; import { basename, isAbsolute, resolve } from "node:path";
import type { DesktopApplicationInventory } from "../../domain/desktop/desktopCapture.js"; import type { DesktopApplicationInventory } from "../../domain/desktop/desktopCapture.js";
import { isWithinAny } from "../../domain/desktop/desktopStateDiff.js"; import { isWithinAny } from "../../domain/desktop/desktopStateDiff.js";
import { SQLITE_SIDECARS } from "../../domain/desktop/stateFormats.js"; import { SQLITE_SIDECARS } from "../../domain/desktop/stateFormats.js";
import type { WindowMatching } from "../../desktop/DesktopSessionObservations.js"; import type { WindowMatching } from "../../desktop/DesktopSessionObservations.js";
import { absoluteCallerPath } from "../../filesystem/callerPath.js"; import { canonicalPath } from "./DesktopCaptureOutput.js";
import { canonicalCallerPath } from "../../filesystem/canonicalCallerPath.js";
import type { CheckpointScope } from "./DesktopCheckpointRecorder.js"; import type { CheckpointScope } from "./DesktopCheckpointRecorder.js";
const unique = (paths: readonly string[]) => [...new Set(paths)]; const unique = (paths: readonly string[]) => [...new Set(paths)];
@@ -15,10 +14,7 @@ const unique = (paths: readonly string[]) => [...new Set(paths)];
const pathKind = async (path: string): Promise<"file" | "directory"> => const pathKind = async (path: string): Promise<"file" | "directory"> =>
(await lstat(path).then( (await lstat(path).then(
(stats) => stats.isDirectory(), (stats) => stats.isDirectory(),
(cause: unknown) => { () => false,
if ((cause as NodeJS.ErrnoException).code === "ENOENT") return false;
throw cause;
},
)) ))
? "directory" ? "directory"
: "file"; : "file";
@@ -59,12 +55,12 @@ export const captureScope = async (
output: string, output: string,
): Promise<CaptureScope> => { ): Promise<CaptureScope> => {
const canonical = (paths: readonly string[]) => const canonical = (paths: readonly string[]) =>
Promise.all(paths.map((path) => canonicalCallerPath(path))); Promise.all(paths.map((path) => canonicalPath(path)));
const named = (paths: readonly string[]) => const named = (paths: readonly string[]) =>
Promise.all( Promise.all(
paths.map(async (path) => ({ paths.map(async (path) => ({
path: absoluteCallerPath(path), path: resolve(path),
physical: await canonicalCallerPath(path), physical: await canonicalPath(path),
})), })),
); );
const stateDirectories = await canonical( const stateDirectories = await canonical(
@@ -164,7 +164,7 @@ const evidence = (result: Evidence | Error): Evidence => {
return result; return result;
}; };
it("refuses an occupied withheld copy destination before writing the bundle", async () => { it("reports the published bundle when the withheld copy destination is occupied", async () => {
const root = await createTestTempDirectory("desktop-copy-refusal-"); const root = await createTestTempDirectory("desktop-copy-refusal-");
const occupied = join(root, "occupied"); const occupied = join(root, "occupied");
await mkdir(occupied); await mkdir(occupied);
@@ -173,31 +173,21 @@ it("refuses an occupied withheld copy destination before writing the bundle", as
withhold: { paths: [SETTINGS] }, withhold: { paths: [SETTINGS] },
withheldCopy: occupied, withheldCopy: occupied,
}); });
expect(result).toBeInstanceOf(AnalysisError); expect(result).toMatchObject({
expect(await exists(directory)).toBe(false); partialObservation: {
expect(await readFile(join(occupied, "keep"), "utf8")).toBe("existing"); operation: "export_desktop_application",
const occupiedFile = join(root, "file"); normalized_result: { directory },
await writeFile(occupiedFile, "existing file"); },
const file = await exportWithheld({
withhold: { paths: [SETTINGS] },
withheldCopy: occupiedFile,
}); });
expect(file.result).toBeInstanceOf(AnalysisError); expect(await exists(join(directory, "README.md"))).toBe(true);
expect(await exists(file.directory)).toBe(false); expect(await readFile(join(occupied, "keep"), "utf8")).toBe("existing");
expect(await readFile(occupiedFile, "utf8")).toBe("existing file");
}); });
it("retains the bundle cleanup owner when a later withheld copy fails", async () => { it("retains the bundle cleanup owner when a later withheld copy fails", async () => {
const root = await createTestTempDirectory("desktop-copy-close-"); const root = await createTestTempDirectory("desktop-copy-close-");
const occupied = join(root, "occupied"); const occupied = join(root, "occupied");
await mkdir(occupied); await mkdir(occupied);
const commit = SafeOutputTree.prototype.commit; await writeFile(join(occupied, "keep"), "existing");
vi.spyOn(SafeOutputTree.prototype, "commit").mockImplementationOnce(
async function (this: SafeOutputTree, signal) {
await commit.call(this, signal);
await writeFile(join(occupied, "keep"), "existing");
},
);
const original = SafeOutputTree.prototype.closeHandles; const original = SafeOutputTree.prototype.closeHandles;
const release = ArtifactResourceScope.prototype.release; const release = ArtifactResourceScope.prototype.release;
const failures: Extract<ArtifactCleanupAttempt, { kind: "failed" }>[] = []; const failures: Extract<ArtifactCleanupAttempt, { kind: "failed" }>[] = [];
@@ -572,9 +562,7 @@ describe("exportDesktopApplication of withheld paths and copies", () => {
).toContain("49152"); ).toContain("49152");
expect(await exists(directory)).toBe(false); expect(await exists(directory)).toBe(false);
}); });
});
describe("exportDesktopApplication withheld copy destinations", () => {
it("refuses a withheld copy inside the bundle, also through a link, and writes neither", async () => { it("refuses a withheld copy inside the bundle, also through a link, and writes neither", async () => {
const root = await createTestTempDirectory("desktop-withheld-inside-"); const root = await createTestTempDirectory("desktop-withheld-inside-");
const directory = join(root, "bundle"); const directory = join(root, "bundle");
@@ -594,38 +582,6 @@ describe("exportDesktopApplication withheld copy destinations", () => {
}); });
expect(linked.result).toBeInstanceOf(Error); expect(linked.result).toBeInstanceOf(Error);
expect(await readdir(directory)).toEqual([]); expect(await readdir(directory)).toEqual([]);
await mkdir(join(root, "outside"));
await symlink(directory, join(root, "outside", "link"));
const parentTraversal = await exportWithheld({
withhold: { paths: [SETTINGS] },
directory,
withheldCopy: `${root}/outside/link/../bundle/private`,
});
expect(parentTraversal.result).toBeInstanceOf(Error);
expect(await readdir(directory)).toEqual([]);
const unresolved = await exportWithheld({
withhold: { paths: [SETTINGS] },
directory: join(root, "new-bundle"),
withheldCopy: `${root}/missing/../private`,
});
expect(unresolved.result).toBeInstanceOf(Error);
expect(await exists(join(root, "missing"))).toBe(false);
expect(await exists(join(root, "new-bundle"))).toBe(false);
await symlink(join(root, "absent"), join(root, "dangling"));
const dangling = await exportWithheld({
withhold: { paths: [SETTINGS] },
directory: join(root, "new-bundle"),
withheldCopy: join(root, "dangling", "private"),
});
expect(dangling.result).toBeInstanceOf(AnalysisError);
expect(await exists(join(root, "new-bundle"))).toBe(false);
const leafLink = await exportWithheld({
withhold: { paths: [SETTINGS] },
directory: `${root}/alias//`,
withheldCopy: join(root, "outside-copy"),
});
expect(leafLink.result).toBeInstanceOf(Error);
expect(await readdir(directory)).toEqual([]);
}); });
it("reports a written copy whose handles cannot be closed, with the export's Evidence", async () => { it("reports a written copy whose handles cannot be closed, with the export's Evidence", async () => {
@@ -1,9 +1,9 @@
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises"; import { readFile, realpath } from "node:fs/promises";
import { basename, dirname, join, resolve, sep } from "node:path";
import { Readable } from "node:stream"; import { Readable } from "node:stream";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js"; import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { isPathWithinRoot } from "../../domain/localPath.js";
import { import {
consentFileSchema, consentFileSchema,
MAX_WITHHELD_BYTES, MAX_WITHHELD_BYTES,
@@ -24,12 +24,7 @@ import {
import { isWithinAny } from "../../domain/desktop/desktopStateDiff.js"; import { isWithinAny } from "../../domain/desktop/desktopStateDiff.js";
import { safeParseUtf8Json } from "../../domain/safeJson.js"; import { safeParseUtf8Json } from "../../domain/safeJson.js";
import type { SeedVersion } from "./DesktopExportFiles.js"; import type { SeedVersion } from "./DesktopExportFiles.js";
import { import { writeOutputTree, type FailedRelease } from "./DesktopOutputTree.js";
planOutputTree,
writeOutputTree,
type DesktopOutputPlan,
type FailedRelease,
} from "./DesktopOutputTree.js";
import { import {
withRetainedContent, withRetainedContent,
type RetainedContent, type RetainedContent,
@@ -267,7 +262,7 @@ const readWithheld = async (
/** Write the withheld files, by home path, into a new or empty folder outside the bundle; a failed release of a written copy is returned. */ /** Write the withheld files, by home path, into a new or empty folder outside the bundle; a failed release of a written copy is returned. */
const writeWithheldCopy = ( const writeWithheldCopy = (
directory: DesktopOutputPlan, directory: string,
files: ReadonlyMap<string, Buffer>, files: ReadonlyMap<string, Buffer>,
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
): Promise<FailedRelease | undefined> => ): Promise<FailedRelease | undefined> =>
@@ -289,18 +284,32 @@ const writeWithheldCopy = (
signal, signal,
); );
/** A path with every link in its existing part resolved, so two spellings of one folder compare equal. */
const resolvedPath = async (path: string): Promise<string> => {
const rest: string[] = [];
for (let current = resolve(path); ; current = dirname(current)) {
try {
return join(await realpath(current), ...rest);
} catch {
if (dirname(current) === current) return resolve(path);
rest.unshift(basename(current));
}
}
};
const isWithin = (path: string, root: string) =>
path === root || path.startsWith(root.endsWith(sep) ? root : `${root}${sep}`);
/** Refuse a copy folder that is the bundle's, inside it or holding it, also through a link, before either is written. */ /** Refuse a copy folder that is the bundle's, inside it or holding it, also through a link, before either is written. */
const refuseCopyInBundle = ( const refuseCopyInBundle = async (copy: string, bundle: string) => {
copy: DesktopOutputPlan, const [copied, bundled] = await Promise.all([
bundle: DesktopOutputPlan, resolvedPath(copy),
) => { resolvedPath(bundle),
if ( ]);
isPathWithinRoot(bundle.physical, copy.physical) || if (isWithin(copied, bundled) || isWithin(bundled, copied))
isPathWithinRoot(copy.physical, bundle.physical)
)
throw refused( throw refused(
"withheld_copy", "withheld_copy",
`The withheld copy ${copy.requested} lies in the bundle ${bundle.requested}, or holds it, so the withheld files would ship with it.`, `The withheld copy ${copy} lies in the bundle ${bundle}, or holds it, so the withheld files would ship with it.`,
"Name a copy folder outside the bundle's folder.", "Name a copy folder outside the bundle's folder.",
); );
}; };
@@ -316,35 +325,22 @@ export const withheldCopy = async (
captured: ReadonlyMap<string, SeedVersion>, captured: ReadonlyMap<string, SeedVersion>,
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
): Promise<{ ): Promise<{
readonly bundle: DesktopOutputPlan;
readonly write: () => Promise<FailedRelease | undefined>; readonly write: () => Promise<FailedRelease | undefined>;
}> => { }> => {
const directory = request.withheldCopy; const directory = request.withheldCopy;
const withheld = plan.withheld ?? []; const withheld = plan.withheld ?? [];
const bundle = await planOutputTree(request.directory, { if (directory !== undefined)
operation: OPERATION, await refuseCopyInBundle(directory, request.directory);
field: "directory", if (withheld.length === 0) return { write: () => Promise.resolve(undefined) };
});
const copy =
directory === undefined
? undefined
: await planOutputTree(directory, {
operation: OPERATION,
field: "withheld_copy",
});
if (copy !== undefined) refuseCopyInBundle(copy, bundle);
if (withheld.length === 0)
return { bundle, write: () => Promise.resolve(undefined) };
const files = await withRetainedContent( const files = await withRetainedContent(
request.capture, request.capture,
(content) => readWithheld(content, withheld, captured, signal), (content) => readWithheld(content, withheld, captured, signal),
SEED_MAX_BYTES, SEED_MAX_BYTES,
); );
return { return {
bundle,
write: () => write: () =>
copy === undefined directory === undefined
? Promise.resolve(undefined) ? Promise.resolve(undefined)
: writeWithheldCopy(copy, files, signal), : writeWithheldCopy(directory, files, signal),
}; };
}; };
@@ -1,30 +0,0 @@
import { readdir } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { planOutputTree, writeOutputTree } from "./DesktopOutputTree.js";
it("does not create output parents or invoke the writer after cancellation", async () => {
const root = await createTestTempDirectory("rea-output-cancel-");
const directory = join(root, "new-parent", "output");
const refusal = {
operation: "export_desktop_application",
field: "directory",
};
const plan = await planOutputTree(directory, refusal);
const controller = new AbortController();
const reason = new Error("cancelled before output acquisition");
controller.abort(reason);
let written = false;
const failure = await writeOutputTree(
plan,
async () => {
written = true;
},
refusal,
controller.signal,
).catch((cause: unknown) => cause);
expect(written).toBe(false);
expect(await readdir(root)).toEqual([]);
expect(failure).toBe(reason);
});
+7 -72
View File
@@ -1,18 +1,13 @@
import { lstat, mkdir, readdir } from "node:fs/promises"; import { mkdir } from "node:fs/promises";
import { basename, dirname, isAbsolute } from "node:path"; import { dirname } from "node:path";
import { import {
ArtifactResourceScope, ArtifactResourceScope,
type ArtifactCleanupAttempt, type ArtifactCleanupAttempt,
} from "../../artifacts/ArtifactResourceScope.js"; } from "../../artifacts/ArtifactResourceScope.js";
import { SafeOutputTree } from "../../artifacts/SafeOutputTree.js"; import { SafeOutputTree } from "../../artifacts/SafeOutputTree.js";
import { import { outputDirectoryAdmissionRefusal } from "../../artifacts/SafeOutputTreeAdmission.js";
outputDirectoryAdmissionRefusal,
SafeOutputTreeDestinationRefusal,
} from "../../artifacts/SafeOutputTreeAdmission.js";
import { SafeOutputTreeCreationFailure } from "../../artifacts/SafeOutputTreeCreationFailure.js"; import { SafeOutputTreeCreationFailure } from "../../artifacts/SafeOutputTreeCreationFailure.js";
import { canonicalCallerPath } from "../../filesystem/canonicalCallerPath.js";
import { absoluteCallerPath } from "../../filesystem/callerPath.js";
import type { Evidence } from "../../domain/evidence.js"; import type { Evidence } from "../../domain/evidence.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js"; import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
@@ -22,65 +17,6 @@ export type FailedRelease = Extract<
{ readonly kind: "failed" } { readonly kind: "failed" }
>; >;
/** A caller-selected destination and the filesystem path admitted for writing. */
export interface DesktopOutputPlan {
readonly requested: string;
readonly physical: string;
}
/** Establish destination identity before either output tree is created. */
export const planOutputTree = async (
requested: string,
refusal: { readonly operation: string; readonly field: string },
): Promise<DesktopOutputPlan> => {
try {
if (!isAbsolute(requested))
throw new SafeOutputTreeDestinationRefusal(
"Output root must be absolute",
);
if (basename(requested) === ".." || basename(requested) === ".")
throw new SafeOutputTreeDestinationRefusal("Invalid output directory");
const physical = await canonicalCallerPath(requested);
const absolute = absoluteCallerPath(requested);
const selected =
absolute.length > 1 ? absolute.replace(/\/$/u, "") : absolute;
const existing = await lstat(selected).then(
(value) => value,
(cause: unknown) => {
if ((cause as NodeJS.ErrnoException).code === "ENOENT")
return undefined;
throw cause;
},
);
if (existing?.isSymbolicLink())
throw new SafeOutputTreeDestinationRefusal(
"Output root cannot be a symbolic link",
);
if (existing !== undefined && !existing.isDirectory())
throw new SafeOutputTreeDestinationRefusal(
"Output root must be a directory",
);
if (existing !== undefined && (await readdir(selected)).length > 0)
throw new SafeOutputTreeDestinationRefusal(
"Output directory is not empty",
);
return Object.freeze({ requested, physical });
} catch (cause: unknown) {
const error =
cause instanceof SafeOutputTreeDestinationRefusal
? cause
: new SafeOutputTreeDestinationRefusal(
"Output destination cannot be resolved",
{ cause },
);
throw outputDirectoryAdmissionRefusal(
refusal.operation,
refusal.field,
error,
);
}
};
/** /**
* Write a new or empty folder through the safe output tree, rolling an * Write a new or empty folder through the safe output tree, rolling an
* unpublished tree back on any failure so no partial copy remains. A * unpublished tree back on any failure so no partial copy remains. A
@@ -88,13 +24,12 @@ export const planOutputTree = async (
* refused destination names the caller's `field` under `operation`. * refused destination names the caller's `field` under `operation`.
*/ */
export const writeOutputTree = async ( export const writeOutputTree = async (
plan: DesktopOutputPlan, directory: string,
write: (tree: SafeOutputTree) => Promise<void>, write: (tree: SafeOutputTree) => Promise<void>,
refusal: { readonly operation: string; readonly field: string }, refusal: { readonly operation: string; readonly field: string },
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
): Promise<FailedRelease | undefined> => { ): Promise<FailedRelease | undefined> => {
signal?.throwIfAborted(); await mkdir(dirname(directory), { recursive: true, mode: 0o700 });
await mkdir(dirname(plan.physical), { recursive: true, mode: 0o700 });
const resources = new ArtifactResourceScope(); const resources = new ArtifactResourceScope();
let tree: SafeOutputTree | undefined; let tree: SafeOutputTree | undefined;
let outcome: let outcome:
@@ -102,7 +37,7 @@ export const writeOutputTree = async (
| { readonly kind: "failed"; readonly cause: unknown }; | { readonly kind: "failed"; readonly cause: unknown };
try { try {
try { try {
tree = await SafeOutputTree.create(plan.physical, process.platform, { tree = await SafeOutputTree.create(directory, process.platform, {
acceptExistingEmpty: true, acceptExistingEmpty: true,
}); });
} catch (cause: unknown) { } catch (cause: unknown) {
@@ -118,7 +53,7 @@ export const writeOutputTree = async (
if (tree !== undefined) { if (tree !== undefined) {
const released = await resources.release({ const released = await resources.release({
kind: "output-tree", kind: "output-tree",
resource: plan.requested, resource: directory,
tree, tree,
}); });
if (released.kind === "failed") { if (released.kind === "failed") {
@@ -7,8 +7,6 @@ import {
} from "../../domain/analysisErrorCore.js"; } from "../../domain/analysisErrorCore.js";
import { createEvidence } from "../../domain/evidence.js"; import { createEvidence } from "../../domain/evidence.js";
import { err, ok } from "../../domain/result.js"; import { err, ok } from "../../domain/result.js";
import type { SqliteDatabase } from "../../domain/sqlite/sqliteDatabase.js";
import { diffStateViews } from "../../domain/desktop/stateChanges.js";
import { readSqliteStateView } from "./DesktopSqliteState.js"; import { readSqliteStateView } from "./DesktopSqliteState.js";
const OPERATION = "inspect_sqlite_database"; const OPERATION = "inspect_sqlite_database";
@@ -36,7 +34,7 @@ const table = (name: string) => ({
}); });
/** A SQLite report with two tables, and one row of the selected table. */ /** A SQLite report with two tables, and one row of the selected table. */
const report = (selected: string | undefined): SqliteDatabase => ({ const report = (selected: string | undefined) => ({
artifact: { path: "/staged/state.db", sha256: "a".repeat(64), bytes: 4096 }, artifact: { path: "/staged/state.db", sha256: "a".repeat(64), bytes: 4096 },
wal: null, wal: null,
engine: { name: "SQLite", version: "3.50.2" }, engine: { name: "SQLite", version: "3.50.2" },
@@ -88,79 +86,6 @@ const service = (failure: (table: string) => AnalysisError | undefined) => ({
}); });
describe("readSqliteStateView", () => { describe("readSqliteStateView", () => {
it("distinguishes special REAL values from text when comparing captured rows", async () => {
const read = async (type: "real" | "text") =>
readSqliteStateView(
{
inspect: async (input: unknown) => {
const selected = (input as { table?: string }).table;
const result: SqliteDatabase = report(selected);
result.schema.tables = [table("notes")];
if (result.rows !== null) {
result.rows.values = (
["Infinity", "-Infinity", "-0"] as const
).map((value) => [
type === "real"
? { type: "real", value }
: {
type: "text",
value,
bytes_hex: Buffer.from(value).toString("hex"),
encoding: "UTF-8",
},
]);
result.rows.returned_rows = 3;
}
return ok(
createEvidence(
undefined,
{ id: "sqlite", name: "SQLite", version: "1" },
{
operation: OPERATION,
parameters: {},
result,
confidence: "observed",
},
),
);
},
},
"/staged/state.db",
);
const real = await read("real");
const text = await read("text");
expect(real).toMatchObject({
ok: true,
view: {
tables: {
notes: {
rows: [
[{ real: "Infinity" }],
[{ real: "-Infinity" }],
[{ real: "-0" }],
],
},
},
},
});
expect(text).toMatchObject({
ok: true,
view: {
tables: {
notes: {
rows: [["Infinity"], ["-Infinity"], ["-0"]],
},
},
},
});
if (!real.ok || !text.ok) throw new Error("Expected readable SQLite views");
expect(diffStateViews("sqlite", real.view, text.view)).toMatchObject({
added_count: 3,
removed_count: 3,
complete: true,
});
});
it("keeps the tables that read and records the error of one that did not", async () => { it("keeps the tables that read and records the error of one that did not", async () => {
const read = await readSqliteStateView( const read = await readSqliteStateView(
service((name) => service((name) =>
@@ -27,7 +27,7 @@ type Cell = z.infer<typeof sqliteValueSchema>;
/** /**
* SQLite cells as JSON. Values that plain JSON would conflate keep a tagged * SQLite cells as JSON. Values that plain JSON would conflate keep a tagged
* form: integers beyond the safe range, text whose bytes did not decode * form: integers beyond the safe range, text whose bytes did not decode
* cleanly, special REAL values, and blobs. * cleanly, and blobs.
*/ */
const cellValue = (cell: Cell): JsonValue => { const cellValue = (cell: Cell): JsonValue => {
switch (cell.type) { switch (cell.type) {
@@ -38,7 +38,7 @@ const cellValue = (cell: Cell): JsonValue => {
return Number.isSafeInteger(value) ? value : { integer: cell.value }; return Number.isSafeInteger(value) ? value : { integer: cell.value };
} }
case "real": case "real":
return typeof cell.value === "string" ? { real: cell.value } : cell.value; return cell.value;
case "text": case "text":
return cell.value.includes("\uFFFD") return cell.value.includes("\uFFFD")
? { text: cell.value, bytes_hex: cell.bytes_hex } ? { text: cell.value, bytes_hex: cell.bytes_hex }
@@ -4,7 +4,6 @@ import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCo
import { err, ok } from "../../domain/result.js"; import { err, ok } from "../../domain/result.js";
import { createAnalysisExecution } from "../AnalysisProvider.js"; import { createAnalysisExecution } from "../AnalysisProvider.js";
import { sqliteDatabaseSchema } from "../../domain/sqlite/sqliteDatabase.js"; import { sqliteDatabaseSchema } from "../../domain/sqlite/sqliteDatabase.js";
import { createAnalysisProfile } from "../../domain/analysisProfile.js";
const fixture = () => ({ const fixture = () => ({
artifact: { path: "/selected.db", sha256: "a".repeat(64), bytes: 4096 }, artifact: { path: "/selected.db", sha256: "a".repeat(64), bytes: 4096 },
@@ -237,59 +236,39 @@ it.each(["path", "digest", "subject", "wal", "table"])(
}, },
); );
it.each([false, true])( it("preserves completed observed evidence when cancellation arrives after provider completion", async () => {
"retains completed SQLite facts with late cancellation=%s", const value = fixture();
async (cancelled) => { const service = new SqliteDatabaseService({
const value = fixture(); inspect: () => Promise.resolve(ok(execution(value))),
const observed = execution(value); });
const profile = createAnalysisProfile( expect(await service.inspect({ path: "/selected.db" })).toMatchObject({
{ ...observed.provider, version: value.engine.version }, ok: true,
{ value: {
schema: "inventory",
},
);
const relationshipCoverage = {
status: "unknown",
reason: "Relationships were not inspected",
} as const;
const completed = {
...observed,
analysisProfile: profile,
relationshipCoverage,
};
const controller = new AbortController();
const reason = Object.freeze({
requested: "cancel after database inspection",
});
const service = new SqliteDatabaseService({
inspect: () => {
if (cancelled) controller.abort(reason);
return Promise.resolve(ok(completed));
},
});
const result = await service.inspect(
{ path: "/selected.db" },
{ signal: controller.signal },
);
expect(result.ok).toBe(!cancelled);
if (!result.ok) {
expect(result.error._tag).toBe("AnalysisCancelledError");
expect(result.error.cause).toBe(reason);
}
const evidence = result.ok ? result.value : result.error.partialObservation;
expect(evidence).toMatchObject({
operation: "inspect_sqlite_database",
confidence: "observed", confidence: "observed",
subject: { subject: {
local_path: "/selected.db", local_path: "/selected.db",
digest: { sha256: "a".repeat(64) }, digest: { sha256: "a".repeat(64) },
}, },
normalized_result: value, normalized_result: value,
analysis_profile: profile, },
relationship_coverage: relationshipCoverage, });
}); const controller = new AbortController();
}, const cancelled = new SqliteDatabaseService({
); inspect: () => {
controller.abort();
return Promise.resolve(ok(execution(value)));
},
});
expect(
await cancelled.inspect(
{ path: "/selected.db" },
{ signal: controller.signal },
),
).toMatchObject({
ok: true,
value: { normalized_result: value },
});
});
it("preserves the exact caller reason when inspection is already aborted", async () => { it("preserves the exact caller reason when inspection is already aborted", async () => {
const reason = Object.freeze({ requested: "stop" }); const reason = Object.freeze({ requested: "stop" });
+18 -13
View File
@@ -3,10 +3,11 @@ import type { ExecutionOptions } from "../AnalysisProvider.js";
import type { SqliteDatabasePort } from "./SqliteDatabasePort.js"; import type { SqliteDatabasePort } from "./SqliteDatabasePort.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js"; import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import { AnalysisOutputError } from "../../domain/analysisErrorCore.js"; import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import type { Evidence, EvidenceSubjectTarget } from "../../domain/evidence.js"; import {
import { createAnalysisExecutionEvidence } from "../../domain/analysisObservation.js"; createEvidence,
import { ownedJsonValueSchema } from "../../domain/jsonValue.js"; type Evidence,
import { completeProviderEvidence } from "../ProviderOperationExecution.js"; type EvidenceSubjectTarget,
} from "../../domain/evidence.js";
import { import {
inspectSqliteDatabaseInputSchema, inspectSqliteDatabaseInputSchema,
sqliteDatabaseSchema, sqliteDatabaseSchema,
@@ -18,7 +19,7 @@ import {
sqliteSchemaScope, sqliteSchemaScope,
sqliteInspectionParameters, sqliteInspectionParameters,
} from "../../domain/sqlite/sqliteDatabase.js"; } from "../../domain/sqlite/sqliteDatabase.js";
import { err, type Result } from "../../domain/result.js"; import { err, ok, type Result } from "../../domain/result.js";
const OPERATION = "inspect_sqlite_database"; const OPERATION = "inspect_sqlite_database";
@@ -101,14 +102,18 @@ export class SqliteDatabaseService {
"SQLite adapter returned malformed data or changed the selected database identity/table/row limit/worker budget.", "SQLite adapter returned malformed data or changed the selected database identity/table/row limit/worker budget.",
), ),
); );
return completeProviderEvidence( return ok(
createAnalysisExecutionEvidence( createEvidence(execution.subject, execution.provider, {
OPERATION, operation: OPERATION,
sqliteInspectionParameters(input.value), parameters: sqliteInspectionParameters(input.value),
{ ...execution, result: ownedJsonValueSchema.parse(report.data) }, result: report.data,
"observed", ...(execution.rawResult === null
), ? {}
options?.signal, : { rawResult: execution.rawResult }),
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
}),
); );
} }
} }
@@ -20,6 +20,11 @@ it("reads a Python 3.9 reference-flagged PYZ table", () => {
]); ]);
}); });
it("reads marshal one-byte string tags as Latin-1", () => {
for (const hex of ["6101000000e9", "4101000000e9", "7a01e9", "5a01e9"])
expect(readMarshalSubset(bytes(hex))).toEqual({ ok: true, value: "é" });
});
it("decodes CPython marshal Unicode with surrogatepass", () => { it("decodes CPython marshal Unicode with surrogatepass", () => {
expect( expect(
readMarshalSubset( readMarshalSubset(
+2 -8
View File
@@ -168,14 +168,8 @@ const readAsciiBytes = (
): Result<string, PyInstallerFailure> => { ): Result<string, PyInstallerFailure> => {
const bytes = readBytes(cursor, length); const bytes = readBytes(cursor, length);
if (!bytes.ok) return bytes; if (!bytes.ok) return bytes;
if (bytes.value.some((byte) => byte > 0x7f)) // These tags use CPython's one-byte Unicode constructor, not an ASCII check.
return err( return ok(bytes.value.toString("latin1"));
pyInstallerFailure(
"format",
"PYZ marshal ASCII string contains a non-ASCII byte",
),
);
return ok(bytes.value.toString("ascii"));
}; };
const readUtf8 = ( const readUtf8 = (
-158
View File
@@ -1,158 +0,0 @@
import { createHash } from "node:crypto";
import { expect, it, vi } from "vitest";
import { StructuredContentMcpClient } from "../providers/StructuredContentMcpClient.js";
import { AgentEnvComputer } from "./AgentEnvComputer.js";
import type { ComputerHelper } from "./AgentEnvComputerHelper.js";
it.each(["http", "invalid_json", "readiness"] as const)(
"redacts the complete control token in %s failure diagnostics",
async (failure) => {
const token = "fake-control-token-keep-private-123456789";
const status = failure === "http" ? 403 : 200;
const ordinary =
failure === "http" ? "x".repeat(500) : "ordinary target text";
const body =
failure === "readiness"
? JSON.stringify({
ready: false,
checks: [
{
name: "app",
ok: false,
final: true,
detail: `${token} ${ordinary}`,
},
],
})
: failure === "http"
? `${ordinary}${token}`
: `${token} ${ordinary}`;
const connect = vi
.spyOn(StructuredContentMcpClient.prototype, "connect")
.mockResolvedValue(undefined);
const close = vi
.spyOn(StructuredContentMcpClient.prototype, "close")
.mockResolvedValue(undefined);
const fetch = vi
.spyOn(globalThis, "fetch")
.mockResolvedValue(new Response(body, { status }));
try {
const error = await AgentEnvComputer.connect(
{
handshake: async () => ({
sandbox_id: "controlled-sandbox",
base_url: "http://127.0.0.1:1",
mcp_url: "http://127.0.0.1:1/mcp",
control_token: token,
}),
} as unknown as ComputerHelper,
new AbortController().signal,
).catch((cause: unknown) => cause);
expect(error).toBeInstanceOf(Error);
if (!(error instanceof Error)) throw error;
expect(error.message).not.toContain(token.slice(0, 10));
expect(error.message).toContain("[control tok");
expect(error.message).toContain(ordinary.slice(0, 100));
if (failure !== "readiness") {
expect(error.cause).toBeInstanceOf(Error);
expect(String(error.cause)).not.toContain(token.slice(0, 10));
}
} finally {
fetch.mockRestore();
close.mockRestore();
connect.mockRestore();
}
},
);
it.each(["canonical", "missing", "rpc_error"] as const)(
"validates recording replies and redacts their errors (%s)",
async (answer) => {
const present = answer === "canonical";
const token = "fake-recording-control-token-123456789";
const path = "rec-0123/0000.mp4";
const homePath = `.agentenv-recordings/${path}`;
const bytes = Buffer.from("controlled recording");
const connect = vi
.spyOn(StructuredContentMcpClient.prototype, "connect")
.mockResolvedValue(undefined);
const close = vi
.spyOn(StructuredContentMcpClient.prototype, "close")
.mockResolvedValue(undefined);
const call = vi
.spyOn(StructuredContentMcpClient.prototype, "callTool")
.mockResolvedValue({ content: [] });
const fetch = vi
.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(Response.json({ ready: true }))
.mockResolvedValueOnce(
Response.json(
answer === "rpc_error"
? { error: { message: `${token} ordinary target text` } }
: {
result: {
parts: [
{
kind: "file",
file: { bytes: bytes.toString("base64") },
metadata: {
"urn:unrelated": {
path: present ? "wrong-path" : homePath,
},
...(present
? {
"urn:agentenv:data-objects/v1": {
path: homePath,
},
}
: {}),
},
},
],
},
},
),
);
try {
const { computer } = await AgentEnvComputer.connect(
{
handshake: async () => ({
sandbox_id: "controlled-sandbox",
base_url: "http://127.0.0.1:1",
mcp_url: "http://127.0.0.1:1/mcp",
control_token: token,
}),
release: async () => ({ verified: true, teardown: "agentenv" }),
} as unknown as ComputerHelper,
new AbortController().signal,
);
const read = computer.recordingSegments({
recording_id: "rec-0123",
segments: [
{
index: 0,
path,
bytes: bytes.length,
sha256: createHash("sha256").update(bytes).digest("hex"),
},
],
});
if (present) await expect(read).resolves.toEqual([{ path, bytes }]);
else if (answer === "rpc_error") {
const failure = await read.catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(Error);
if (!(failure instanceof Error)) throw failure;
expect(failure.message).toContain("ordinary target text");
expect(failure.message).toContain("[control token]");
expect(failure.message).not.toContain(token);
} else await expect(read).rejects.toThrow("missing or differs");
await computer.close();
} finally {
fetch.mockRestore();
call.mockRestore();
close.mockRestore();
connect.mockRestore();
}
},
);
+7 -17
View File
@@ -14,7 +14,6 @@ import type {
ComputerRelease, ComputerRelease,
} from "./AgentEnvComputerHelper.js"; } from "./AgentEnvComputerHelper.js";
import { import {
DATA_OBJECTS,
dataGetSchema, dataGetSchema,
readHomeFiles, readHomeFiles,
writeHomeFiles, writeHomeFiles,
@@ -499,10 +498,10 @@ export class AgentEnvComputer {
); );
if ("error" in answer) if ("error" in answer)
throw new Error( throw new Error(
`data/get of recording segment ${segment.path} failed: ${this.redacted(answer.error.message)}`, `data/get of recording segment ${segment.path} failed: ${answer.error.message}`,
); );
const encoded = answer.result.parts.find( const encoded = answer.result.parts.find(
(part) => part.metadata?.[DATA_OBJECTS]?.path === path, (part) => Object.values(part.metadata ?? {})[0]?.path === path,
)?.file?.bytes; )?.file?.bytes;
const bytes = const bytes =
encoded === undefined ? undefined : Buffer.from(encoded, "base64"); encoded === undefined ? undefined : Buffer.from(encoded, "base64");
@@ -654,11 +653,9 @@ export class AgentEnvComputer {
const failed = waiting.find((check) => check.final === true); const failed = waiting.find((check) => check.final === true);
if (failed !== undefined || Date.now() > deadline) if (failed !== undefined || Date.now() > deadline)
throw new ComputerStartFailure( throw new ComputerStartFailure(
this.redacted( failed === undefined
failed === undefined ? `The computer was not ready within ${String(READY_TIMEOUT_MS / 1000)} s; waiting on ${waiting.map((check) => check.name).join(", ")}.`
? `The computer was not ready within ${String(READY_TIMEOUT_MS / 1000)} s; waiting on ${waiting.map((check) => check.name).join(", ")}.` : `The computer's ${failed.name} check failed: ${failed.detail ?? "no detail"}.`,
: `The computer's ${failed.name} check failed: ${failed.detail ?? "no detail"}.`,
),
"Read the helper's output; `agent-env run <bundle> --task smoke --no-emulate` reproduces the deploy.", "Read the helper's output; `agent-env run <bundle> --task smoke --no-emulate` reproduces the deploy.",
{ reason: "not_ready" }, { reason: "not_ready" },
); );
@@ -705,15 +702,8 @@ export class AgentEnvComputer {
const text = await readBounded(response, limit.bytes, limit.what); const text = await readBounded(response, limit.bytes, limit.what);
if (response.status < 200 || response.status > 299) if (response.status < 200 || response.status > 299)
throw new Error( throw new Error(
`The computer answered ${path} with HTTP ${String(response.status)}: ${this.redacted(text).slice(0, 512)}`, `The computer answered ${path} with HTTP ${String(response.status)}: ${this.redacted(text.slice(0, 512))}`,
); );
try { return JSON.parse(text) as unknown;
return JSON.parse(text) as unknown;
} catch {
// Parser diagnostics can quote a token fragment that exact redaction cannot match.
throw new Error(
`The computer answered ${path} with invalid JSON: ${this.redacted(text).slice(0, 512)}`,
);
}
} }
} }
@@ -1,54 +0,0 @@
import { chmod, lstat, mkdir } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { desktopApplicationInventorySchema } from "../domain/desktop/desktopCapture.js";
import { stateDirectories } from "./ApplicationStateDirectories.js";
it.skipIf(process.platform === "win32" || process.getuid?.() === 0)(
"preserves unknown state-path existence when permission denies the probe",
async () => {
const home = await createTestTempDirectory("state-path-existence-");
const sealed = join(home, "sealed");
const denied = join(sealed, "notes");
const present = join(home, ".notes");
const absent = join(home, ".local/share/notes");
await mkdir(denied, { recursive: true });
await mkdir(present);
await chmod(sealed, 0o000);
try {
await expect(lstat(denied)).rejects.toMatchObject({ code: "EACCES" });
const directories = await stateDirectories(
["notes"],
{
home,
environment: { XDG_CONFIG_HOME: sealed },
},
[],
);
const facts = new Map(directories.map((entry) => [entry.path, entry]));
expect(facts.get(present)).toEqual({
path: present,
exists_at_start: true,
});
expect(facts.get(absent)).toEqual({
path: absent,
exists_at_start: false,
});
expect(facts.get(denied)).toMatchObject({
path: denied,
exists_at_start: null,
existence_error: expect.stringContaining("EACCES"),
});
expect(
desktopApplicationInventorySchema.shape.state_directories.parse(
directories,
),
).toEqual(directories);
} finally {
await chmod(sealed, 0o700);
}
},
);
+4 -16
View File
@@ -102,21 +102,9 @@ export const stateDirectories = async (
...extra, ...extra,
]); ]);
return Promise.all( return Promise.all(
[...paths].map(async (path) => { [...paths].map(async (path) => ({
try { path,
await lstat(path); exists_at_start: (await lstat(path).catch(() => undefined)) !== undefined,
return { path, exists_at_start: true }; })),
} catch (cause: unknown) {
const code = (cause as NodeJS.ErrnoException).code;
return code === "ENOENT" || code === "ENOTDIR"
? { path, exists_at_start: false }
: {
path,
exists_at_start: null,
existence_error:
cause instanceof Error ? cause.message : String(cause),
};
}
}),
); );
}; };
+1 -81
View File
@@ -1,5 +1,5 @@
import { existsSync } from "node:fs"; import { existsSync } from "node:fs";
import { mkdir, writeFile } from "node:fs/promises"; import { writeFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
@@ -7,89 +7,9 @@ import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { import {
APT_LOCATIONS, APT_LOCATIONS,
inspectAptRepositories,
resolveAptLocations, resolveAptLocations,
} from "./AptRepositoryInventory.js"; } from "./AptRepositoryInventory.js";
import { runHostQuery, type QueryRunner } from "./HostQuery.js"; import { runHostQuery, type QueryRunner } from "./HostQuery.js";
import {
GPGV_A_ONLY,
VENDOR_A,
VENDOR_B,
} from "../domain/desktop/aptKeys.fixture.js";
it.each(["list", "sources"] as const)(
"does not broaden %s Signed-By fingerprint restrictions to every key in a keyring",
async (format) => {
const root = await createTestTempDirectory("apt-signed-by-");
const sources = join(root, `vendor.${format}`);
const key = join(root, "vendor.gpg");
const trustedParts = join(root, "trusted.gpg.d");
await mkdir(trustedParts);
await writeFile(key, VENDOR_A.key);
await writeFile(
join(root, "vendor.example_repo_InRelease"),
"controlled release",
);
const source = "https://vendor.example/repo stable/main amd64 Packages";
const targets = [
`Description: ${source}`,
`Sourcesentry: ${sources}:1`,
"Repo-URI: https://vendor.example/repo/",
"Release: stable",
"Component: main",
"Base-URI: https://vendor.example/repo/",
`Filename: ${join(root, "Packages")}`,
].join("\n");
let verifications = 0;
const runQuery: QueryRunner = async (command) => {
if (command === "apt-get") return { stdout: targets, exitCode: 0 };
if (command !== "gpgv") throw new Error(`Unexpected query ${command}`);
verifications += 1;
return { stdout: GPGV_A_ONLY, exitCode: 1 };
};
const inspect = async (signedBy: string) => {
await writeFile(
sources,
format === "list"
? `deb [signed-by=${signedBy.replaceAll(" ", ",")}] https://vendor.example/repo stable main\n`
: `Types: deb\nURIs: https://vendor.example/repo\nSuites: stable\nComponents: main\nSigned-By: ${signedBy}\n`,
);
return inspectAptRepositories(
{ sources: [source], origins: ["Vendor"] },
{
runQuery,
apt: { lists: root, trustedParts, trusted: key },
},
undefined,
);
};
const pathOnly = await inspect(key);
expect(pathOnly.repositories[0]?.signing_key).toMatchObject({
path: key,
fingerprint: VENDOR_A.fingerprint,
});
expect(pathOnly.keys.size).toBe(1);
for (const selector of [VENDOR_B.fingerprint, `${VENDOR_B.fingerprint}!`]) {
for (const signedBy of [selector, `${key} ${selector}`]) {
const restricted = await inspect(signedBy);
expect(restricted.repositories[0]).toMatchObject({
source,
sources_file: sources,
uri: "https://vendor.example/repo",
suite: "stable",
components: ["main"],
signing_key: null,
signing_key_reason: expect.stringContaining(selector),
});
expect(restricted.repositories[0]?.signing_key_reason).toContain(
"not supported",
);
expect(restricted.keys.size).toBe(0);
}
}
expect(verifications).toBe(1);
},
);
describe.skipIf( describe.skipIf(
process.platform !== "linux" || !existsSync("/usr/bin/apt-config"), process.platform !== "linux" || !existsSync("/usr/bin/apt-config"),
+2 -6
View File
@@ -138,7 +138,8 @@ const candidateKeys = async (
entry: AptSourceEntry, entry: AptSourceEntry,
apt: AptLocations, apt: AptLocations,
): Promise<readonly string[]> => { ): Promise<readonly string[]> => {
if (entry.signedBy.length > 0) return entry.signedBy; const signedBy = entry.signedBy.filter((value) => value.startsWith("/"));
if (signedBy.length > 0) return signedBy;
return [ return [
...(await fileNames(apt.trustedParts)) ...(await fileNames(apt.trustedParts))
.filter((name) => name.endsWith(".gpg") || name.endsWith(".asc")) .filter((name) => name.endsWith(".gpg") || name.endsWith(".asc"))
@@ -198,11 +199,6 @@ const keyCheck = async (
return { return {
reason: `no sources entry at ${target.sourcesFile}:${String(target.position)} configures it`, reason: `no sources entry at ${target.sourcesFile}:${String(target.position)} configures it`,
}; };
const selectors = entry.signedBy.filter((value) => !value.startsWith("/"));
if (selectors.length > 0)
return {
reason: `Signed-By selectors ${selectors.map((value) => JSON.stringify(value)).join(", ")} are not supported; REA cannot verify this repository's fingerprint restrictions, so no signing key is retained`,
};
const release = await signedRelease(target); const release = await signedRelease(target);
if (release === undefined) if (release === undefined)
return { return {
+2 -2
View File
@@ -1,7 +1,7 @@
import { z } from "zod"; import { z } from "zod";
/** Where a data-plane file part names its path in the home. */ /** Where a data-plane file part names its path in the home. */
export const DATA_OBJECTS = "urn:agentenv:data-objects/v1"; const DATA_OBJECTS = "urn:agentenv:data-objects/v1";
/** A `data/get` answer: file parts, each naming its home path, or an error. */ /** A `data/get` answer: file parts, each naming its home path, or an error. */
export const dataGetSchema = z.union([ export const dataGetSchema = z.union([
z.object({ z.object({
@@ -74,7 +74,7 @@ export const readHomeFiles = async (
? [] ? []
: [ : [
{ {
path: part.metadata?.[DATA_OBJECTS]?.path, path: Object.values(part.metadata ?? {})[0]?.path,
bytes: Buffer.from(encoded, "base64"), bytes: Buffer.from(encoded, "base64"),
}, },
]; ];
@@ -1045,52 +1045,3 @@ describe.skipIf(process.platform === "win32")(
}); });
}, },
); );
it.skipIf(process.platform === "win32" || process.getuid?.() === 0)(
"keeps apt list presence unknown when its directory cannot be read",
async () => {
const { root, inspect } = await writerFixture();
const lists = join(root, "lists");
await chmod(lists, 0o000);
try {
const { installed, limitations } = await inspect({
executable: "libreoffice",
});
expect(installed.package_lists_present).toBeNull();
expect(limitations.join("\n")).toContain("could not be read");
expect(limitations.join("\n")).toContain("EACCES");
} finally {
await chmod(lists, 0o755);
}
},
);
it.skipIf(process.platform === "win32")(
"keeps package ownership unknown for a literal path dpkg would treat as a pattern",
async () => {
const { root, runQuery, inspect } = await writerFixture();
const executable = join(root, "bin/libreoffice[1]");
await writeFile(executable, "#!/bin/sh\n");
await chmod(executable, 0o755);
const result = await inspect(
{ executable },
{
run: async (command, argv, signal) => {
if (
command === "dpkg-query" &&
argv[0] === "-S" &&
argv.includes(executable)
)
// A glob search can name another file without establishing who owns the literal brackets.
return {
stdout: `other-package: ${executable.replace("[1]", "1")}`,
exitCode: 0,
};
return runQuery(command, argv, signal);
},
},
);
expect(result.installed).toMatchObject({ kind: "unknown", name: null });
expect(result.limitations.join("\n")).toContain("pattern characters");
},
);
+4 -17
View File
@@ -1,5 +1,4 @@
import type { DesktopLibraryPolicy } from "../domain/desktop/libraryPolicy.js"; import type { DesktopLibraryPolicy } from "../domain/desktop/libraryPolicy.js";
import { describeValidationFailure } from "../domain/validationFailure.js";
import { readdir } from "node:fs/promises"; import { readdir } from "node:fs/promises";
import { import {
@@ -204,11 +203,6 @@ const dpkgOwner = async (
{ host, signal }: InventoryContext, { host, signal }: InventoryContext,
): Promise<Ownership> => { ): Promise<Ownership> => {
for (const path of ownershipCandidates(paths)) { for (const path of ownershipCandidates(paths)) {
if (/[*?[\\]/u.test(path))
return {
kind: "unknown",
reason: `dpkg-query cannot safely query ${path} containing pattern characters`,
};
const outcome = await host.runQuery("dpkg-query", ["-S", path], signal); const outcome = await host.runQuery("dpkg-query", ["-S", path], signal);
if (outcome === "missing") if (outcome === "missing")
return { kind: "unknown", reason: "dpkg-query is not installed" }; return { kind: "unknown", reason: "dpkg-query is not installed" };
@@ -289,17 +283,10 @@ export const debianPackage = async (
await host.runQuery("apt-cache", ["policy", name], signal), await host.runQuery("apt-cache", ["policy", name], signal),
); );
const apt = await resolveAptLocations(host.runQuery, signal, host.apt); const apt = await resolveAptLocations(host.runQuery, signal, host.apt);
const listsPresent = await readdir(apt.lists).then( const listsPresent = (
(lists) => lists.some((list) => /_Packages(?:\.[a-z0-9]+)?$/u.test(list)), await readdir(apt.lists).catch(() => [] as string[])
(cause: unknown) => { ).some((list) => /_Packages(?:\.[a-z0-9]+)?$/u.test(list));
if ((cause as NodeJS.ErrnoException).code === "ENOENT") return false; if (!listsPresent)
limitations.push(
`The apt package lists directory ${apt.lists} could not be read, so whether package lists are present is unknown: ${describeValidationFailure(cause)}`,
);
return null;
},
);
if (listsPresent === false)
limitations.push( limitations.push(
"No apt package lists are present, so whether the installed version comes from a package archive is unknown.", "No apt package lists are present, so whether the installed version comes from a package archive is unknown.",
); );
@@ -143,50 +143,6 @@ describe.skipIf(process.platform !== "linux")(
describe.skipIf(process.platform === "win32")( describe.skipIf(process.platform === "win32")(
"runDesktopSession stderr", "runDesktopSession stderr",
() => { () => {
it.each([
{ fails: false, cancelAt: "cleanup" },
{ fails: true, cancelAt: "cleanup" },
{ fails: false, cancelAt: "exit" },
])(
"preserves the session outcome when cancellation arrives during $cancelAt (checkpoint fails: $fails)",
async ({ fails, cancelAt }) => {
const root = await createTestTempDirectory(
"desktop-session-final-cancel-",
);
const controller = new AbortController();
const reason = new Error("cancelled during cleanup");
const failure = new Error("stop checkpoint failed");
const terminate = vi.mocked(terminateTokenOwnedProcesses);
terminate.mockImplementationOnce(async (...args) => {
if (cancelAt === "cleanup") controller.abort(reason);
const original = await vi.importActual<
typeof import("../process/TokenOwnedTermination.js")
>("../process/TokenOwnedTermination.js");
return original.terminateTokenOwnedProcesses(...args);
});
const checkpoints: string[] = [];
const result = await runDesktopSession({
...shellPlan(root, "exit 0"),
signal: controller.signal,
nextRequest: () => ({ kind: "stop" }),
checkpoint: async (name) => {
checkpoints.push(name);
if (cancelAt === "exit" && name === "exit")
controller.abort(reason);
if (fails) throw failure;
},
}).catch((cause: unknown) => cause);
if (fails) expect(result).toBe(failure);
else
expect(result).toMatchObject({
session: { end_reason: "cancelled", cleanup: { verified: true } },
});
expect(checkpoints).toEqual(
cancelAt === "exit" ? ["stop", "exit"] : ["stop"],
);
},
);
it("keeps the end of an app's stderr when it writes more than the session keeps", async () => { it("keeps the end of an app's stderr when it writes more than the session keeps", async () => {
const root = await createTestTempDirectory("desktop-session-stderr-"); const root = await createTestTempDirectory("desktop-session-stderr-");
const cause = const cause =
+1 -4
View File
@@ -364,13 +364,10 @@ export const runDesktopSession = async (
"stop", "stop",
); );
cleanup = await releaseOnce(); cleanup = await releaseOnce();
signal.throwIfAborted();
await checkpointNow(plan, observations, "exit", "exit"); await checkpointNow(plan, observations, "exit", "exit");
signal.throwIfAborted();
} catch (cause: unknown) { } catch (cause: unknown) {
const cancelled = signal.aborted;
cleanup = await releaseOnce(); cleanup = await releaseOnce();
if (!cancelled) { if (!signal.aborted) {
const left = unreleased(spawned, supervisor, cleanup); const left = unreleased(spawned, supervisor, cleanup);
if (left === undefined) throw cause; if (left === undefined) throw cause;
throw new DesktopSessionCleanupError(cause, left); throw new DesktopSessionCleanupError(cause, left);
+1 -60
View File
@@ -2,14 +2,12 @@ import { createHash } from "node:crypto";
import { mkdir, readFile, readdir, rm, writeFile } from "node:fs/promises"; import { mkdir, readFile, readdir, rm, writeFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import type { DesktopPathState } from "../domain/desktop/desktopStateDiff.js"; import type { DesktopPathState } from "../domain/desktop/desktopStateDiff.js";
import { DesktopContentStore } from "./DesktopContentStore.js"; import { DesktopContentStore } from "./DesktopContentStore.js";
import { walkHomeState } from "./HomeStateWalker.js"; import { walkHomeState } from "./HomeStateWalker.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { RegularFileCleanupFailure } from "../filesystem/OwnedRegularFile.js";
const sha256 = (text: string) => const sha256 = (text: string) =>
createHash("sha256").update(text).digest("hex"); createHash("sha256").update(text).digest("hex");
@@ -35,63 +33,6 @@ const store = async (root: string, maxFileBytes: number, maxBytes: number) => {
return created; return created;
}; };
it("DesktopContentStore leaves an occupied partial filename unowned", async () => {
const root = await createTestTempDirectory("desktop-content-occupied-");
const content = await store(root, 1024, 1024);
const bytes = Buffer.from("new content");
const partial = join(content.directory, `${sha256("new content")}.partial`);
await writeFile(partial, "foreign content");
await expect(content.retainBytes(bytes)).rejects.toMatchObject({
code: "EEXIST",
});
await expect(content.discard()).rejects.toThrow();
expect(await readFile(partial, "utf8")).toBe("foreign content");
});
it("DesktopContentStore retains every unpublished file until discard", async () => {
const root = await createTestTempDirectory("desktop-content-unpublished-");
const content = await store(root, 1024, 1024);
const digests = [sha256("first"), sha256("second")];
for (const [index, text] of ["first", "second"].entries()) {
await mkdir(join(content.directory, digests[index]!));
await expect(content.retainBytes(Buffer.from(text))).rejects.toThrow();
}
await expect(content.discard()).rejects.toThrow();
expect((await readdir(content.directory)).sort()).toEqual(digests.sort());
});
it("DesktopContentStore retains a failed output close until cleanup retry", async () => {
const root = await createTestTempDirectory("desktop-content-close-");
const content = await store(root, 1024, 1024);
const closeFailure = new Error("output descriptor close failed");
const close = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockRejectedValue(closeFailure);
try {
const failure = await content
.retainBytes(Buffer.from("pending"))
.catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(RegularFileCleanupFailure);
if (!(failure instanceof RegularFileCleanupFailure))
throw new Error("Expected retained output close failure");
await expect(content.close()).rejects.toMatchObject({
cleanup: {
resources: [join(content.directory, `${sha256("pending")}.partial`)],
},
});
close.mockRestore();
await content.close();
await expect(failure.owner.handle.stat()).rejects.toMatchObject({
code: "EBADF",
});
await content.discard();
expect(await readdir(join(root, "capture"))).toEqual([]);
} finally {
close.mockRestore();
await content.close();
}
});
describe("DesktopContentStore", () => { describe("DesktopContentStore", () => {
it("stores each distinct content once and charges the budget once", async () => { it("stores each distinct content once and charges the budget once", async () => {
const root = await createTestTempDirectory("desktop-content-"); const root = await createTestTempDirectory("desktop-content-");
+9 -26
View File
@@ -1,14 +1,13 @@
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { mkdir, open, rename, rm, rmdir } from "node:fs/promises"; import { mkdir, rename, rm, rmdir, writeFile } from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import type { DesktopApplicationCapture } from "../domain/desktop/desktopCapture.js"; import type { DesktopApplicationCapture } from "../domain/desktop/desktopCapture.js";
import type { DesktopPathState } from "../domain/desktop/desktopStateDiff.js"; import type { DesktopPathState } from "../domain/desktop/desktopStateDiff.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; import type { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { import {
RegularFileCleanupFailure, RegularFileCleanupFailure,
withOwnedFileHandle,
withOwnedRegularFile, withOwnedRegularFile,
} from "../filesystem/OwnedRegularFile.js"; } from "../filesystem/OwnedRegularFile.js";
import { import {
@@ -46,8 +45,8 @@ export class DesktopContentStore {
readonly #omissions: Omission[] = []; readonly #omissions: Omission[] = [];
/** Handles whose close failed; kept owned so `close` can retry them. */ /** Handles whose close failed; kept owned so `close` can retry them. */
readonly #unclosed = new ArtifactResourceScope(); readonly #unclosed = new ArtifactResourceScope();
/** Every partial file this store created but has not published or removed. */ /** The partial file being written, removed by `discard` if it was left. */
readonly #unpublished = new Set<string>(); #writing: string | undefined;
#remainingBytes: number; #remainingBytes: number;
constructor( constructor(
@@ -95,10 +94,8 @@ export class DesktopContentStore {
* when the directory holds anything else or a removal fails. * when the directory holds anything else or a removal fails.
*/ */
async discard(): Promise<void> { async discard(): Promise<void> {
for (const path of this.#unpublished) { if (this.#writing !== undefined) await rm(this.#writing, { force: true });
await rm(path, { force: true }); this.#writing = undefined;
this.#unpublished.delete(path);
}
for (const sha256 of this.#stored.keys()) { for (const sha256 of this.#stored.keys()) {
await rm(join(this.#directory, sha256), { force: true }); await rm(join(this.#directory, sha256), { force: true });
this.#stored.delete(sha256); this.#stored.delete(sha256);
@@ -142,24 +139,10 @@ export class DesktopContentStore {
return { kind: "omitted", reason: "budget_exhausted" }; return { kind: "omitted", reason: "budget_exhausted" };
const target = join(this.#directory, sha256); const target = join(this.#directory, sha256);
const partial = `${target}.partial`; const partial = `${target}.partial`;
const owner = new OwnedFileHandle(await open(partial, "wx", 0o600)); this.#writing = partial;
this.#unpublished.add(partial); await writeFile(partial, bytes, { mode: 0o600, flag: "wx" });
await withOwnedFileHandle(
partial,
owner,
{
retainFailedClose: (handle) => {
this.#unclosed.retain({
kind: "file-handle",
resource: partial,
handle,
});
},
},
(file) => file.writeFile(bytes),
);
await rename(partial, target); await rename(partial, target);
this.#unpublished.delete(partial); this.#writing = undefined;
this.#stored.set(sha256, size); this.#stored.set(sha256, size);
this.#storedSizes.add(size); this.#storedSizes.add(size);
this.#remainingBytes -= size; this.#remainingBytes -= size;
-43
View File
@@ -85,49 +85,6 @@ describe.skipIf(!hasGpgv)("withGpgvKeyCheck with gpgv", () => {
}); });
describe("withGpgvKeyCheck cleanup", () => { describe("withGpgvKeyCheck cleanup", () => {
it.each(["completed", "failed"])(
"observes cancellation during final keyring cleanup after %s work",
async (completion) => {
const cancel = new AbortController();
const reason = new Error("cancelled during keyring cleanup");
const primary = new Error("verification failed");
const create = PrivateRuntimeRoot.create.bind(PrivateRuntimeRoot);
let path: string | undefined;
vi.spyOn(PrivateRuntimeRoot, "create").mockImplementationOnce(
async (options) => {
const root = await create(options);
path = root.path;
const close = root.close.bind(root);
vi.spyOn(root, "close").mockImplementationOnce(async () => {
cancel.abort(reason);
await close();
});
return root;
},
);
const { release, paths } = await files({ "a.gpg": VENDOR_A.key });
const result = withGpgvKeyCheck(
async () => ({ stdout: "", exitCode: 2 }),
cancel.signal,
async (check) => {
const verified = await check.firstVerifying(
[paths["a.gpg"] ?? ""],
[release],
);
if (completion === "failed") throw primary;
return verified;
},
);
await expect(result).rejects.toBe(
completion === "failed" ? primary : reason,
);
expect(path).toBeDefined();
await expect(access(path ?? "")).rejects.toMatchObject({
code: "ENOENT",
});
},
);
it("keeps the private keyring folder's owner when its removal fails", async () => { it("keeps the private keyring folder's owner when its removal fails", async () => {
const create = PrivateRuntimeRoot.create.bind(PrivateRuntimeRoot); const create = PrivateRuntimeRoot.create.bind(PrivateRuntimeRoot);
let owned: PrivateRuntimeRoot | undefined; let owned: PrivateRuntimeRoot | undefined;
-1
View File
@@ -174,6 +174,5 @@ export const withGpgvKeyCheck = async <Value>(
throw resources.cleanupFailure(outcome, [removed]); throw resources.cleanupFailure(outcome, [removed]);
} }
if (outcome.kind === "failed") throw outcome.cause; if (outcome.kind === "failed") throw outcome.cause;
signal?.throwIfAborted();
return outcome.value; return outcome.value;
}; };
+1 -37
View File
@@ -1,54 +1,18 @@
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js"; import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js";
import { readHomeFiles } from "./ComputerHomeFiles.js";
import { import {
AgentEnvComputer, AgentEnvComputer,
GUEST_FILE_ANSWER_COPIES, GUEST_FILE_ANSWER_COPIES,
GUEST_FILE_ANSWER_SOURCE, GUEST_FILE_ANSWER_SOURCE,
guestFileAnswerBytes, guestFileAnswerBytes,
guestFileAnswerCeiling,
guestFileAnswerRefusal, guestFileAnswerRefusal,
guestFileAnswerWorkingBytes, guestFileAnswerWorkingBytes,
retainGuestFiles, retainGuestFiles,
} from "./AgentEnvComputer.js"; } from "./AgentEnvComputer.js";
describe("guest file answers", () => { describe("guest file answers", () => {
it.each([true, false])(
"uses only the named data-object metadata for file identity (present: %s)",
async (present) => {
const files = await readHomeFiles(
async () => ({
result: {
parts: [
{
kind: "file",
file: { bytes: Buffer.from("source bytes").toString("base64") },
metadata: {
"urn:unrelated": { path: "other.txt" },
...(present
? {
"urn:agentenv:data-objects/v1": {
path: "selected.txt",
},
}
: {}),
},
},
],
},
}),
(text) => text,
["selected.txt"],
);
expect(files).toEqual([
{
path: present ? "selected.txt" : undefined,
bytes: Buffer.from("source bytes"),
},
]);
},
);
it("counts base64, the envelope, and the copies held while the answer is parsed", () => { it("counts base64, the envelope, and the copies held while the answer is parsed", () => {
expect(guestFileAnswerBytes(3, 1)).toBeGreaterThan(3); expect(guestFileAnswerBytes(3, 1)).toBeGreaterThan(3);
expect(guestFileAnswerBytes(3, 2)).toBeGreaterThan( expect(guestFileAnswerBytes(3, 2)).toBeGreaterThan(
+2 -2
View File
@@ -54,7 +54,7 @@ it.skipIf(process.platform === "win32")(
const bin = join(root, "bin"); const bin = join(root, "bin");
const lib = join(root, "lib"); const lib = join(root, "lib");
await Promise.all([mkdir(bin), mkdir(lib)]); await Promise.all([mkdir(bin), mkdir(lib)]);
const paths = Array.from({ length: 201 }, (_, index) => const paths = Array.from({ length: 102 }, (_, index) =>
join(lib, `lib${index}.so`), join(lib, `lib${index}.so`),
); );
const executable = join(bin, "app"); const executable = join(bin, "app");
@@ -77,7 +77,7 @@ it.skipIf(process.platform === "win32")(
expect(batches).toBe(2); expect(batches).toBe(2);
expect(result.library_packages).toEqual(["owner"]); expect(result.library_packages).toEqual(["owner"]);
expect(result.unpackaged_libraries).toContain(paths[1]); expect(result.unpackaged_libraries).toContain(paths[1]);
expect(result.unpackaged_libraries).not.toContain(paths[200]); expect(result.unpackaged_libraries).not.toContain(paths[101]);
expect(result.library_packages_reason).toContain("dpkg-query"); expect(result.library_packages_reason).toContain("dpkg-query");
}, },
); );
+2 -190
View File
@@ -1,20 +1,10 @@
import { import { chmod, mkdir, stat, symlink, writeFile } from "node:fs/promises";
chmod,
mkdir,
realpath,
stat,
symlink,
writeFile,
} from "node:fs/promises";
import { join } from "node:path"; import { join } from "node:path";
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { inspectLinuxApplication } from "./LinuxApplicationInventory.js"; import { inspectLinuxApplication } from "./LinuxApplicationInventory.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { ArtifactFileCleanupFailure } from "../artifacts/ArtifactFileOperation.js";
import { desktopOperationFailure } from "../application/desktop/DesktopCaptureRefusals.js";
const fixture = async () => { const fixture = async () => {
const root = await createTestTempDirectory("desktop-inventory-"); const root = await createTestTempDirectory("desktop-inventory-");
@@ -47,58 +37,6 @@ const fixture = async () => {
}; };
}; };
describe.skipIf(process.platform === "win32")(
"explicit executable traversal",
() => {
it("refuses a directory symlink as an executable while accepting a file symlink", async () => {
const { home, bin, environment } = await fixture();
const directoryAlias = join(bin, "directory-alias");
const fileAlias = join(bin, "file-alias");
await symlink(home, directoryAlias);
await symlink(join(bin, "notes-app"), fileAlias);
const inspect = (executable: string) =>
inspectLinuxApplication({
desktopEntry: undefined,
executable,
arguments: [],
environment,
home,
callerDirectory: home,
});
expect(await inspect(fileAlias)).toMatchObject({ ok: true });
expect(await inspect(directoryAlias)).toEqual({
ok: false,
error: { reason: "executable_not_found", program: directoryAlias },
});
});
it("preserves symlink and parent traversal for the selected executable", async () => {
const { home, environment } = await fixture();
const physical = join(home, "physical");
const named = join(home, "named");
await mkdir(join(physical, "child"), { recursive: true });
await mkdir(named);
await symlink(join(physical, "child"), join(named, "link"));
await writeFile(join(physical, "app"), "#!/bin/sh\n", { mode: 0o755 });
await writeFile(join(named, "app"), "#!/bin/sh\n", { mode: 0o755 });
const selected = `${named}/link/../app`;
const result = await inspectLinuxApplication({
desktopEntry: undefined,
executable: selected,
arguments: [],
environment,
home,
callerDirectory: home,
});
if (!result.ok) throw new Error(result.error.reason);
expect(result.value.inventory.argv[0]).toBe(selected);
expect(result.value.inventory.executable.resolved_path).toBe(
await realpath(join(physical, "app")),
);
});
},
);
describe.skipIf(process.platform === "win32")("inspectLinuxApplication", () => { describe.skipIf(process.platform === "win32")("inspectLinuxApplication", () => {
it("resolves a desktop entry id into a launch command and state directories", async () => { it("resolves a desktop entry id into a launch command and state directories", async () => {
const { home, bin, environment } = await fixture(); const { home, bin, environment } = await fixture();
@@ -788,129 +726,3 @@ describe.skipIf(process.platform === "win32")(
}); });
}, },
); );
it.skipIf(process.platform === "win32").each([
{ stage: "desktop entry", closeAt: 1, oversized: false, cancel: false },
{
stage: "desktop entry capacity failure",
closeAt: 1,
oversized: true,
cancel: false,
},
{ stage: "executable header", closeAt: 2, oversized: false, cancel: false },
{
stage: "optional Electron probe",
closeAt: 3,
oversized: false,
cancel: false,
},
{
stage: "desktop entry final cancellation",
closeAt: 1,
oversized: false,
cancel: true,
},
{
stage: "capacity failure before final cancellation",
closeAt: 1,
oversized: true,
cancel: true,
},
])(
"retains and retries a failed close during $stage",
async ({ closeAt, oversized, cancel }) => {
const { home, bin, environment } = await fixture();
const entry = join(
environment.XDG_DATA_DIRS,
"applications/org.example.notes.desktop",
);
if (oversized) await writeFile(entry, "x".repeat(1024 * 1024 + 1));
const controller = new AbortController();
const reason = new Error("cancelled during close");
const denied = new Error("descriptor close denied");
const originalClose = OwnedFileHandle.prototype.close;
let closes = 0;
let retained: OwnedFileHandle | undefined;
const close = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockImplementation(function (this: OwnedFileHandle) {
closes += 1;
if (closes === closeAt || this === retained) {
// oxlint-disable-next-line typescript/no-this-alias -- Retain the exact failed descriptor owner for retry.
retained = this;
if (cancel) controller.abort(reason);
return Promise.reject(denied);
}
return originalClose.call(this);
});
try {
const failure = await inspectLinuxApplication(
{
desktopEntry: "org.example.notes",
executable: undefined,
arguments: [],
environment,
home,
callerDirectory: home,
},
controller.signal,
{
runQuery: async () => "missing",
apt: {
trusted: join(home, "trusted"),
trustedParts: join(home, "trusted.d"),
lists: join(home, "lists"),
},
},
).catch((cause: unknown) => cause);
expect(retained).toBeDefined();
if (!(failure instanceof ArtifactFileCleanupFailure))
throw new Error("expected transferable descriptor cleanup", {
cause: failure,
});
const expectedPath =
closeAt === 1 ? entry : await realpath(join(bin, "notes-app"));
expect(failure.cleanup?.resources).toEqual([expectedPath]);
expect(failure.fileCleanupFailure.owner).toBe(retained);
expect(failure.fileCleanupFailure.cleanupCause).toBe(denied);
const outcome = failure.fileCleanupFailure.outcome;
if (oversized) {
expect(outcome).toMatchObject({
kind: "failed",
cause: {
maximumBytes: 1024 * 1024,
observedBytes: BigInt(1024 * 1024 + 1),
},
});
} else if (closeAt === 3) {
expect(outcome).toMatchObject({
kind: "failed",
cause: { kind: "unsupported" },
});
} else {
expect(outcome.kind).toBe("completed");
}
if (cancel && !oversized)
expect(failure.cancellation?.reason).toBe(reason);
else expect(failure.reason).not.toBe("cancelled");
const projected = desktopOperationFailure(
failure,
controller.signal,
"capture_desktop_application",
"Desktop capture",
);
expect(projected.cleanupIncomplete).toBe(true);
expect(projected.cleanupResources).toEqual([expectedPath]);
await expect(failure.cleanupLease.release()).rejects.toBe(denied);
expect(projected.cleanupIncomplete).toBe(true);
close.mockRestore();
await failure.cleanupLease.release();
expect(projected.cleanupIncomplete).toBe(false);
expect(projected.cleanupResources).toEqual([]);
expect(retained?.handle.fd).toBe(-1);
} finally {
close.mockRestore();
await retained?.close();
}
},
);
+5 -16
View File
@@ -3,7 +3,7 @@ import {
type DesktopLibraryPolicyInput, type DesktopLibraryPolicyInput,
} from "../domain/desktop/libraryPolicy.js"; } from "../domain/desktop/libraryPolicy.js";
import { lstat, realpath, stat } from "node:fs/promises"; import { lstat, realpath, stat } from "node:fs/promises";
import { basename, dirname, isAbsolute, join } from "node:path"; import { basename, dirname, isAbsolute, join, resolve } from "node:path";
import type { DesktopApplicationInventory } from "../domain/desktop/desktopCapture.js"; import type { DesktopApplicationInventory } from "../domain/desktop/desktopCapture.js";
import { import {
@@ -19,12 +19,7 @@ import {
} from "../domain/desktop/linuxPackages.js"; } from "../domain/desktop/linuxPackages.js";
import { err, ok, type Result } from "../domain/result.js"; import { err, ok, type Result } from "../domain/result.js";
import { readElectronFuseImage } from "../electron/ElectronFuseReader.js"; import { readElectronFuseImage } from "../electron/ElectronFuseReader.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js"; import { withOwnedRegularFile } from "../filesystem/OwnedRegularFile.js";
import {
RegularFileCleanupFailure,
withOwnedRegularFile,
} from "../filesystem/OwnedRegularFile.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { import {
RegularFileCapacityError, RegularFileCapacityError,
readStableFileBytes, readStableFileBytes,
@@ -38,7 +33,7 @@ import {
type DebianPackageHost, type DebianPackageHost,
type InventoryContext, type InventoryContext,
} from "./DebianPackageInventory.js"; } from "./DebianPackageInventory.js";
import { hostCloseFailure, isRefusedRead } from "./HostFileRead.js"; import { isRefusedRead } from "./HostFileRead.js";
import { runHostQuery } from "./HostQuery.js"; import { runHostQuery } from "./HostQuery.js";
import { resolveProgram } from "./ProgramLookup.js"; import { resolveProgram } from "./ProgramLookup.js";
@@ -235,8 +230,6 @@ const readDesktopEntry = async (
) )
).toString("utf8"); ).toString("utf8");
} catch (cause: unknown) { } catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure)
throw hostCloseFailure(cause);
signal?.throwIfAborted(); signal?.throwIfAborted();
if (!isRefusedRead(cause)) throw cause; if (!isRefusedRead(cause)) throw cause;
return err({ return err({
@@ -274,8 +267,6 @@ const readHeader = async (
(await readStableFilePrefix(handle, path, 16, options)).bytes, (await readStableFilePrefix(handle, path, 16, options)).bytes,
); );
} catch (cause: unknown) { } catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure)
throw hostCloseFailure(cause);
signal?.throwIfAborted(); signal?.throwIfAborted();
if (!isRefusedRead(cause)) throw cause; if (!isRefusedRead(cause)) throw cause;
return new Uint8Array(); return new Uint8Array();
@@ -337,9 +328,7 @@ const electronMarkers = async (
); );
if (image.slices.some((slice) => slice.sentinel_count > 0)) if (image.slices.some((slice) => slice.sentinel_count > 0))
markers.push("fuse-sentinel"); markers.push("fuse-sentinel");
} catch (cause: unknown) { } catch {
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined)
throw cause;
signal?.throwIfAborted(); signal?.throwIfAborted();
// Scripts and other non-binary launchers carry no fuse wire. // Scripts and other non-binary launchers carry no fuse wire.
} }
@@ -369,7 +358,7 @@ const launchCommand = async (
return ok({ return ok({
argv: [ argv: [
executable.includes("/") executable.includes("/")
? absoluteCallerPath(executable, { cwd: request.callerDirectory }) ? resolve(request.callerDirectory, executable)
: executable, : executable,
...request.arguments, ...request.arguments,
], ],
+2 -2
View File
@@ -1,11 +1,11 @@
import { constants } from "node:fs"; import { constants } from "node:fs";
import { access, stat } from "node:fs/promises"; import { access, lstat } from "node:fs/promises";
import { delimiter, isAbsolute } from "node:path"; import { delimiter, isAbsolute } from "node:path";
const isExecutable = async (path: string): Promise<boolean> => { const isExecutable = async (path: string): Promise<boolean> => {
try { try {
await access(path, constants.X_OK); await access(path, constants.X_OK);
return (await stat(path)).isFile(); return (await lstat(path)).isDirectory() === false;
} catch { } catch {
return false; return false;
} }
+2 -2
View File
@@ -218,9 +218,9 @@ def _cell(value, encoding: str):
return _sqlite_text(data, encoding) if value[0] == "t" else {"blob_hex": data.hex()} return _sqlite_text(data, encoding) if value[0] == "t" else {"blob_hex": data.hex()}
if isinstance(value, int) and abs(value) > MAX_SAFE_INTEGER: if isinstance(value, int) and abs(value) > MAX_SAFE_INTEGER:
return {"integer": str(value)} return {"integer": str(value)}
# Keep JSON's missing REAL values distinct from literal TEXT spellings. # JSON has no infinities or negative zero; REA's SQLite reader spells them as strings.
if isinstance(value, float) and (math.isinf(value) or (value == 0 and math.copysign(1, value) < 0)): if isinstance(value, float) and (math.isinf(value) or (value == 0 and math.copysign(1, value) < 0)):
return {"real": "Infinity" if value == math.inf else "-Infinity" if value == -math.inf else "-0"} return "Infinity" if value == math.inf else "-Infinity" if value == -math.inf else "-0"
return value return value
+2 -32
View File
@@ -118,9 +118,7 @@ describe("withheld files", () => {
expect(gapped(4, { truncated: true })).toMatchObject({ ok: true }); expect(gapped(4, { truncated: true })).toMatchObject({ ok: true });
expect(gapped(1, { unconfirmed_changes: 1 })).toMatchObject({ ok: true }); expect(gapped(1, { unconfirmed_changes: 1 })).toMatchObject({ ok: true });
}); });
});
describe("withheld file settings", () => {
it("names each withheld file's setting, environment variable and handle", () => { it("names each withheld file's setting, environment variable and handle", () => {
expect(withheldSettings("typora", [".config/Typora/mAxvdiPNmi"])).toEqual([ expect(withheldSettings("typora", [".config/Typora/mAxvdiPNmi"])).toEqual([
{ {
@@ -142,24 +140,8 @@ describe("withheld file settings", () => {
expect(other?.variable).not.toBe(again?.variable); expect(other?.variable).not.toBe(again?.variable);
}); });
it("names settings Python and Pydantic accept, distinct whatever the files' order", () => { it("names settings Python takes as arguments, distinct whatever the files' order", () => {
const files = [ const files = [".pass", "1license", "file_1license", "class"];
".pass",
"1license",
"file_1license",
"class",
"model_config",
"model_post_init",
"model_dump",
"model_dump_json",
"model_validate",
"model_validate_json",
"model_validate_strings",
"file_model_config",
"model_id",
"model_copy",
"model_fields",
];
for (const order of [files, [...files].reverse()]) { for (const order of [files, [...files].reverse()]) {
const settings = withheldSettings("an-app", order).map( const settings = withheldSettings("an-app", order).map(
({ setting }) => setting, ({ setting }) => setting,
@@ -168,19 +150,7 @@ describe("withheld file settings", () => {
for (const setting of settings) { for (const setting of settings) {
expect(setting).toMatch(/^[a-z]\w*$/u); expect(setting).toMatch(/^[a-z]\w*$/u);
expect(["pass", "class"]).not.toContain(setting); expect(["pass", "class"]).not.toContain(setting);
expect([
"model_config",
"model_post_init",
"model_dump",
"model_dump_json",
"model_validate",
"model_validate_json",
"model_validate_strings",
]).not.toContain(setting);
} }
expect(settings[order.indexOf("model_id")]).toBe("model_id");
expect(settings[order.indexOf("model_copy")]).toBe("model_copy");
expect(settings[order.indexOf("model_fields")]).toBe("model_fields");
} }
}); });
+1 -14
View File
@@ -169,23 +169,10 @@ const PYTHON_KEYWORDS = new Set(
), ),
); );
/** Pydantic >=2.12: configuration, its instance initialization hook and protected inherited methods. */
const PYDANTIC_RESERVED_FIELDS = new Set([
"model_config",
"model_post_init",
"model_dump",
"model_dump_json",
"model_validate",
"model_validate_json",
"model_validate_strings",
]);
/** A setting Python takes as an argument's name, distinct from those `settings` holds, which it joins. */ /** A setting Python takes as an argument's name, distinct from those `settings` holds, which it joins. */
const distinctSetting = (named: string, settings: Set<string>): string => { const distinctSetting = (named: string, settings: Set<string>): string => {
const valid = const valid =
/^[a-z]/u.test(named) && /^[a-z]/u.test(named) && !PYTHON_KEYWORDS.has(named)
!PYTHON_KEYWORDS.has(named) &&
!PYDANTIC_RESERVED_FIELDS.has(named)
? named ? named
: `file_${named}`; : `file_${named}`;
let setting = valid; let setting = valid;
-15
View File
@@ -5,25 +5,10 @@ import {
checkpointTaken, checkpointTaken,
desktopApplicationInventorySchema, desktopApplicationInventorySchema,
desktopCaptureEventSchema, desktopCaptureEventSchema,
desktopFileFactsSchema,
type DesktopCheckpoint, type DesktopCheckpoint,
type DesktopStateChange, type DesktopStateChange,
} from "./desktopCapture.js"; } from "./desktopCapture.js";
it.each(["-315619200000000000", "0", "1"])(
"retains filesystem modification timestamp %s without changing its spelling",
(mtime_ns) => {
const value = {
type: "file",
size: 3,
mode: 0o644,
mtime_ns,
sha256: null,
};
expect(desktopFileFactsSchema.parse(value)).toEqual(value);
},
);
describe("checkpointNameRefusal", () => { describe("checkpointNameRefusal", () => {
it("accepts an omitted name and a new one", () => { it("accepts an omitted name and a new one", () => {
expect(checkpointNameRefusal(undefined, new Set(["saved"]))).toBe( expect(checkpointNameRefusal(undefined, new Set(["saved"]))).toBe(
+2 -5
View File
@@ -11,7 +11,7 @@ export const desktopFileFactsSchema = z.strictObject({
type: z.enum(["file", "directory", "symlink", "other"]), type: z.enum(["file", "directory", "symlink", "other"]),
size: z.int().min(0), size: z.int().min(0),
mode: z.int().min(0), mode: z.int().min(0),
mtime_ns: z.string().regex(/^-?\d+$/u), mtime_ns: z.string().regex(/^\d+$/u),
sha256: sha256.nullable(), sha256: sha256.nullable(),
}); });
export type DesktopFileFacts = z.infer<typeof desktopFileFactsSchema>; export type DesktopFileFacts = z.infer<typeof desktopFileFactsSchema>;
@@ -271,10 +271,7 @@ export const desktopApplicationInventorySchema = z.strictObject({
state_directories: z.array( state_directories: z.array(
z.strictObject({ z.strictObject({
path: z.string().min(1), path: z.string().min(1),
/** Null when the filesystem probe could not establish presence or absence. */ exists_at_start: z.boolean(),
exists_at_start: z.boolean().nullable(),
/** The failed existence probe's reason; absent for confirmed presence or absence. */
existence_error: z.string().optional(),
}), }),
), ),
}); });
+13 -25
View File
@@ -22,6 +22,7 @@ const VALUE_ESCAPES: Readonly<Record<string, string>> = {
t: "\t", t: "\t",
r: "\r", r: "\r",
"\\": "\\", "\\": "\\",
";": "\\;",
}; };
/** Undo the escapes the desktop entry format allows in string values. */ /** Undo the escapes the desktop entry format allows in string values. */
@@ -68,25 +69,13 @@ export const escapeDesktopEntryValue = (value: string): string => {
return escaped; return escaped;
}; };
/** Decode list separators and escapes together, before either loses its meaning. */ const listValue = (value: string | undefined): readonly string[] =>
const listValue = (value: string | undefined): readonly string[] => { value === undefined
if (value === undefined) return []; ? []
const entries: string[] = []; : value
let entry = ""; .split(/(?<!\\);/u)
for (let index = 0; index < value.length; index += 1) { .map((part) => part.replaceAll("\\;", ";"))
const character = value.charAt(index); .filter((part) => part.length > 0);
const next = value.charAt(index + 1);
if (character === "\\" && next.length > 0) {
entry += next === ";" ? ";" : (VALUE_ESCAPES[next] ?? `\\${next}`);
index += 1;
} else if (character === ";") {
entries.push(entry);
entry = "";
} else entry += character;
}
if (entry.length > 0) entries.push(entry);
return entries;
};
/** Read the main group of a freedesktop desktop entry, ignoring localized keys. */ /** Read the main group of a freedesktop desktop entry, ignoring localized keys. */
export const parseDesktopEntry = ( export const parseDesktopEntry = (
@@ -112,14 +101,13 @@ export const parseDesktopEntry = (
// Keep the existing enum/boolean token treatment before value unescaping. // Keep the existing enum/boolean token treatment before value unescaping.
values.set( values.set(
key, key,
key === "Type" || key === "Terminal" ? value.trimEnd() : value, unescapeValue(
key === "Type" || key === "Terminal" ? value.trimEnd() : value,
),
); );
} }
if (!found) return err("missing_desktop_entry_group"); if (!found) return err("missing_desktop_entry_group");
const field = (key: string) => { const field = (key: string) => values.get(key) ?? null;
const value = values.get(key);
return value === undefined ? null : unescapeValue(value);
};
return ok({ return ok({
type: field("Type"), type: field("Type"),
name: field("Name"), name: field("Name"),
@@ -130,7 +118,7 @@ export const parseDesktopEntry = (
startup_wm_class: field("StartupWMClass"), startup_wm_class: field("StartupWMClass"),
categories: listValue(values.get("Categories")), categories: listValue(values.get("Categories")),
mime_types: listValue(values.get("MimeType")), mime_types: listValue(values.get("MimeType")),
terminal: field("Terminal") === "true", terminal: values.get("Terminal") === "true",
}); });
}; };
@@ -82,16 +82,3 @@ it("does not broaden Type or Terminal recognition by trimming decoded escapes",
if (!parsed.ok) throw new Error(parsed.error); if (!parsed.ok) throw new Error(parsed.error);
expect(parsed.value).toMatchObject({ type: "Application ", terminal: false }); expect(parsed.value).toMatchObject({ type: "Application ", terminal: false });
}); });
it.each([
[String.raw`First\\;Second;`, ["First\\", "Second"]],
[String.raw`First\;Second;`, ["First;Second"]],
[String.raw`First\\\;Second;`, ["First\\;Second"]],
["First;;", ["First", ""]],
["First;", ["First"]],
["", []],
])("decodes string-list escapes and terminators once: %s", (wire, expected) => {
const parsed = parseDesktopEntry(`[Desktop Entry]\nCategories=${wire}\n`);
if (!parsed.ok) throw new Error(parsed.error);
expect(parsed.value.categories).toEqual(expected);
});
-37
View File
@@ -2,7 +2,6 @@ import { describe, expect, it } from "vitest";
import { import {
isAppImageHeader, isAppImageHeader,
ownershipCandidates,
packageFromPath, packageFromPath,
parseAptPolicyOrigins, parseAptPolicyOrigins,
parseAptPolicyPackageSources, parseAptPolicyPackageSources,
@@ -12,42 +11,6 @@ import {
} from "./linuxPackages.js"; } from "./linuxPackages.js";
describe("dpkg output", () => { describe("dpkg output", () => {
it("adds merged-/usr ownership candidates only for merged system directories", () => {
const paths = [
"/usr/bin/editor",
"/sbin/helper",
"/usr/lib/x86_64-linux-gnu/libexample.so",
"/lib64/ld-linux-x86-64.so.2",
"/usr/lib32/libexample.so",
"/libx32/libexample.so",
"/usr/libo32/libexample.so",
"/opt/vendor/app",
"/home/ada/bin/app",
"/usr/share/applications/app.desktop",
"/usr/local/bin/app",
];
expect(ownershipCandidates(paths)).toEqual([
"/usr/bin/editor",
"/bin/editor",
"/sbin/helper",
"/usr/sbin/helper",
"/usr/lib/x86_64-linux-gnu/libexample.so",
"/lib/x86_64-linux-gnu/libexample.so",
"/lib64/ld-linux-x86-64.so.2",
"/usr/lib64/ld-linux-x86-64.so.2",
"/usr/lib32/libexample.so",
"/lib32/libexample.so",
"/libx32/libexample.so",
"/usr/libx32/libexample.so",
"/usr/libo32/libexample.so",
"/libo32/libexample.so",
"/opt/vendor/app",
"/home/ada/bin/app",
"/usr/share/applications/app.desktop",
"/usr/local/bin/app",
]);
});
it("reads every owner of a path and skips diversions", () => { it("reads every owner of a path and skips diversions", () => {
expect( expect(
parseDpkgOwners( parseDpkgOwners(
+4 -8
View File
@@ -26,18 +26,14 @@ export const parseDpkgOwners = (
return [...owners]; return [...owners];
}; };
/** Only merged system directories have alternate `/usr` spellings; application, home and shared-data paths retain their own identity. */ /** `/bin/x` and `/usr/bin/x` name the same file on merged-/usr systems, and dpkg may know either. */
export const ownershipCandidates = ( export const ownershipCandidates = (
paths: readonly string[], paths: readonly string[],
): readonly string[] => [ ): readonly string[] => [
...new Set( ...new Set(
paths.flatMap((path) => { paths.flatMap((path) =>
if (!/^\/(?:usr\/)?(?:bin|sbin|lib(?:32|64|o32|x32)?)\//u.test(path)) path.startsWith("/usr/") ? [path, path.slice(4)] : [path, `/usr${path}`],
return [path]; ),
return path.startsWith("/usr/")
? [path, path.slice(4)]
: [path, `/usr${path}`];
}),
), ),
]; ];
-39
View File
@@ -1,39 +0,0 @@
import { lstat, realpath, stat } from "node:fs/promises";
import { basename, dirname, join } from "node:path";
import { absoluteCallerPath } from "./callerPath.js";
/** Resolve existing path components without changing the caller's traversal. */
export const canonicalCallerPath = async (path: string): Promise<string> => {
const remaining: string[] = [];
let current = absoluteCallerPath(path);
for (;;) {
try {
const found = await stat(current);
if (remaining.length > 0 && !found.isDirectory())
throw Object.assign(
new Error(`Path parent is not a directory: ${current}`),
{ code: "ENOTDIR" },
);
return join(await realpath(current), ...remaining);
} catch (cause: unknown) {
if ((cause as NodeJS.ErrnoException).code !== "ENOENT") throw cause;
// A dangling symlink is not an absent destination: its target cannot be
// established. A missing component followed by .. is likewise unknown.
const existing = await lstat(current).then(
() => true,
(error: unknown) => {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return false;
throw error;
},
);
if (
existing ||
basename(current) === ".." ||
dirname(current) === current
)
throw cause;
remaining.unshift(basename(current));
current = dirname(current);
}
}
};
+54
View File
@@ -407,6 +407,30 @@ it("charges long digits to the string budget and keeps large previews bounded",
); );
}); });
it("reads marshal one-byte string tags as Latin-1 with Unicode identity", () => {
for (const tag of [0x61, 0x41, 0x7a, 0x5a]) {
const encoded = Buffer.concat([
Buffer.from([tag]),
tag === 0x61 || tag === 0x41 ? u32(1) : Buffer.from([1]),
Buffer.from([0xe9]),
]);
const sameUnicode = marshalUnicode(Buffer.from("é", "utf8"));
const set = Buffer.concat([
Buffer.from([0x3c]),
u32(2),
encoded,
sameUnicode,
]);
const report = root(parse(withConsts([encoded, set])));
expect(report.consts[0]).toMatchObject({
kind: "str",
value: "é",
bytes: 1,
});
expect(report.consts[1]).toEqual({ kind: "set", length: 1 });
}
});
it("decodes marshal strings with CPython surrogatepass rules", () => { it("decodes marshal strings with CPython surrogatepass rules", () => {
const report = parse( const report = parse(
withConsts([marshalUnicode(Buffer.from("efbbbf73eda080efbbbf", "hex"))]), withConsts([marshalUnicode(Buffer.from("efbbbf73eda080efbbbf", "hex"))]),
@@ -1534,6 +1558,36 @@ it("hashes a long TYPE_REF tuple chain without recursing", () => {
expect(pythonPycSchema.safeParse(report).success).toBe(true); expect(pythonPycSchema.safeParse(report).success).toBe(true);
}); });
it("hashes shallow TYPE_REF code chains without using the JavaScript stack", () => {
const links = 3000;
const codes = Array.from({ length: links }, (_, index) =>
marshalCode("py311", {
flagged: true,
name: "f",
consts: index === 0 ? [Buffer.from([0x4e])] : [marshalRef(index - 1)],
}),
);
const bytes = Buffer.concat([
pythonPycHeader(3531),
marshalCode("py311", {
constsObject: Buffer.concat([
Buffer.from([0x28]),
u32(links + 1),
...codes,
Buffer.from([0x3c]),
u32(1),
marshalRef(links - 1),
]),
}),
]);
const report = parseWith(bytes, { max_objects: 65_536 });
expect(root(report).consts.at(-1)).toEqual({ kind: "set", length: 1 });
expect(report.coverage).toMatchObject({
status: "complete",
code_objects: links + 1,
});
});
it("rejects code CPython treats as corrupt instead of complete coverage", () => { it("rejects code CPython treats as corrupt instead of complete coverage", () => {
const file = ( const file = (
layout: "py37" | "py38" | "py311", layout: "py37" | "py38" | "py311",
+83 -17
View File
@@ -332,7 +332,7 @@ class MarshalReader {
readonly #codeKeys = new WeakMap<CodeNode, string>(); readonly #codeKeys = new WeakMap<CodeNode, string>();
readonly #codeIdentities = new WeakMap<CodeNode, number>(); readonly #codeIdentities = new WeakMap<CodeNode, number>();
readonly #codeEquality = new WeakMap<CodeNode, CodeEquality>(); readonly #codeEquality = new WeakMap<CodeNode, CodeEquality>();
/** Python hashability per code object. False while that check is in progress. */ /** Completed Python hashability checks per code object. */
readonly #codeHashable = new WeakMap<CodeNode, boolean>(); readonly #codeHashable = new WeakMap<CodeNode, boolean>();
/** Tuples whose items are still being read; CPython cannot hash them yet. */ /** Tuples whose items are still being read; CPython cannot hash them yet. */
readonly #readingTuples = new WeakSet<MarshalValue>(); readonly #readingTuples = new WeakSet<MarshalValue>();
@@ -436,13 +436,19 @@ class MarshalReader {
}; };
case TYPE_INTERNED: case TYPE_INTERNED:
case TYPE_UNICODE: case TYPE_UNICODE:
return this.utf8(this.nonNegativeInt32("Marshal string length"), false); return this.string(
this.nonNegativeInt32("Marshal string length"),
"utf8",
);
case TYPE_ASCII: case TYPE_ASCII:
case TYPE_ASCII_INTERNED: case TYPE_ASCII_INTERNED:
return this.utf8(this.nonNegativeInt32("Marshal string length"), true); return this.string(
this.nonNegativeInt32("Marshal string length"),
"latin1",
);
case TYPE_SHORT_ASCII: case TYPE_SHORT_ASCII:
case TYPE_SHORT_ASCII_INTERNED: case TYPE_SHORT_ASCII_INTERNED:
return this.utf8(this.u8(), true); return this.string(this.u8(), "latin1");
case TYPE_TUPLE: case TYPE_TUPLE:
return this.sequenceValue( return this.sequenceValue(
"tuple", "tuple",
@@ -1309,13 +1315,70 @@ class MarshalReader {
#isCodeHashable(node: CodeNode): boolean { #isCodeHashable(node: CodeNode): boolean {
const known = this.#codeHashable.get(node); const known = this.#codeHashable.get(node);
if (known !== undefined) return known; if (known !== undefined) return known;
this.#codeHashable.set(node, false); const frames: { value: MarshalValue; finish: boolean }[] = [
const material = { value: { kind: "code", value: node }, finish: false },
this.#codeEquality.get(node) ?? ];
malformed("Code object is missing equality material."); const gray = new Set<MarshalValue | CodeNode>();
const hashable = this.#equalityKey(material.constsTuple, "hash") !== null; const complete = new Set<MarshalValue>();
this.#codeHashable.set(node, hashable); const reject = (): false => {
return hashable; this.#codeHashable.set(node, false);
return false;
};
while (frames.length > 0) {
this.#takeKeyStep();
const frame = frames.pop()!;
const value = frame.value;
const identity = value.kind === "code" ? value.value : value;
if (frame.finish) {
gray.delete(identity);
if (value.kind === "code") {
const material =
this.#codeEquality.get(value.value) ??
malformed("Code object is missing equality material.");
// Child code checks are now complete, so the existing tuple key
// walker preserves its equality and expansion budgets without recursion.
const hashable =
this.#equalityKey(material.constsTuple, "hash") !== null;
this.#codeHashable.set(value.value, hashable);
if (!hashable) return reject();
} else complete.add(value);
continue;
}
if (value.kind === "code") {
const cached = this.#codeHashable.get(value.value);
if (cached === false) return reject();
if (cached === true) continue;
} else {
const cached = this.#keyCache.get(value);
if (cached === null) return reject();
if (cached !== undefined || complete.has(value)) continue;
}
if (
gray.has(identity) ||
this.#readingTuples.has(value) ||
value.kind === "list" ||
value.kind === "set" ||
value.kind === "dict" ||
value.kind === "null"
)
return reject();
const children =
value.kind === "code"
? [
(
this.#codeEquality.get(value.value) ??
malformed("Code object is missing equality material.")
).constsTuple,
]
: value.kind === "tuple" || value.kind === "frozenset"
? value.items
: [];
gray.add(identity);
frames.push({ value, finish: true });
for (let index = children.length - 1; index >= 0; index -= 1)
frames.push({ value: children[index]!, finish: false });
}
return true;
} }
#codeIdentity(node: CodeNode): number { #codeIdentity(node: CodeNode): number {
@@ -1492,17 +1555,20 @@ class MarshalReader {
); );
} }
utf8(length: number, ascii: boolean): MarshalValue { string(length: number, encoding: "latin1" | "utf8"): MarshalValue {
const bytes = this.blob(length); const bytes = this.blob(length);
if (ascii) { if (encoding === "latin1") {
if (bytes.some((byte) => byte > 0x7f)) // CPython reads the ASCII-tagged forms as one-byte Unicode, even when
malformed("Marshal ASCII string contains a non-ASCII byte."); // high bytes appear. Identity must agree with the UTF-8 string form.
const value = bytes.toString("latin1");
return { return {
kind: "str", kind: "str",
value: bytes.toString("latin1"), value,
previewEnd: Math.min(length, PYTHON_PYC_CONST_PREVIEW_LIMIT), previewEnd: Math.min(length, PYTHON_PYC_CONST_PREVIEW_LIMIT),
bytes: length, bytes: length,
raw: bytes, raw: bytes.some((byte) => byte > 0x7f)
? Buffer.from(value, "utf8")
: bytes,
}; };
} }
const decoded = decodeMarshalUtf8(bytes, PYTHON_PYC_CONST_PREVIEW_LIMIT); const decoded = decodeMarshalUtf8(bytes, PYTHON_PYC_CONST_PREVIEW_LIMIT);
@@ -763,7 +763,7 @@ print(json.dumps({"deep": deep, "peak": peak, "edge": _deeper_than(nested, 501),
expect(result.peak).toBeLessThan(1024 * 1024); expect(result.peak).toBeLessThan(1024 * 1024);
}); });
it("keeps special SQLite numbers distinct from text in exported table views", () => { it("spells infinite and negative-zero SQLite reals as REA's SQLite reader does", () => {
const imports = PYTHON_IMPORTS.split("\n") const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol")) .filter((line) => !line.includes("agentenv_protocol"))
.join("\n"); .join("\n");
@@ -772,33 +772,12 @@ it("keeps special SQLite numbers distinct from text in exported table views", ()
"-I", "-I",
"-B", "-B",
"-c", "-c",
`${imports}${PYTHON_HELPERS} `${imports}${PYTHON_HELPERS}\nprint(json.dumps([_cell(v, "utf-8") for v in (math.inf, -math.inf, -0.0, 0.0, 1.5)]))`,
with sqlite3.connect(":memory:") as db:
db.execute("create table cells(value)")
db.executemany("insert into cells values(?)", [(v,) for v in (math.inf, -math.inf, "Infinity", "-Infinity", "-0", 9007199254740993, "9007199254740993")])
print(json.dumps({"table": _table(db, "cells", [], "utf-8"), "reals": [_cell(v, "utf-8") for v in (-0.0, 0.0, 1.5)]}))
`,
], ],
{ encoding: "utf8" }, { encoding: "utf8" },
); );
expect(JSON.parse(cells)).toEqual({ expect(JSON.parse(cells)).toEqual(["Infinity", "-Infinity", "-0", 0, 1.5]);
table: {
columns: ["value"],
primary_key: [],
rows: [
[{ real: "Infinity" }],
[{ real: "-Infinity" }],
["Infinity"],
["-Infinity"],
["-0"],
[{ integer: "9007199254740993" }],
["9007199254740993"],
],
truncated: false,
},
reals: [{ real: "-0" }, 0, 1.5],
});
}); });
it("keeps the same keyfile whitespace as the retained state view", async () => { it("keeps the same keyfile whitespace as the retained state view", async () => {