mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
fix(workspaces): resume archived imports from their saved upload (#19396)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Durable workspaces let a replacement web server restore saved task files. > - Legacy workspace imports keep a journal and an encrypted upload receipt across interruption. > - An archived Git import can rebuild a missing source from its saved task commit and edits. > - A fresh clone can also contain newer remote refs. Those refs change the archive manifest even when the task files are unchanged. > - This pull request recovers the exact authenticated upload for an interrupted archived Git import. > - The original journal, source binding, task binding, and execution checks still control publication. ## Linked Issues or Issue Description **What happened?** An archived Git workspace import stopped after its upload persisted. A retry rebuilt the source from the remote repository. New remote refs changed the archive manifest, so the original upload receipt rejected the retry with `remote_workspace_snapshot_operation_content_changed`. **Expected behavior** The retry must recover its original saved upload and complete the same import. It must keep the saved task commit, staged edits, unstaged edits, lifecycle state, and task binding. A changed source or task binding must still block publication. **Steps to reproduce** 1. Archive a Git task with staged and unstaged edits, save its restoration recipe, and remove its local repository. 2. Start the workspace import and lose the upload reply after object persistence, or interrupt its independent restore after the upload is acknowledged. 3. Advance the remote default branch and add an unrelated branch. 4. Retry the original import. Before this change, both interruption cases fail with the same content-change error. **Paperclip version or commit** Reproduced on master at `d8b0eae68772a3bd2bb4a8163b5ce95764b97cff`. **Deployment mode** Cloud workspace migration with external PostgreSQL and durable encrypted object storage. The regression tests use real disposable PostgreSQL, an HTTP Git remote, and encrypted filesystem-backed object storage. Related merged work: #18826 and #19138. No duplicate open fix was found. ## What Changed - Add scoped recovery of an acknowledged snapshot or a pending upload whose object persisted. Authenticate the archive before returning it. Recovery alone does not publish the snapshot. - Resume archived Git imports from those exact saved bytes. Keep the original operation and all publication checks. - Lock and recheck each saved nested source’s company, project, and repository identity in the final publication transaction. - Keep mutable local sources on the existing capture and content-comparison path. - Add regressions for both interruption points, unrelated remote changes, source revocation during recovery, and local edits after interruption. - Document the retry behavior. ## Verification - Before the repair: both new archived retry regressions fail with `remote_workspace_snapshot_operation_content_changed`; the mutable-source control passes. - After the repair: all 204 tests in the migration, snapshot storage, archive, and archive environment suites pass. The suites use real PostgreSQL and encrypted storage. - Server package typecheck and full repository typecheck pass on `bffebbab60edbdf50b42fc37a9e58beb01ceb94d`. - An added source-revocation test initially expected a later expiry error. The existing publication fingerprint rejected the change first. The test now requires that earlier `remote_workspace_migration_publication_changed` error and still checks that authority was not published. - The review finding was reproduced before the safety repair: changing a saved nested repository URL after upload still allowed publication. The new regression requires rejection, and unchanged approved nested sources still restore. - Full repository typecheck and build pass on the final head. The full local test command stopped at one Slack manager authorization fixture with `socket hang up`; the exact case then passed in isolation on the same head, and all three CI chat shards passed. The original local failure is retained. - The first CI server shard run had 4,724 passes and one runtime-process readiness failure in an unchanged fixture. The exact fixture passed in isolation. One unchanged rerun of that shard passed. The first failure remains recorded in the workflow history. - The earlier candidate’s full local test run was stopped after the confirmed review finding; that interrupted run is not reported as a pass. - All CI gates, including `ci / verify` and `ci / e2e`, pass on `bffebbab60edbdf50b42fc37a9e58beb01ceb94d`. Greptile is 5/5 on that head, with no remaining findings or unresolved review threads. ## Risks Recovery applies to archived Git sources, whose saved recipe and lifecycle checks still run before publication. A source change during recovery must keep the snapshot unpublished. Missing or corrupt saved objects still fail authentication or retain the existing import obligation. Mutable local sources still reject changed contents. This change adds no database migration and changes no saved archive format. ## Model Used OpenAI GPT-6 in Codex, with code editing, tool use, and test execution. The exact runtime model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <[email protected]>
This commit is contained in:
1 parent
5c185db77c
commit
da34bcd48d
6 files changed
+151
-21
No files matched your search
@@ -1930,6 +1930,15 @@ corrupt, or unbound sources block the import. Publication still requires an
|
||||
independent download and restore of the new encrypted snapshot. The old folder
|
||||
remains absent, and the importer does not change the original restore point.
|
||||
|
||||
An interrupted archived Git import reuses its original uploaded archive after
|
||||
authentication, including when the upload reply was lost. It does not rebuild
|
||||
that archive from today's remote branch pointers. The original import journal,
|
||||
restoration recipe, task binding, and execution fences must still match before
|
||||
publication. Saved nested sources must still belong to the same company and
|
||||
project with the same repository URL; publication locks and rechecks those source
|
||||
rows. Imports from mutable local folders still recapture and compare the
|
||||
current files; a saved upload does not permit later local edits to be discarded.
|
||||
|
||||
Shared physical folders require Cloud's offline enrollment maintenance process.
|
||||
A task-level hold cannot fence another workspace record pointing at the same
|
||||
folder. Maintenance verifies that legacy deployments have stopped and that no
|
||||
|
||||
@@ -9,7 +9,7 @@ import { PROJECT_REPOSITORIES_DIR } from "@paperclipai/adapter-utils/git-workspa
|
||||
import type { StorageService } from "../storage/types.js";
|
||||
import { createIssueGitRemoteAuthProvider } from "./git-credentials.js";
|
||||
import { applyGitRestorePointToWorktree, ensureGitRestorePointCommit, loadGitRestorePoint } from "./git-restore-point-storage.js";
|
||||
import { restoreGitWorkspaceContents } from "./git-workspace-contents.js";
|
||||
import { restoreGitWorkspaceContents, type ManagedCheckoutSource } from "./git-workspace-contents.js";
|
||||
import { ensureSavedGitCommit, gitRestorePointExpired, managedSourcePaths, readGitCacheRestoration,
|
||||
verifiedGitCacheSource, verifiedManagedGitObjectSource, type GitCacheRestoration } from "./git-workspace-restoration.js";
|
||||
import { localRepositoryPath } from "./local-repository-path.js";
|
||||
@@ -164,6 +164,22 @@ export async function inspectArchivedGitMigrationSource(db: Pick<Db, "select">,
|
||||
|
||||
/** Restore only into private migration scratch. Original repositories, task
|
||||
* folders, lifecycle state and restore-point metadata remain unchanged. */
|
||||
async function authorizeNestedSource(db: Pick<Db, "select">, workspace: Workspace, saved: ManagedCheckoutSource, lock = false) {
|
||||
if (!workspace.projectId) throw unavailable();
|
||||
const query = db.select({ repoUrl: projectWorkspaces.repoUrl }).from(projectWorkspaces).where(and(
|
||||
eq(projectWorkspaces.id, saved.id), eq(projectWorkspaces.companyId, workspace.companyId),
|
||||
eq(projectWorkspaces.projectId, workspace.projectId),
|
||||
));
|
||||
const [row] = await (lock ? query.for("share", { noWait: true }) : query);
|
||||
if (!row || row.repoUrl !== saved.repoUrl) throw unavailable();
|
||||
}
|
||||
|
||||
export async function lockArchivedGitMigrationSources(db: Pick<Db, "select">, workspace: Workspace, source: ArchivedGitMigrationSource) {
|
||||
for (const checkout of [...(source.recipe?.contents?.checkouts ?? [])].sort((a, b) => a.source.id.localeCompare(b.source.id))) {
|
||||
await authorizeNestedSource(db, workspace, checkout.source, true);
|
||||
}
|
||||
}
|
||||
|
||||
export async function materializeArchivedGitMigrationSource(input: {
|
||||
db: Db; storage: StorageService; workspace: Workspace; source: ArchivedGitMigrationSource; scratch: string;
|
||||
}) {
|
||||
@@ -217,10 +233,7 @@ export async function materializeArchivedGitMigrationSource(input: {
|
||||
await restoreGitWorkspaceContents({ cwd, companyId: workspace.companyId, workspaceId: workspace.id,
|
||||
projectId: workspace.projectId, contents: source.recipe?.contents, storage: input.storage, resolveGitAuth,
|
||||
authorizeSource: async saved => {
|
||||
if (!workspace.projectId) throw unavailable();
|
||||
const [row] = await input.db.select().from(projectWorkspaces).where(and(eq(projectWorkspaces.id, saved.id),
|
||||
eq(projectWorkspaces.companyId, workspace.companyId), eq(projectWorkspaces.projectId, workspace.projectId)));
|
||||
if (!row || row.repoUrl !== saved.repoUrl) throw unavailable();
|
||||
await authorizeNestedSource(input.db, workspace, saved);
|
||||
const local = localRepositoryPath(saved.repoUrl); if (local) approvedLocalSources.add(local);
|
||||
},
|
||||
prepareLocalSource: async local => {
|
||||
|
||||
@@ -660,8 +660,8 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
|
||||
const address = server.address(); if (!address || typeof address === "string") throw new Error("test server not bound");
|
||||
return { bare, repoUrl: `http://127.0.0.1:${address.port}/repository.git` };
|
||||
}
|
||||
it.each(["present", "pruned", "retry", "missing-base", "nested", "unapproved-nested", "missing", "expired", "corrupt"])("preserves archived Git edits or refuses an unavailable restore point (%s)", async state => {
|
||||
const remote = state === "missing-base" ? await serveRepository() : null;
|
||||
it.each(["present", "pruned", "retry", "missing-base", "missing-base-retry", "missing-base-uploaded-retry", "missing-base-changed-retry", "nested", "nested-retry", "nested-source-changed-retry", "unapproved-nested", "missing", "expired", "corrupt"])("preserves archived Git edits or refuses an unavailable restore point (%s)", async state => {
|
||||
const remote = state.startsWith("missing-base") ? await serveRepository() : null;
|
||||
const f = await gitArchiveFixture(remote?.repoUrl);
|
||||
if (remote) { await git(remote.bare, "fetch", f.base, "main:refs/heads/main"); await git(remote.bare, "update-server-info"); }
|
||||
await fs.writeFile(path.join(f.cwd, "saved.txt"), "staged edit\n"); await git(f.cwd, "add", "saved.txt");
|
||||
@@ -676,7 +676,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
|
||||
sha256: sealed.sha256, key: sealed.key, head: captured.manifest.head, index: captured.manifest.index,
|
||||
worktree: captured.manifest.worktree, ignored: captured.manifest.ignored, modes: captured.manifest.modes, headOnRemote: false };
|
||||
let contents;
|
||||
if (state === "nested" || state === "unapproved-nested") {
|
||||
if (state.startsWith("nested") || state === "unapproved-nested") {
|
||||
const sourceCwd = path.join(root, "nested-source");
|
||||
await fs.mkdir(sourceCwd); await git(sourceCwd, "init", "-q", "-b", "main");
|
||||
await fs.writeFile(path.join(sourceCwd, "nested.txt"), "nested source work\n");
|
||||
@@ -688,7 +688,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
|
||||
await fs.writeFile(path.join(nestedCwd, ".claude", "settings.local.json"), '{"permissions":{"allow":["Read"]}}');
|
||||
contents = await captureGitWorkspaceContents(f.cwd, { companyId: f.companyId, workspaceId: f.workspaceId },
|
||||
[await readManagedCheckoutContents(nestedCwd, name, source)]);
|
||||
if (state === "nested") await db.insert(projectWorkspaces).values({ ...source, companyId: f.companyId, projectId: f.projectId, name: "Nested source" });
|
||||
if (state !== "unapproved-nested") await db.insert(projectWorkspaces).values({ ...source, companyId: f.companyId, projectId: f.projectId, name: "Nested source" });
|
||||
}
|
||||
await db.update(executionWorkspaces).set({ status: "archived", closedAt: new Date(), metadata: {
|
||||
createdByRuntime: true, gitBranchOwnershipVersion: 1, gitCacheRestoration: { schema: "paperclip.git-cache-restoration.v2", companyId: f.companyId,
|
||||
@@ -697,7 +697,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
|
||||
gitBranchOwnershipVersion: 1, verifiedAt: new Date().toISOString(), restorePoint, contents },
|
||||
} }).where(eq(executionWorkspaces.id, f.workspaceId));
|
||||
await git(f.base, "worktree", "remove", "--force", f.cwd); await git(f.base, "branch", "-D", f.branchName);
|
||||
if (state === "missing-base") await fs.rm(f.base, { recursive: true });
|
||||
if (state.startsWith("missing-base")) await fs.rm(f.base, { recursive: true });
|
||||
if (state === "unapproved-nested") {
|
||||
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow();
|
||||
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
|
||||
@@ -725,12 +725,64 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
|
||||
expect(await fs.stat(f.cwd).catch(() => null)).toBeNull();
|
||||
return;
|
||||
}
|
||||
if (state === "retry") {
|
||||
if (["retry", "missing-base-retry", "missing-base-changed-retry", "nested-retry", "nested-source-changed-retry"].includes(state)) {
|
||||
const put = storage.putFile.bind(storage); let fail = true;
|
||||
storage.putFile = async input => { const result = await put(input); if (fail) { fail = false; throw new Error("upload reply lost"); } return result; };
|
||||
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("upload reply lost");
|
||||
}
|
||||
if (state === "missing-base-uploaded-retry") {
|
||||
const get = storage.getObject.bind(storage); let reads = 0;
|
||||
storage.getObject = async (...args) => {
|
||||
if (args[1].includes("/workspace-snapshots/") && ++reads === 2) throw new Error("restore reply lost");
|
||||
return get(...args);
|
||||
};
|
||||
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("restore reply lost");
|
||||
expect((await db.select().from(executionWorkspaceSnapshots))[0]?.status).toBe("uploaded");
|
||||
}
|
||||
const interrupted = state.startsWith("missing-base-") || state.startsWith("nested-")
|
||||
? (await db.select().from(workspaceOperations).where(eq(workspaceOperations.phase, "remote_workspace_migration")))[0] : null;
|
||||
if (interrupted && remote) {
|
||||
const tree = await git(remote!.bare, "rev-parse", "main^{tree}");
|
||||
const next = await git(remote!.bare, "commit-tree", tree, "-p", f.upstream, "-m", "unrelated remote release");
|
||||
await git(remote!.bare, "update-ref", "refs/heads/main", next);
|
||||
await git(remote!.bare, "update-ref", "refs/heads/new-unrelated-branch", next);
|
||||
await git(remote!.bare, "update-server-info");
|
||||
}
|
||||
if (state === "nested-source-changed-retry") {
|
||||
await db.update(projectWorkspaces).set({ repoUrl: "https://example.invalid/replaced-repository.git" })
|
||||
.where(eq(projectWorkspaces.id, contents!.checkouts[0]!.source.id));
|
||||
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true }))
|
||||
.rejects.toThrow("remote_workspace_migration_archived_git_source_unavailable");
|
||||
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
|
||||
expect((await workspace(f.workspaceId)).status).toBe("archived");
|
||||
return;
|
||||
}
|
||||
if (state === "missing-base-changed-retry") {
|
||||
const get = storage.getObject.bind(storage); let expire = true;
|
||||
storage.getObject = async (...args) => {
|
||||
const result = await get(...args);
|
||||
if (expire && args[1].includes("/workspace-snapshots/")) {
|
||||
expire = false;
|
||||
const row = await workspace(f.workspaceId);
|
||||
const recipe = row.metadata!.gitCacheRestoration as Record<string, unknown>;
|
||||
await db.update(executionWorkspaces).set({ metadata: { ...row.metadata, gitCacheRestoration: {
|
||||
...recipe, restorePoint: { ...restorePoint, expiredAt: new Date().toISOString() },
|
||||
} } }).where(eq(executionWorkspaces.id, f.workspaceId));
|
||||
}
|
||||
return result;
|
||||
};
|
||||
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("remote_workspace_migration_publication_changed");
|
||||
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
|
||||
expect((await workspace(f.workspaceId)).status).toBe("archived");
|
||||
expect((await db.select().from(executionWorkspaceSnapshots))[0]?.status).toBe("uploaded");
|
||||
return;
|
||||
}
|
||||
await migrateRemoteWorkspace({ db, storage, ...f, apply: true });
|
||||
if (interrupted) {
|
||||
const operations = await db.select().from(workspaceOperations).where(eq(workspaceOperations.phase, "remote_workspace_migration"));
|
||||
expect(operations).toHaveLength(1);
|
||||
expect(operations[0]).toMatchObject({ id: interrupted.id, status: "succeeded" });
|
||||
}
|
||||
const authority = readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)!;
|
||||
await fs.rm(f.base, { recursive: true, force: true }); await fs.rm(path.join(root, "nested-source"), { recursive: true, force: true }); await storage.deleteObject(f.companyId, objectKey);
|
||||
const ref = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! });
|
||||
@@ -738,7 +790,7 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
|
||||
await loadWorkspaceSnapshotArchive({ storage, reference: ref!, targetPath: archivePath });
|
||||
await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: ref!.manifestDigest });
|
||||
expect(await git(targetDir, "rev-parse", "HEAD")).toBe(f.commit);
|
||||
if (state === "nested") {
|
||||
if (state.startsWith("nested")) {
|
||||
expect(await fs.readFile(path.join(targetDir, ".paperclip-repositories", contents!.checkouts[0]!.name, "nested.txt"), "utf8")).toBe("nested source work\n");
|
||||
expect(JSON.parse(await fs.readFile(path.join(targetDir, ".paperclip-repositories", contents!.checkouts[0]!.name, ".claude/settings.local.json"), "utf8")))
|
||||
.toEqual({ permissions: { allow: ["Read"] } });
|
||||
@@ -901,6 +953,15 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
|
||||
expect(await migrateRemoteWorkspace({ db, storage, ...f, apply: true })).toMatchObject({ applied: true });
|
||||
const operations = await db.select().from(workspaceOperations); expect(operations).toHaveLength(1); expect(operations[0]?.id).toBe(before.operationId);
|
||||
});
|
||||
it("still rejects local source edits after an interrupted upload", async () => {
|
||||
const f = await seed(), put = storage.putFile.bind(storage); let fail = true;
|
||||
storage.putFile = async input => { const saved = await put(input); if (fail) { fail = false; throw new Error("upload reply lost"); } return saved; };
|
||||
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("upload reply lost");
|
||||
await fs.writeFile(path.join(f.cwd, "saved.txt"), "new local edits\n");
|
||||
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("remote_workspace_snapshot_operation_content_changed");
|
||||
expect(readRemoteWorkspaceAuthority((await workspace(f.workspaceId)).metadata)).toBeNull();
|
||||
expect(await fs.readFile(path.join(f.cwd, "saved.txt"), "utf8")).toBe("new local edits\n");
|
||||
});
|
||||
it("imports dirty unpushed Git work and restores it after the original folder disappears", async () => {
|
||||
const f = await seed(); await git(f.cwd, "init", "-q", "-b", "feature");
|
||||
await git(f.cwd, "add", "."); await git(f.cwd, "commit", "-qm", "unpushed work");
|
||||
|
||||
@@ -15,11 +15,11 @@ import type { StorageService } from "../storage/types.js";
|
||||
import { persistActivity, publishActivity } from "./activity-log.js";
|
||||
import { withExecutionWorkspaceLifecycleLock } from "./execution-workspace-lifecycle-lock.js";
|
||||
import { captureRemoteWorkspaceDirectory, restoreRemoteWorkspaceArchive } from "./remote-workspace-snapshot-archive.js";
|
||||
import { beginWorkspaceSnapshot, loadWorkspaceSnapshotArchive, uploadWorkspaceSnapshot } from "./remote-workspace-snapshot-storage.js";
|
||||
import { beginWorkspaceSnapshot, loadWorkspaceSnapshotArchive, recoverWorkspaceSnapshotArchive, uploadWorkspaceSnapshot } from "./remote-workspace-snapshot-storage.js";
|
||||
import { notifyRemoteWorkspaceSnapshots } from "./remote-workspace-snapshot-retention.js";
|
||||
import { materializeTaskDirectorySnapshot, removeTaskDirectoryTree } from "./isolated-task-directory-snapshot.js";
|
||||
import { loadTaskDirectoryRestorePoint, mayBeIsolatedTaskDirectoryWorkspace, readTaskDirectoryRetention, validateTaskDirectoryRestorePointRef } from "./isolated-task-directory-retention.js";
|
||||
import { inspectArchivedGitMigrationSource, materializeArchivedGitMigrationSource, type ArchivedGitMigrationSource } from "./remote-workspace-migration-git.js";
|
||||
import { inspectArchivedGitMigrationSource, lockArchivedGitMigrationSources, materializeArchivedGitMigrationSource, type ArchivedGitMigrationSource } from "./remote-workspace-migration-git.js";
|
||||
import { readNativeWorkspaceSyncReference } from "./native-runtime/native-workspace-sync.js";
|
||||
import { isLegacyWorkspaceTaskRelated, readLegacyWorkspaceRecoveryAllocations, LEGACY_WORKSPACE_RECOVERY_MIGRATION, type LegacyWorkspaceRecoverySource } from "./legacy-workspace-recovery-migration.js";
|
||||
import { offlineWorkspaceMigrationBlockers, prepareNestedWorkspaceImports, validateImportedWorkspaceChildren, workspaceMigrationLayoutBlockers } from "./remote-workspace-migration-tree.js";
|
||||
@@ -315,7 +315,12 @@ export async function migrateRemoteWorkspace(input: {
|
||||
try {
|
||||
input.signal?.throwIfAborted();
|
||||
const retained = reserved.retainedDirectory?.restorePoint;
|
||||
const sourceRoot = retained ? await materializeTaskDirectorySnapshot({
|
||||
const archivePath = path.join(scratch, "workspace.pcws");
|
||||
const recovered = reserved.archivedGitSource ? await recoverWorkspaceSnapshotArchive({
|
||||
db: input.db, storage: input.storage, companyId: input.companyId, workspaceId: input.workspaceId,
|
||||
snapshotId: snapshot.id, targetPath: archivePath, signal: input.signal,
|
||||
}) : null;
|
||||
const sourceRoot = recovered ? null : retained ? await materializeTaskDirectorySnapshot({
|
||||
payload: await loadTaskDirectoryRestorePoint({ storage: input.storage, workspace: reserved.workspace, ref: retained }),
|
||||
target: path.join(scratch, "retained-source"), expectedDigest: retained.digest,
|
||||
}) : reserved.archivedGitSource ? await materializeArchivedGitMigrationSource({ db: input.db, storage: input.storage,
|
||||
@@ -338,16 +343,17 @@ export async function migrateRemoteWorkspace(input: {
|
||||
operation = saved;
|
||||
}
|
||||
const exclude = nested.map(child => child.relativePath);
|
||||
const archivePath = path.join(scratch, "workspace.pcws");
|
||||
const captured = await captureRemoteWorkspaceDirectory({ cwd: sourceRoot, archivePath, exclude, signal: input.signal });
|
||||
const captured = recovered ?? await captureRemoteWorkspaceDirectory({ cwd: sourceRoot!, archivePath, exclude, signal: input.signal });
|
||||
const ref = await uploadWorkspaceSnapshot({ db: input.db, storage: input.storage, snapshotId: snapshot.id,
|
||||
companyId: input.companyId, workspaceId: input.workspaceId, archivePath, manifestDigest: captured.manifestDigest, signal: input.signal });
|
||||
const restoredArchive = path.join(scratch, "verified.pcws"), restored = path.join(scratch, "restored");
|
||||
await loadWorkspaceSnapshotArchive({ storage: input.storage, reference: ref, targetPath: restoredArchive, signal: input.signal });
|
||||
await restoreRemoteWorkspaceArchive({ archivePath: restoredArchive, targetDir: restored, expectedManifestDigest: ref.manifestDigest, signal: input.signal });
|
||||
const currentArchive = path.join(scratch, "current.pcws");
|
||||
const current = await captureRemoteWorkspaceDirectory({ cwd: sourceRoot, archivePath: currentArchive, exclude, signal: input.signal });
|
||||
if (current.manifestDigest !== captured.manifestDigest) throw new Error("remote_workspace_migration_source_changed");
|
||||
if (!recovered) {
|
||||
const currentArchive = path.join(scratch, "current.pcws");
|
||||
const current = await captureRemoteWorkspaceDirectory({ cwd: sourceRoot!, archivePath: currentArchive, exclude, signal: input.signal });
|
||||
if (current.manifestDigest !== captured.manifestDigest) throw new Error("remote_workspace_migration_source_changed");
|
||||
}
|
||||
if (nested.length) {
|
||||
const verifyScratch = path.join(scratch, "verify-nested"); await fs.mkdir(verifyScratch);
|
||||
const verified = await prepareNestedWorkspaceImports({ ...input, workspace: reserved.workspace, scratch: verifyScratch, recordedChildren: nested });
|
||||
@@ -385,6 +391,7 @@ export async function migrateRemoteWorkspace(input: {
|
||||
if (Boolean(validation.retainedDirectory) !== Boolean(reserved.retainedDirectory)) throw new Error("remote_workspace_migration_source_changed");
|
||||
if (canonical(gitSourceProof(validation.archivedGitSource)) !== canonical(gitSourceProof(reserved.archivedGitSource))) throw new Error("remote_workspace_migration_source_changed");
|
||||
if (validation.reasons.length) throw new Error(`remote_workspace_migration_blocked:${validation.reasons.join(",")}`);
|
||||
if (reserved.archivedGitSource) await lockArchivedGitMigrationSources(tx as unknown as Db, workspace, reserved.archivedGitSource);
|
||||
const [uploaded] = await tx.select().from(executionWorkspaceSnapshots).where(and(eq(executionWorkspaceSnapshots.companyId, workspace.companyId),
|
||||
eq(executionWorkspaceSnapshots.executionWorkspaceId, workspace.id), eq(executionWorkspaceSnapshots.id, ref.id))).for("update");
|
||||
if (!uploaded || uploaded.status !== "uploaded" || uploaded.sourceOperationId !== operation.id) throw new Error("remote_workspace_migration_snapshot_changed");
|
||||
|
||||
@@ -11,7 +11,7 @@ import { createLocalDiskStorageProvider } from "../storage/local-disk-provider.j
|
||||
import { createStorageService } from "../storage/service.js";
|
||||
import type { StorageService } from "../storage/types.js";
|
||||
import { captureRemoteWorkspaceDirectory, restoreRemoteWorkspaceArchive } from "./remote-workspace-snapshot-archive.js";
|
||||
import { beginWorkspaceSnapshot, getWorkspaceSnapshot, loadWorkspaceSnapshotArchive, uploadWorkspaceSnapshot } from "./remote-workspace-snapshot-storage.js";
|
||||
import { beginWorkspaceSnapshot, getWorkspaceSnapshot, loadWorkspaceSnapshotArchive, recoverWorkspaceSnapshotArchive, uploadWorkspaceSnapshot } from "./remote-workspace-snapshot-storage.js";
|
||||
|
||||
const externalDatabaseUrl = process.env.PAPERCLIP_TEST_DATABASE_URL;
|
||||
if (externalDatabaseUrl && !/^remote_workspace_[a-z_]*test$/.test(new URL(externalDatabaseUrl).pathname.slice(1))) {
|
||||
@@ -75,6 +75,21 @@ suite("remote workspace snapshot storage", () => {
|
||||
expect(row?.status).toBe("pending"); expect((row?.metadata?.uploads as unknown[]).length).toBe(1);
|
||||
await expect(uploadWorkspaceSnapshot({ ...f.upload, manifestDigest: "b".repeat(64) })).rejects.toThrow("operation_content_changed");
|
||||
});
|
||||
it("authenticates a lost-reply archive without publishing its pending upload", async () => {
|
||||
const f = await fixture(), put = storage.putFile.bind(storage);
|
||||
storage.putFile = async input => { await put(input); throw new Error("lost upload reply"); };
|
||||
await expect(uploadWorkspaceSnapshot(f.upload)).rejects.toThrow("lost upload reply");
|
||||
await fs.rm(f.archivePath);
|
||||
const targetPath = path.join(root, "recovered.pcws");
|
||||
const ref = await recoverWorkspaceSnapshotArchive({ ...f.upload, targetPath });
|
||||
expect(ref?.manifestDigest).toBe(f.captured.manifestDigest);
|
||||
expect((await db.select().from(executionWorkspaceSnapshots))[0]?.status).toBe("pending");
|
||||
expect(await getWorkspaceSnapshot(db, { ...f, snapshotId: f.row.id })).toBeNull();
|
||||
await restoreRemoteWorkspaceArchive({ archivePath: targetPath, targetDir: path.join(root, "restored"), expectedManifestDigest: ref!.manifestDigest });
|
||||
expect(await fs.readFile(path.join(root, "restored", "saved.txt"), "utf8")).toBe("acknowledged workspace content\n");
|
||||
await expect(recoverWorkspaceSnapshotArchive({ ...f.upload, companyId: randomUUID(), targetPath: path.join(root, "wrong-company.pcws") }))
|
||||
.rejects.toThrow("remote_workspace_snapshot_missing");
|
||||
});
|
||||
it("keeps operation identity and company/workspace scope exact", async () => {
|
||||
const f = await fixture();
|
||||
expect((await beginWorkspaceSnapshot(db, { ...f, generation: 1, epoch: 3 })).id).toBe(f.row.id);
|
||||
|
||||
@@ -95,6 +95,31 @@ export async function getWorkspaceSnapshot(db: Db, input: { companyId: string; w
|
||||
return row && ["uploaded", "committed", "superseded"].includes(row.status) ? reference(row) : null;
|
||||
}
|
||||
|
||||
export async function recoverWorkspaceSnapshotArchive(input: {
|
||||
db: Db; storage: StorageService; snapshotId: string; companyId: string; workspaceId: string;
|
||||
targetPath: string; signal?: AbortSignal;
|
||||
}): Promise<WorkspaceSnapshotRef | null> {
|
||||
const [row] = await input.db.select().from(executionWorkspaceSnapshots).where(scope(input));
|
||||
if (!row) failure("missing");
|
||||
if (["uploaded", "committed", "superseded"].includes(row!.status)) {
|
||||
const ref = reference(row!);
|
||||
await loadWorkspaceSnapshotArchive({ ...input, reference: ref });
|
||||
return ref;
|
||||
}
|
||||
if (row!.status !== "pending") failure("not_writable");
|
||||
const attempts = Array.isArray(row!.metadata?.uploads) ? row!.metadata!.uploads as UploadAttempt[] : [];
|
||||
for (const attempt of attempts) {
|
||||
input.signal?.throwIfAborted();
|
||||
const ref: WorkspaceSnapshotRef = { schema: SCHEMA, ...attempt, id: row!.id, companyId: row!.companyId,
|
||||
workspaceId: row!.executionWorkspaceId, generation: row!.generation };
|
||||
validateReference(ref);
|
||||
if (!(await input.storage.headObject(ref.companyId, ref.objectKey, { signal: input.signal })).exists) continue;
|
||||
await loadWorkspaceSnapshotArchive({ ...input, reference: ref });
|
||||
return ref;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Upload attempts are journaled before remote I/O. The caller owns publication. */
|
||||
export async function uploadWorkspaceSnapshot(input: {
|
||||
db: Db; storage: StorageService; snapshotId: string; companyId: string; workspaceId: string;
|
||||
|
||||
Reference in new issue
Block a user