mirror of
https://github.com/mvschwarz/openrig.git
synced 2026-11-08 17:31:56 -10:00
fix(daemon,cli): rig up <rigName> falls back to manual snapshot when no auto-pre-down exists
Lifecycle v2 L3b builds on L3 (66785c8) by closing the rig-up-by-name + manual-snapshot gap reproduced in the L4 VM proof atfad10a9cycle 1. Before L3b, an operator running `rig up <rigName>` against a rig that has only manual snapshots got HTTP 404 with `code: "no_snapshot"` and the message "Rig exists but has no auto-pre-down snapshot." even though `rig restore <snapshotId> --rig <rigId>` succeeded against the same manual snapshot. After L3b, `rig up <rigName>` consults the latest snapshot of any kind whose metadata is sufficient for `RestoreOrchestrator.restore`'s pre-validation, preferring `auto-pre-down` when present. The selected snapshot kind is echoed in the API response so operators see what was used; the CLI prints a one-line manual-fallback note when the kind is not `auto-pre-down`. Decisions resolved (per plan, ratified by orch-lead with amendments): A. Re-use `rig up <rigName>` (no new explicit operator command). The 404-on-manual-only behavior was a missing-implementation symptom, not a safety feature. Adding a separate `rig recover` would force operators to memorize a "use this command for this snapshot kind" dichotomy that PM spec D2 warns against. Per orch amendments (overriding the plan where they conflict): 1. **Validate against actual SnapshotData.** There is no `data.bindings[]` field, and `Session` has no `runtime` field — runtime lives on `nodes`. Resume tokens are NOT required (resume-tokenless rigs remain restorable). The new helper checks the same minimum metadata `RestoreOrchestrator.validatePreRestore` actually requires: rig with id, nodes, edges, sessions arrays, checkpoints object; and when sessions is non-empty each session has a non-empty `sessionName` and `nodeId`. Empty sessions arrays are accepted (matches the validator). 2. **Candidate selection does not LIMIT 1 before validation.** The SQL orders snapshots by `(kind = 'auto-pre-down') DESC, created_at DESC, id DESC`, then a JS loop validates each candidate and skips invalid rows (corrupted JSON, missing structural metadata, missing per-session linkage). Auto-pre-down preference is preserved. 3. **Test files** follow repo conventions: `snapshot-repository.test.ts`, `up-route.test.ts`, `rigs-routes.test.ts`, and `up.test.ts`. Implementation: - `packages/daemon/src/domain/snapshot-repository.ts`: new method `findLatestRestoreUsable(rigId): Snapshot | null`. Free helper `isRestoreUsableSnapshotData(data)` validates structural metadata. `findLatestAutoPreDown` and `findLatestUsableSnapshot` (L2) are unchanged. - `packages/daemon/src/routes/up.ts`: `restoreByRigId` calls the new helper. Response `snapshotKind: snapshot.kind` echoed on success and on pre-restore-validation 409 paths. 404 message updated to "Rig exists but has no restore-usable snapshot. Start fresh with: rig up <spec-path>". - `packages/daemon/src/routes/rigs.ts`: `/:id/up` mirrors the same changes. - `packages/cli/src/commands/up.ts`: when the daemon response carries `snapshotKind` and it's not `auto-pre-down`, the CLI prints `Restoring from manual snapshot (kind=<kind>); no auto-pre-down snapshot available.` BEFORE the "Rig restored" line so operators see the fallback explicitly. Tests added (matching plan closeout checklist + orch amendment #3): - `daemon/test/snapshot-repository.test.ts`: 10 cases for `findLatestRestoreUsable` — auto-pre-down preferred over manual, manual returned when no auto-pre-down, latest of multiple manuals (created_at DESC, id DESC), auto-pre-down preferred over a NEWER manual, null when no snapshot exists, skips corrupted JSON and considers next, skips snapshot with empty sessionName on a session, skips snapshot with missing nodeId on a session, null when no snapshot has restore-usable structural metadata, accepts empty sessions array (matches validatePreRestore), plus a regression confirming `findLatestAutoPreDown` is unchanged. - `daemon/test/up-route.test.ts`: 3 cases — auto-pre-down preference; manual-fallback restore returns `snapshotKind: "manual"`; 404 with updated `restore-usable` message and no `auto-pre-down snapshot` text. - `daemon/test/rigs-routes.test.ts`: 3 cases mirroring the above for `/api/rigs/:id/up`. - `cli/test/up.test.ts`: 2 cases — fallback note printed when `snapshotKind="manual"`; NOT printed when `snapshotKind="auto-pre-down"`. Verification: - focused daemon: `npx vitest run \ test/snapshot-repository.test.ts \ test/up-route.test.ts \ test/rigs-routes.test.ts` → 65/65 green. - focused CLI: `npx vitest run test/up.test.ts` → 27/27 green. - full daemon suite: 2085/2085 (+17 from L3 baseline of 2068). - full CLI suite: 4 pre-existing failures (port 7433 in use; chatroom timeout; config env reads). Confirmed not L3b regressions; same set L2/L3 documented. - `npx tsc --noEmit` clean on both packages. Invariants preserved: - **Auto-pre-down preference unchanged** when both kinds are present. - **`findLatestAutoPreDown` unchanged**; still returns auto-pre-down only with no kind fallback (regression-checked). - **`findLatestUsableSnapshot` (L2) untouched** — different concept (requires resume tokens; consumed by lifecycle projection). - **Direct `rig restore <snapshot> --rig <id>` unchanged.** - **Fail-closed when no usable snapshot exists**: returns 404 with actionable message, no silent fallback to incompatible snapshots. - **No schema migration**; the existing `snapshots.kind` text column is queried, not constrained. - **No `RestoreOrchestrator` changes**; helper validation matches what the orchestrator already enforces in `validatePreRestore`. - **No UI/MCP/Auth-C/OAuth/account/quota/`~/.codex` changes**; `routes/up.ts` and `routes/rigs.ts` /:id/up are the only routes touched. - **No host lifecycle mutation.** L3b implementation accept may close L3b directly without a new VM cycle per the plan (the L4 proof atfad10a9already reproduced the gap; the fix is unit/integration-test-coverable; no new runtime behavior beyond snapshot selection). If guard requests a focused VM cycle, tester can run one targeted L4-3 cycle. Refs: - rigs/openrig-build/state/orch/velocity-slice-L3b-snapshot-selection-rig-up-by-name-plan-20260428.md - rigs/openrig-build/state/orch/velocity-conveyor-state-l4-vm-reboot-proof-fad10a9-20260428.md (cycle 1 reproduced the gap) - L3 predecessor:66785c8feat(daemon,cli): restore attempt id, attention_required classification, runtime-truth reconciliation - L2 predecessor:fad10a9feat(daemon,cli): add recoverable/degraded/attention_required lifecycle projection - L1 predecessor:3ad1df6fix(daemon): classify tmux socket-absence as missing session and log startup reconcile Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This commit is contained in:
1 parent
66785c8ee7
commit
e70694f660
8 files changed
+397
-6
No files matched your search
@@ -216,6 +216,14 @@ Examples:
|
||||
const rigId = res.data["rigId"] as string;
|
||||
const rigName = res.data["rigName"] as string | undefined;
|
||||
const rigResult = res.data["rigResult"] as string | undefined;
|
||||
const snapshotKind = res.data["snapshotKind"] as string | undefined;
|
||||
// L3b: when the daemon falls back to a non-auto-pre-down snapshot, surface
|
||||
// the kind so operators see the manual fallback explicitly. The note is
|
||||
// printed BEFORE the "Rig restored" line so it's visible in the typical
|
||||
// top-of-output scan.
|
||||
if (snapshotKind && snapshotKind !== "auto-pre-down") {
|
||||
console.log(`Restoring from manual snapshot (kind=${snapshotKind}); no auto-pre-down snapshot available.`);
|
||||
}
|
||||
console.log(`Rig "${rigName ?? rigId}" restored (ID: ${rigId})`);
|
||||
if (rigResult) console.log(`Result: ${rigResult}`);
|
||||
const nodes = (res.data["nodes"] as Array<{ logicalId: string; status: string }>) ?? [];
|
||||
|
||||
@@ -826,4 +826,72 @@ describe("Up CLI", () => {
|
||||
const joined = logs.join("");
|
||||
expect(() => JSON.parse(joined)).not.toThrow();
|
||||
});
|
||||
|
||||
// L3b: manual-fallback note printed when restore came from a non-auto-pre-down snapshot.
|
||||
it("L3b: prints manual-fallback note when daemon restored from a manual snapshot", async () => {
|
||||
const origListeners = server.listeners("request");
|
||||
server.removeAllListeners("request");
|
||||
server.on("request", async (req: http.IncomingMessage, res: http.ServerResponse) => {
|
||||
let body = "";
|
||||
for await (const chunk of req) body += chunk;
|
||||
if (req.url === "/api/specs/library" && req.method === "GET") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify([]));
|
||||
} else if (req.url === "/api/rigs/summary" && req.method === "GET") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify([{ id: "r-l3b", name: "manual-only-rig", nodeCount: 1, lifecycleState: "recoverable" }]));
|
||||
} else if (req.url === "/api/up" && req.method === "POST") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify({ status: "restored", rigId: "r-l3b", rigName: "manual-only-rig", rigResult: "fully_restored", nodes: [], warnings: [], snapshotKind: "manual" }));
|
||||
} else {
|
||||
res.writeHead(404);
|
||||
res.end();
|
||||
}
|
||||
});
|
||||
|
||||
const { logs, exitCode } = await captureLogs(async () => {
|
||||
await makeCmd().parseAsync(["node", "rig", "up", "manual-only-rig"]);
|
||||
});
|
||||
|
||||
server.removeAllListeners("request");
|
||||
for (const l of origListeners) server.on("request", l as (...args: unknown[]) => void);
|
||||
|
||||
const output = logs.join("\n");
|
||||
expect(output).toContain("Restoring from manual snapshot (kind=manual); no auto-pre-down snapshot available.");
|
||||
expect(output).toContain('Rig "manual-only-rig" restored');
|
||||
expect(exitCode).toBeUndefined();
|
||||
});
|
||||
|
||||
it("L3b: does NOT print manual-fallback note when restore came from auto-pre-down", async () => {
|
||||
const origListeners = server.listeners("request");
|
||||
server.removeAllListeners("request");
|
||||
server.on("request", async (req: http.IncomingMessage, res: http.ServerResponse) => {
|
||||
let body = "";
|
||||
for await (const chunk of req) body += chunk;
|
||||
if (req.url === "/api/specs/library" && req.method === "GET") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify([]));
|
||||
} else if (req.url === "/api/rigs/summary" && req.method === "GET") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify([{ id: "r-l3b-auto", name: "auto-rig", nodeCount: 1, lifecycleState: "recoverable" }]));
|
||||
} else if (req.url === "/api/up" && req.method === "POST") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify({ status: "restored", rigId: "r-l3b-auto", rigName: "auto-rig", rigResult: "fully_restored", nodes: [], warnings: [], snapshotKind: "auto-pre-down" }));
|
||||
} else {
|
||||
res.writeHead(404);
|
||||
res.end();
|
||||
}
|
||||
});
|
||||
|
||||
const { logs } = await captureLogs(async () => {
|
||||
await makeCmd().parseAsync(["node", "rig", "up", "auto-rig"]);
|
||||
});
|
||||
|
||||
server.removeAllListeners("request");
|
||||
for (const l of origListeners) server.on("request", l as (...args: unknown[]) => void);
|
||||
|
||||
const output = logs.join("\n");
|
||||
expect(output).not.toContain("Restoring from manual snapshot");
|
||||
expect(output).toContain('Rig "auto-rig" restored');
|
||||
});
|
||||
});
|
||||
@@ -43,6 +43,43 @@ export class SnapshotRepository {
|
||||
return row ? this.rowToSnapshot(row) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* L3b: returns the latest snapshot whose persisted `data` carries the minimum
|
||||
* structural metadata `RestoreOrchestrator.restore`'s pre-validation requires.
|
||||
* Prefers `auto-pre-down` over other kinds when both are present.
|
||||
*
|
||||
* The single SQL query orders snapshots by `(kind = 'auto-pre-down') DESC,
|
||||
* created_at DESC, id DESC` so an auto-pre-down candidate always comes first
|
||||
* if any exists. The in-memory loop then validates each candidate and skips
|
||||
* snapshots with corrupted JSON or missing topology metadata, returning the
|
||||
* first usable row. Returns null when no usable snapshot exists.
|
||||
*
|
||||
* Distinct from `findLatestUsableSnapshot` (rig-repository.ts, L2): that
|
||||
* helper requires at least one persisted resume token and is consumed by
|
||||
* the lifecycle projection. This helper only requires structural metadata
|
||||
* `RestoreOrchestrator.restore` actually inspects, so terminal-only or
|
||||
* resume-tokenless rigs still resolve.
|
||||
*/
|
||||
findLatestRestoreUsable(rigId: string): Snapshot | null {
|
||||
const rows = this.db
|
||||
.prepare(
|
||||
"SELECT * FROM snapshots WHERE rig_id = ? ORDER BY (kind = 'auto-pre-down') DESC, created_at DESC, id DESC"
|
||||
)
|
||||
.all(rigId) as SnapshotRow[];
|
||||
|
||||
for (const row of rows) {
|
||||
let data: SnapshotData;
|
||||
try {
|
||||
data = JSON.parse(row.data) as SnapshotData;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (!isRestoreUsableSnapshotData(data)) continue;
|
||||
return this.rowToSnapshot(row);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
getLatestSnapshot(rigId: string): Snapshot | null {
|
||||
const row = this.db
|
||||
.prepare(
|
||||
@@ -119,3 +156,37 @@ interface SnapshotRow {
|
||||
data: string;
|
||||
created_at: string;
|
||||
}
|
||||
|
||||
// Validates `SnapshotData` carries the minimum structural metadata
|
||||
// `RestoreOrchestrator.restore`'s pre-validation requires.
|
||||
//
|
||||
// Per the L3b orch amendment: validate against actual SnapshotData. There is
|
||||
// NO `data.bindings[]` field and `Session` has NO `runtime` field (runtime
|
||||
// lives on nodes). Resume tokens are NOT required (resume-tokenless rigs are
|
||||
// still restorable).
|
||||
//
|
||||
// Required:
|
||||
// - rig with non-empty id
|
||||
// - nodes array (may be empty — restore handles empty topologies)
|
||||
// - edges array (may be empty)
|
||||
// - sessions array (may be empty — `validatePreRestore` accepts empty)
|
||||
// - checkpoints object
|
||||
//
|
||||
// When sessions is non-empty, each session must have a non-empty sessionName
|
||||
// and nodeId so node linkage can be resolved during restore. We do NOT check
|
||||
// session.runtime because that field doesn't exist on Session (orch amendment).
|
||||
function isRestoreUsableSnapshotData(data: unknown): data is SnapshotData {
|
||||
if (!data || typeof data !== "object") return false;
|
||||
const d = data as SnapshotData;
|
||||
if (!d.rig || typeof d.rig.id !== "string" || d.rig.id.length === 0) return false;
|
||||
if (!Array.isArray(d.nodes)) return false;
|
||||
if (!Array.isArray(d.edges)) return false;
|
||||
if (!Array.isArray(d.sessions)) return false;
|
||||
if (!d.checkpoints || typeof d.checkpoints !== "object") return false;
|
||||
for (const s of d.sessions) {
|
||||
if (!s || typeof s !== "object") return false;
|
||||
if (typeof s.sessionName !== "string" || s.sessionName.length === 0) return false;
|
||||
if (typeof s.nodeId !== "string" || s.nodeId.length === 0) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
@@ -288,7 +288,10 @@ rigsRoutes.post("/:id/attach-self", async (c) => {
|
||||
}
|
||||
});
|
||||
|
||||
// POST /api/rigs/:id/up — power-on an existing rig from auto-pre-down snapshot
|
||||
// POST /api/rigs/:id/up — power-on an existing rig from its latest restore-usable snapshot
|
||||
// L3b: prefers `auto-pre-down` when present but falls back to the latest manual
|
||||
// snapshot whose structural metadata satisfies pre-validation. Echoes
|
||||
// `snapshotKind` so operators see which snapshot was used.
|
||||
rigsRoutes.post("/:id/up", async (c) => {
|
||||
const rigId = c.req.param("id")!;
|
||||
const repo = getRepo(c);
|
||||
@@ -296,9 +299,9 @@ rigsRoutes.post("/:id/up", async (c) => {
|
||||
if (!rig) return c.json({ error: `Rig "${rigId}" not found. List rigs with: rig ps` }, 404);
|
||||
|
||||
const snapshotRepo = c.get("snapshotRepo" as never) as SnapshotRepository;
|
||||
const snapshot = snapshotRepo.findLatestAutoPreDown(rigId);
|
||||
const snapshot = snapshotRepo.findLatestRestoreUsable(rigId);
|
||||
if (!snapshot) {
|
||||
return c.json({ error: `Rig "${rig.rig.name}" exists but has no auto-pre-down snapshot. Start fresh with: rig up <spec-path>`, code: "no_snapshot" }, 404);
|
||||
return c.json({ error: `Rig "${rig.rig.name}" exists but has no restore-usable snapshot. Start fresh with: rig up <spec-path>`, code: "no_snapshot" }, 404);
|
||||
}
|
||||
|
||||
const restoreOrch = c.get("restoreOrchestrator" as never) as RestoreOrchestrator | undefined;
|
||||
@@ -320,6 +323,7 @@ rigsRoutes.post("/:id/up", async (c) => {
|
||||
rigName: rig.rig.name,
|
||||
error: result.message,
|
||||
code: result.code,
|
||||
snapshotKind: snapshot.kind,
|
||||
...result.result,
|
||||
remediation: result.result.blockers?.map((blocker) => blocker.remediation) ?? [],
|
||||
}, 409);
|
||||
@@ -338,6 +342,7 @@ rigsRoutes.post("/:id/up", async (c) => {
|
||||
rigId,
|
||||
rigName: rig.rig.name,
|
||||
snapshotId: snapshot.id,
|
||||
snapshotKind: snapshot.kind,
|
||||
rigResult: result.result.rigResult,
|
||||
nodes: result.result.nodes,
|
||||
warnings: result.result.warnings,
|
||||
|
||||
@@ -24,15 +24,22 @@ function getDeps(c: { get: (key: string) => unknown }) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore a rig by ID from its latest auto-pre-down snapshot.
|
||||
* Restore a rig by ID from its latest restore-usable snapshot.
|
||||
*
|
||||
* L3b: prefers `auto-pre-down` when present (existing behavior preserved as a
|
||||
* preference signal) but falls back to the latest manual snapshot whose
|
||||
* structural metadata satisfies `RestoreOrchestrator.restore`'s pre-validation.
|
||||
* The response payload echoes `snapshotKind` so the operator/CLI can surface
|
||||
* which snapshot was used.
|
||||
*
|
||||
* Shared helper used by both /api/up (rig_name) and /api/rigs/:rigId/up (Explorer).
|
||||
*/
|
||||
async function restoreByRigId(rigId: string, rigName: string | null, deps: ReturnType<typeof getDeps>, c: { json: (data: unknown, status?: number) => Response }) {
|
||||
const { snapshotRepo, restoreOrchestrator } = deps;
|
||||
|
||||
const snapshot = snapshotRepo.findLatestAutoPreDown(rigId);
|
||||
const snapshot = snapshotRepo.findLatestRestoreUsable(rigId);
|
||||
if (!snapshot) {
|
||||
return c.json({ error: `Rig exists but has no auto-pre-down snapshot. Start fresh with: rig up <spec-path>`, code: "no_snapshot" }, 404);
|
||||
return c.json({ error: `Rig exists but has no restore-usable snapshot. Start fresh with: rig up <spec-path>`, code: "no_snapshot" }, 404);
|
||||
}
|
||||
|
||||
if (!restoreOrchestrator) {
|
||||
@@ -52,6 +59,7 @@ async function restoreByRigId(rigId: string, rigName: string | null, deps: Retur
|
||||
rigName,
|
||||
error: result.message,
|
||||
code: result.code,
|
||||
snapshotKind: snapshot.kind,
|
||||
...result.result,
|
||||
remediation: result.result.blockers?.map((blocker) => blocker.remediation) ?? [],
|
||||
}, 409);
|
||||
@@ -70,6 +78,7 @@ async function restoreByRigId(rigId: string, rigName: string | null, deps: Retur
|
||||
rigId,
|
||||
rigName,
|
||||
snapshotId: snapshot.id,
|
||||
snapshotKind: snapshot.kind,
|
||||
rigResult: result.result.rigResult,
|
||||
nodes: result.result.nodes,
|
||||
warnings: result.result.warnings,
|
||||
|
||||
@@ -564,4 +564,42 @@ describe("Rig CRUD routes", () => {
|
||||
expect(res.status).toBe(404);
|
||||
expect((await res.json()).error).toContain("not found");
|
||||
});
|
||||
|
||||
// L3b: /:id/up mirrors the rig-name path: prefer auto-pre-down, fall back to
|
||||
// latest restore-usable manual snapshot, echo `snapshotKind`.
|
||||
it("L3b: POST /api/rigs/:id/up falls back to manual snapshot when no auto-pre-down exists; echoes snapshotKind", async () => {
|
||||
const rig = repo.createRig("manual-only-byid");
|
||||
repo.addNode(rig.id, "worker", { role: "worker" });
|
||||
snapshotCapture.captureSnapshot(rig.id, "manual");
|
||||
|
||||
const res = await app.request(`/api/rigs/${rig.id}/up`, { method: "POST" });
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(body.status).toBe("restored");
|
||||
expect(body.snapshotKind).toBe("manual");
|
||||
});
|
||||
|
||||
it("L3b: POST /api/rigs/:id/up prefers auto-pre-down over a newer manual snapshot", async () => {
|
||||
const rig = repo.createRig("auto-pref-byid");
|
||||
repo.addNode(rig.id, "worker", { role: "worker" });
|
||||
// Capture order doesn't matter; the helper orders by `(kind = 'auto-pre-down') DESC`.
|
||||
snapshotCapture.captureSnapshot(rig.id, "manual");
|
||||
snapshotCapture.captureSnapshot(rig.id, "auto-pre-down");
|
||||
|
||||
const res = await app.request(`/api/rigs/${rig.id}/up`, { method: "POST" });
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(body.snapshotKind).toBe("auto-pre-down");
|
||||
});
|
||||
|
||||
it("L3b: POST /api/rigs/:id/up returns 404 with updated 'no restore-usable snapshot' message", async () => {
|
||||
const rig = repo.createRig("no-usable-snap");
|
||||
|
||||
const res = await app.request(`/api/rigs/${rig.id}/up`, { method: "POST" });
|
||||
expect(res.status).toBe(404);
|
||||
const body = await res.json();
|
||||
expect(body.code).toBe("no_snapshot");
|
||||
expect(body.error).toContain("restore-usable");
|
||||
expect(body.error).not.toContain("auto-pre-down snapshot");
|
||||
});
|
||||
});
|
||||
@@ -147,4 +147,139 @@ describe("SnapshotRepository", () => {
|
||||
expect(remaining).toHaveLength(2);
|
||||
expect(remaining.map((s) => s.id)).toEqual(["s5", "s4"]); // newest 2 survive
|
||||
});
|
||||
|
||||
// L3b: findLatestRestoreUsable
|
||||
describe("findLatestRestoreUsable (L3b)", () => {
|
||||
function dataWithSession(opts?: { sessionName?: string; nodeId?: string; rigId?: string }): SnapshotData {
|
||||
return {
|
||||
rig: { id: opts?.rigId ?? "rig-1", name: "r01", createdAt: "2026-04-28", updatedAt: "2026-04-28" },
|
||||
nodes: [],
|
||||
edges: [],
|
||||
sessions: [{
|
||||
id: "sess-1",
|
||||
nodeId: opts?.nodeId ?? "node-a",
|
||||
sessionName: opts?.sessionName ?? "r01-worker",
|
||||
status: "detached",
|
||||
resumeType: null,
|
||||
resumeToken: null,
|
||||
restorePolicy: "resume_if_possible",
|
||||
lastSeenAt: null,
|
||||
createdAt: "2026-04-28T00:00:00Z",
|
||||
origin: "launched",
|
||||
startupStatus: "ready",
|
||||
startupCompletedAt: null,
|
||||
}],
|
||||
checkpoints: {},
|
||||
};
|
||||
}
|
||||
|
||||
function insertRaw(id: string, rigId: string, kind: string, dataJson: string, createdAt: string): void {
|
||||
db.prepare("INSERT INTO snapshots (id, rig_id, kind, status, data, created_at) VALUES (?, ?, ?, ?, ?, ?)")
|
||||
.run(id, rigId, kind, "complete", dataJson, createdAt);
|
||||
}
|
||||
|
||||
it("returns auto-pre-down when one exists (preference signal)", () => {
|
||||
repo.createSnapshot("rig-1", "manual", dataWithSession());
|
||||
const auto = repo.createSnapshot("rig-1", "auto-pre-down", dataWithSession());
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.id).toBe(auto.id);
|
||||
expect(result!.kind).toBe("auto-pre-down");
|
||||
});
|
||||
|
||||
it("returns latest manual when no auto-pre-down exists", () => {
|
||||
const m = repo.createSnapshot("rig-1", "manual", dataWithSession());
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.id).toBe(m.id);
|
||||
expect(result!.kind).toBe("manual");
|
||||
});
|
||||
|
||||
it("returns latest manual when multiple manuals exist (created_at DESC, id DESC)", () => {
|
||||
insertRaw("s1", "rig-1", "manual", JSON.stringify(dataWithSession()), "2026-04-27 10:00:00");
|
||||
insertRaw("s2", "rig-1", "manual", JSON.stringify(dataWithSession()), "2026-04-28 10:00:00");
|
||||
insertRaw("s3", "rig-1", "manual", JSON.stringify(dataWithSession()), "2026-04-28 09:00:00");
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result!.id).toBe("s2"); // newest by created_at
|
||||
});
|
||||
|
||||
it("prefers auto-pre-down over a newer manual snapshot", () => {
|
||||
insertRaw("auto-old", "rig-1", "auto-pre-down", JSON.stringify(dataWithSession()), "2026-04-27 10:00:00");
|
||||
insertRaw("manual-new", "rig-1", "manual", JSON.stringify(dataWithSession()), "2026-04-28 10:00:00");
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result!.id).toBe("auto-old");
|
||||
expect(result!.kind).toBe("auto-pre-down");
|
||||
});
|
||||
|
||||
it("returns null when no snapshot exists", () => {
|
||||
expect(repo.findLatestRestoreUsable("rig-1")).toBeNull();
|
||||
});
|
||||
|
||||
it("skips a corrupted-JSON snapshot and considers next candidate", () => {
|
||||
insertRaw("s-broken", "rig-1", "manual", "{ this is not valid json", "2026-04-28 11:00:00");
|
||||
const good = repo.createSnapshot("rig-1", "manual", dataWithSession());
|
||||
// Force ordering: re-insert good with older created_at so corrupt is "newest"
|
||||
db.prepare("UPDATE snapshots SET created_at = ? WHERE id = ?").run("2026-04-28 10:00:00", good.id);
|
||||
db.prepare("UPDATE snapshots SET created_at = ? WHERE id = ?").run("2026-04-28 11:00:00", "s-broken");
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.id).toBe(good.id);
|
||||
});
|
||||
|
||||
it("skips a snapshot with missing sessionName on a session and considers next", () => {
|
||||
const broken = JSON.parse(JSON.stringify(dataWithSession()));
|
||||
broken.sessions[0].sessionName = "";
|
||||
insertRaw("s-no-session-name", "rig-1", "manual", JSON.stringify(broken), "2026-04-28 11:00:00");
|
||||
const good = repo.createSnapshot("rig-1", "manual", dataWithSession());
|
||||
db.prepare("UPDATE snapshots SET created_at = ? WHERE id = ?").run("2026-04-28 10:00:00", good.id);
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result!.id).toBe(good.id);
|
||||
});
|
||||
|
||||
it("skips a snapshot with missing nodeId on a session and considers next", () => {
|
||||
const broken = JSON.parse(JSON.stringify(dataWithSession()));
|
||||
delete broken.sessions[0].nodeId;
|
||||
insertRaw("s-no-node-id", "rig-1", "manual", JSON.stringify(broken), "2026-04-28 11:00:00");
|
||||
const good = repo.createSnapshot("rig-1", "manual", dataWithSession());
|
||||
db.prepare("UPDATE snapshots SET created_at = ? WHERE id = ?").run("2026-04-28 10:00:00", good.id);
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result!.id).toBe(good.id);
|
||||
});
|
||||
|
||||
it("returns null when no snapshot has restore-usable structural metadata", () => {
|
||||
const noRig = JSON.stringify({ nodes: [], edges: [], sessions: [], checkpoints: {} });
|
||||
const noNodes = JSON.stringify({ rig: { id: "x", name: "r", createdAt: "", updatedAt: "" }, edges: [], sessions: [], checkpoints: {} });
|
||||
insertRaw("s-no-rig", "rig-1", "manual", noRig, "2026-04-28 11:00:00");
|
||||
insertRaw("s-no-nodes", "rig-1", "manual", noNodes, "2026-04-28 12:00:00");
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
it("accepts a snapshot with empty sessions array (matches validatePreRestore)", () => {
|
||||
// RestoreOrchestrator.validatePreRestore allows an empty sessions array
|
||||
// (only missing/non-array is rejected). The helper must match.
|
||||
const empty = sampleData();
|
||||
const s = repo.createSnapshot("rig-1", "manual", empty);
|
||||
|
||||
const result = repo.findLatestRestoreUsable("rig-1");
|
||||
expect(result).not.toBeNull();
|
||||
expect(result!.id).toBe(s.id);
|
||||
});
|
||||
|
||||
it("regression: findLatestAutoPreDown unchanged (still returns auto-pre-down only, no kind fallback)", () => {
|
||||
repo.createSnapshot("rig-1", "manual", dataWithSession());
|
||||
// No auto-pre-down: findLatestAutoPreDown returns null even though a
|
||||
// valid manual exists. findLatestRestoreUsable would return the manual.
|
||||
expect(repo.findLatestAutoPreDown("rig-1")).toBeNull();
|
||||
expect(repo.findLatestRestoreUsable("rig-1")).not.toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -131,4 +131,61 @@ describe("Up API route", () => {
|
||||
expect(body.rigResult).toBe("not_attempted");
|
||||
expect(body.blockers[0].path).toBe(missingPath);
|
||||
});
|
||||
|
||||
// L3b: rig-name path falls back to manual snapshot when no auto-pre-down exists.
|
||||
// Both routes preserve auto-pre-down preference and echo `snapshotKind`.
|
||||
describe("L3b snapshot-selection fallback", () => {
|
||||
it("auto-pre-down preferred when present; response echoes snapshotKind=auto-pre-down", async () => {
|
||||
const rig = rigRepo.createRig("auto-pref");
|
||||
rigRepo.addNode(rig.id, "worker", { role: "worker" });
|
||||
// Capture manual first, then auto-pre-down. Auto-pre-down must win.
|
||||
snapshotCapture.captureSnapshot(rig.id, "manual");
|
||||
snapshotCapture.captureSnapshot(rig.id, "auto-pre-down");
|
||||
|
||||
const res = await app.request("/api/up", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ sourceRef: "auto-pref" }),
|
||||
});
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(body.status).toBe("restored");
|
||||
expect(body.snapshotKind).toBe("auto-pre-down");
|
||||
});
|
||||
|
||||
it("falls back to manual snapshot when no auto-pre-down exists; response echoes snapshotKind=manual", async () => {
|
||||
const rig = rigRepo.createRig("manual-only");
|
||||
rigRepo.addNode(rig.id, "worker", { role: "worker" });
|
||||
snapshotCapture.captureSnapshot(rig.id, "manual");
|
||||
|
||||
const res = await app.request("/api/up", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ sourceRef: "manual-only" }),
|
||||
});
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json();
|
||||
expect(body.status).toBe("restored");
|
||||
expect(body.snapshotKind).toBe("manual");
|
||||
});
|
||||
|
||||
it("returns 404 with updated 'no restore-usable snapshot' message when no usable snapshot exists", async () => {
|
||||
rigRepo.createRig("no-snap");
|
||||
|
||||
const res = await app.request("/api/up", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ sourceRef: "no-snap" }),
|
||||
});
|
||||
|
||||
expect(res.status).toBe(404);
|
||||
const body = await res.json();
|
||||
expect(body.code).toBe("no_snapshot");
|
||||
expect(body.error).toContain("restore-usable");
|
||||
// Old message specifically said "auto-pre-down" — must NOT anymore.
|
||||
expect(body.error).not.toContain("auto-pre-down snapshot");
|
||||
});
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user