mirror of
https://github.com/mvschwarz/openrig.git
synced 2026-11-07 05:56:21 -10:00
feat(daemon): record probe evidence on failed Claude launches and refused attention clears (#1499)
A Claude seat whose resume launch failed reported a fixed message, and a refused `clear-attention` returned only the pane's status and code, so nobody could tell which probe result or which login text the decision rested on. Now `node.startup_failed` and `seat.attention_clear_refused` carry the probe's evidence: its code, which of the fixed login-refusal strings matched and where, the capture's line and byte counts, a SHA-256 of the text the probe read, and the pane size. No pane text or free-form probe detail is stored. Every probe-backed failed return carries it, and a missing strict reconciler records its own refusal without inventing an attempt or evidence. Optional pane geometry has a 250 ms deadline, so a slow or failing read leaves the size empty and the failure is still recorded. Recovery decisions and error messages are unchanged. Limits: the deadline bounds only the wait; it doesn't cancel the tmux command. The geometry is read after the capture, so it isn't an atomic snapshot. An unexpected reconciler exception keeps its existing error path and isn't recorded as a refusal. Thanks to @Aschenbath for the PR, and to @AdamWhitehurst for the issue and for checking the PR against his case. Cost: source +118/-17 (8 files), docs +33 (1 file), tests +237/-10 (5 files). No new refusal or error code; new fields on two existing events.
This commit is contained in:
1 parent
1c186e7640
commit
f9f843bf62
14 files changed
+388
-27
No files matched your search
@@ -88,6 +88,39 @@ Readiness has its own attention codes (`runtime-adapter.ts:74–87`):
|
||||
When `checkReady` reports one, the readiness wait returns at once and startup
|
||||
ends `attention_required`.
|
||||
|
||||
### Evidence for a failed Claude resume
|
||||
|
||||
Previously, a failed Claude launch recorded its error without the probe code or
|
||||
the captured input's identity, and a refused strict full-restore attention clear
|
||||
had no corresponding refusal event. Probe-backed Claude launch failures now
|
||||
carry an optional `probe` in `HarnessLaunchResult` and `node.startup_failed`.
|
||||
Strict full-restore refusals emit `seat.attention_clear_refused`; a
|
||||
`pane_not_usable` refusal includes that probe and the restore attempt ID.
|
||||
A missing strict reconciler emits `strict_reconciler_unavailable` without
|
||||
inventing an attempt ID or probe evidence, and still refuses the clear.
|
||||
|
||||
`native-resume-probe-evidence.ts` records the status, code, a closed-set login
|
||||
refusal and row kind when matched, and the exact capture's line count, byte
|
||||
count and SHA-256. Its new fields contain no captured pane text or free-form
|
||||
probe detail. Existing attention evidence and recovery decisions keep their
|
||||
previous behavior. MCP approval detection still takes precedence over the
|
||||
login-refusal check.
|
||||
|
||||
Pane dimensions are optional, observed after the capture. Their read has a
|
||||
250 ms deadline so a hung `listPanes` cannot indefinitely delay the failure
|
||||
result or its event. On timeout, error, ambiguous pane identity or invalid
|
||||
dimensions, the unavailable dimensions are `null`; late results are ignored.
|
||||
The deadline bounds this diagnostic wait, not the underlying tmux command.
|
||||
|
||||
`native-resume-probe-evidence.test.ts` covers unavailable and late geometry;
|
||||
`startup-orchestrator.test.ts` and `restore-orchestrator.test.ts` exercise the
|
||||
real adapter/orchestrator and SQLite event paths with available, failing and
|
||||
never-settling geometry reads. `seat-attention-reconciler.test.ts` checks that
|
||||
a missing strict reconciler records its refusal without changing restore
|
||||
history or sending input. These use synthetic terminal captures, not a
|
||||
live Claude session. The stub scenario runtime cannot draw Claude's native
|
||||
login-refusal UI, and its scenario runner does not bind the restore action.
|
||||
|
||||
## 2. The runtime adapters
|
||||
|
||||
**5** classes implement `RuntimeAdapter`
|
||||
|
||||
@@ -13,6 +13,7 @@ import type {
|
||||
import { resolveConcreteHint } from "../domain/runtime-adapter.js";
|
||||
import type { ProjectionPlan, ProjectionEntry } from "../domain/projection-planner.js";
|
||||
import { assessNativeResumeProbe } from "../domain/native-resume-probe.js";
|
||||
import { resumeProbeEvidence } from "../domain/native-resume-probe-evidence.js";
|
||||
import { resolveReadinessTimeoutMs } from "../domain/readiness-timeout.js";
|
||||
import { subtreeNamesClaudeExecutable, claudeCommandSessionToken, listNativeProcesses, observeClaudeDelivery, observeClaudePaneProcess, observeClaudePaneRuntime, observeCodexPaneProcess, verifyClaudePaneProcess, verifyCodexPaneProcess, verifyPaneIdleShell, witnessedClaudeVersionLabel, type ClaudeLaunchedProcess, type NativeProcessLister, type VersionLabelWitnessWait } from "../domain/native-process-lineage.js";
|
||||
import { mergeManagedBlock, DEFAULT_CLAUDE_MANAGED_BLOCK_FILE, type ClaudeManagedBlockFile } from "../domain/managed-blocks.js";
|
||||
@@ -604,6 +605,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
|
||||
if (probe.code === "no_conversation_found") {
|
||||
return {
|
||||
ok: false,
|
||||
probe: await resumeProbeEvidence(probe, paneContent, this.tmux, tmuxSession),
|
||||
error: "Claude resume failed: no conversation found for the requested session",
|
||||
recovery: "retry_fresh",
|
||||
missingConversation: true,
|
||||
@@ -620,6 +622,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
|
||||
if (probe.status === "attention_required") {
|
||||
return {
|
||||
ok: false,
|
||||
probe: await resumeProbeEvidence(probe, paneContent, this.tmux, tmuxSession),
|
||||
error: probe.detail,
|
||||
recovery: "attention_required",
|
||||
evidence: paneContent.split("\n").slice(-12).join("\n"),
|
||||
@@ -635,7 +638,8 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
|
||||
}
|
||||
const enterResult = await this.tmux.sendKeys(tmuxSession, ["Enter"]);
|
||||
if (!enterResult.ok) {
|
||||
return { ok: false, error: `Claude trust prompt auto-drive failed: ${enterResult.message}` };
|
||||
return { ok: false, error: `Claude trust prompt auto-drive failed: ${enterResult.message}`,
|
||||
probe: await resumeProbeEvidence(probe, paneContent, this.tmux, tmuxSession) };
|
||||
}
|
||||
await this.sleep(200);
|
||||
continue;
|
||||
@@ -688,10 +692,12 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
|
||||
if (finalProbe.status === "resumed") {
|
||||
return { ok: true };
|
||||
}
|
||||
const probeEvidence = await resumeProbeEvidence(finalProbe, finalContent, this.tmux, tmuxSession);
|
||||
|
||||
if (finalProbe.status === "attention_required" || finalProbe.code === "claude_auto_identity_required") {
|
||||
return {
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: finalProbe.detail,
|
||||
recovery: "attention_required",
|
||||
evidence: finalContent.split("\n").slice(-12).join("\n"),
|
||||
@@ -702,6 +708,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
|
||||
if (finalProbe.code === "no_conversation_found") {
|
||||
return {
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: no conversation found for the requested session",
|
||||
recovery: "retry_fresh",
|
||||
missingConversation: true,
|
||||
@@ -711,6 +718,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
|
||||
if (await this.paneReturnedToShell(binding, finalCommand)) {
|
||||
return {
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: pane returned to shell instead of entering Claude",
|
||||
recovery: "retry_fresh",
|
||||
};
|
||||
@@ -726,11 +734,12 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
|
||||
return {
|
||||
ok: false,
|
||||
error: `Claude resume did not finish within ${Math.round(attempts * pollMs / 1000)}s; Claude may still be starting, so the launch was kept`,
|
||||
probe: probeEvidence,
|
||||
recovery: "attention_required",
|
||||
evidence: finalContent.split("\n").slice(-12).join("\n"),
|
||||
};
|
||||
}
|
||||
return { ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" };
|
||||
return { ok: false, error: "Claude resume failed: timed out waiting for Claude to become active", probe: probeEvidence };
|
||||
}
|
||||
|
||||
private projectEntry(entry: ProjectionEntry, cwd: string, managedBlockFile: ClaudeManagedBlockFile, warnings: string[], seatSkillsRoot?: string, seatSkillsRefused?: string): boolean {
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import type { TmuxAdapter, TmuxPane } from "../adapters/tmux.js";
|
||||
import type { NativeResumeProbeResult } from "./native-resume-probe.js";
|
||||
|
||||
const GEOMETRY_TIMEOUT_MS = 250;
|
||||
|
||||
/** Bounded metadata from the exact capture assessed by the probe; never its text or detail. */
|
||||
export interface NativeResumeProbeEvidence {
|
||||
status: NativeResumeProbeResult["status"];
|
||||
code: string;
|
||||
loginRefusal?: NonNullable<NativeResumeProbeResult["loginRefusal"]>;
|
||||
capture: {
|
||||
lines: number;
|
||||
bytes: number;
|
||||
sha256: string;
|
||||
/** Geometry observed after the capture; null when unavailable within the diagnostic deadline. */
|
||||
columns: number | null;
|
||||
rows: number | null;
|
||||
};
|
||||
}
|
||||
|
||||
/** Read geometry best-effort, without re-capturing or changing the probe's decision. */
|
||||
export async function resumeProbeEvidence(
|
||||
probe: NativeResumeProbeResult,
|
||||
content: string,
|
||||
tmux: Pick<TmuxAdapter, "listPanes">,
|
||||
target: string,
|
||||
): Promise<NativeResumeProbeEvidence> {
|
||||
let columns: number | null = null;
|
||||
let rows: number | null = null;
|
||||
let timeout: ReturnType<typeof setTimeout> | undefined;
|
||||
const dimension = (value: number | undefined) => Number.isSafeInteger(value) && value! > 0 ? value! : null;
|
||||
try {
|
||||
// Geometry is optional: a stuck tmux read must not hold up an already-decided failure or its event.
|
||||
const panes = await Promise.race([
|
||||
tmux.listPanes(target),
|
||||
new Promise<TmuxPane[]>(resolve => { timeout = setTimeout(() => resolve([]), GEOMETRY_TIMEOUT_MS); }),
|
||||
]);
|
||||
const pane = /^%\d+$/.test(target) ? panes.find(pane => pane.id === target) : panes.length === 1 ? panes[0] : undefined;
|
||||
columns = dimension(pane?.width);
|
||||
rows = dimension(pane?.height);
|
||||
} catch { /* Diagnostics must not replace the original failure when geometry is unavailable. */ }
|
||||
finally { if (timeout !== undefined) clearTimeout(timeout); }
|
||||
return {
|
||||
status: probe.status,
|
||||
code: probe.code,
|
||||
...(probe.loginRefusal ? { loginRefusal: probe.loginRefusal } : {}),
|
||||
capture: {
|
||||
lines: content.length ? content.split(/\r?\n/).length : 0,
|
||||
bytes: Buffer.byteLength(content),
|
||||
sha256: createHash("sha256").update(content).digest("hex"),
|
||||
columns,
|
||||
rows,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -21,6 +21,7 @@ export interface NativeResumeProbeResult {
|
||||
status: NativeResumeProbeStatus;
|
||||
code: string;
|
||||
detail: string;
|
||||
loginRefusal?: { message: ClaudeLoginRefusal; row: "response" | "result" };
|
||||
}
|
||||
|
||||
export interface ProbeShellReadyInput {
|
||||
@@ -127,11 +128,13 @@ export function assessNativeResumeProbe(
|
||||
detail: "Claude is waiting for project MCP server approval before the session can become interactive.",
|
||||
};
|
||||
}
|
||||
if (hasCurrentClaudeLoginRefusal(paneContent)) {
|
||||
const loginRefusal = currentClaudeLoginRefusal(paneContent);
|
||||
if (loginRefusal) {
|
||||
return {
|
||||
status: "failed",
|
||||
code: "login_required",
|
||||
detail: "Claude is running but cannot continue until the user logs in.",
|
||||
loginRefusal,
|
||||
};
|
||||
}
|
||||
if (looksLikeClaudeTui(paneContent)) {
|
||||
@@ -334,11 +337,12 @@ function looksLikeClaudeResumeSelectionPrompt(paneContent: string): boolean {
|
||||
// Claude's interactive refusals of a request for a missing, expired or revoked login (code.claude.com/docs/en/errors).
|
||||
// Only these exact messages count. A 401 reads "Please run /login · API Error: 401 …" with a varying tail, and a
|
||||
// reply or tool output can start with the same words, so this input can't tell a real one apart.
|
||||
const CLAUDE_LOGIN_REFUSALS = new Set([
|
||||
const CLAUDE_LOGIN_REFUSALS = [
|
||||
"Not logged in · Please run /login",
|
||||
"Login expired · Please run /login",
|
||||
"OAuth token revoked · Please run /login",
|
||||
]);
|
||||
] as const;
|
||||
type ClaudeLoginRefusal = typeof CLAUDE_LOGIN_REFUSALS[number];
|
||||
|
||||
// A refusal blocks only as Claude's latest response. The persistent "Not logged in · Run /login" footer says only
|
||||
// that Claude holds no credential of its own, and a gateway can still authenticate its requests. Claude starts a
|
||||
@@ -346,12 +350,12 @@ const CLAUDE_LOGIN_REFUSALS = new Set([
|
||||
// row; an indented marker is quoted text. The last ❯ row is the input box only when no response follows it;
|
||||
// otherwise it is a submitted prompt and no box is drawn. A refusal counts when it is the last response after the
|
||||
// last submitted prompt, so older scrollback, quoted text and a refusal followed by a newer reply or prompt do not.
|
||||
function hasCurrentClaudeLoginRefusal(paneContent: string): boolean {
|
||||
function currentClaudeLoginRefusal(paneContent: string): NativeResumeProbeResult["loginRefusal"] {
|
||||
const lines = paneContent.split(/\r?\n/);
|
||||
const isPrompt = (line: string) => /^\s*❯/.test(line);
|
||||
const isResponse = (line: string) => /^[●⏺]/.test(line) || /^ {2}⎿/.test(line);
|
||||
const prompts: number[] = [];
|
||||
const responses: Array<{ at: number; text: string }> = [];
|
||||
const responses: Array<{ at: number; text: string; row: "response" | "result" }> = [];
|
||||
for (let at = 0; at < lines.length; at++) {
|
||||
const line = lines[at]!;
|
||||
if (isPrompt(line)) prompts.push(at);
|
||||
@@ -360,13 +364,14 @@ function hasCurrentClaudeLoginRefusal(paneContent: string): boolean {
|
||||
// Claude wraps its own rows with a short hanging indent; a right-aligned footer is not a continuation.
|
||||
for (let next = at + 1; next < lines.length && /^ {1,6}\S/.test(lines[next]!) && !isPrompt(lines[next]!)
|
||||
&& !isResponse(lines[next]!); next++) text += ` ${lines[next]}`;
|
||||
responses.push({ at, text: text.replace(/\s+/g, " ").trim() });
|
||||
responses.push({ at, text: text.replace(/\s+/g, " ").trim(), row: /^ {2}⎿/.test(line) ? "result" : "response" });
|
||||
}
|
||||
const lastPrompt = prompts.at(-1) ?? -1;
|
||||
const inputBox = responses.some(({ at }) => at > lastPrompt) ? lines.length : lastPrompt;
|
||||
const turnStart = prompts.filter((at) => at < inputBox).at(-1) ?? -1;
|
||||
const latest = responses.filter(({ at }) => at > turnStart && at < inputBox).at(-1)?.text;
|
||||
return latest !== undefined && CLAUDE_LOGIN_REFUSALS.has(latest);
|
||||
const latest = responses.filter(({ at }) => at > turnStart && at < inputBox).at(-1);
|
||||
const message = CLAUDE_LOGIN_REFUSALS.find(message => message === latest?.text);
|
||||
return message && latest ? { message, row: latest.row } : undefined;
|
||||
}
|
||||
|
||||
function looksLikeClaudeMcpApprovalPrompt(paneContent: string): boolean {
|
||||
|
||||
@@ -24,6 +24,7 @@ import type { PiResumeAdapter } from "../adapters/pi-resume.js";
|
||||
import type { OmpResumeAdapter } from "../adapters/omp-resume.js";
|
||||
import type { TranscriptStore } from "./transcript-store.js";
|
||||
import { assessNativeResumeProbe } from "./native-resume-probe.js";
|
||||
import { resumeProbeEvidence, type NativeResumeProbeEvidence } from "./native-resume-probe-evidence.js";
|
||||
import { observeClaudeDelivery, observeClaudeResumeLaunch, verifyClaudePaneProcess, verifyPaneIdleShell, type ClaudeLaunchedProcess } from "./native-process-lineage.js";
|
||||
import { reanchorBuiltinStartupFile, reanchorShippedProjectionEntry } from "./builtin-startup-files.js";
|
||||
import { seatGuidance, restoreMissingGuidance, missingGuidanceBlocks } from "./restore-guidance.js";
|
||||
@@ -72,6 +73,8 @@ export type ReconcileNodeResult =
|
||||
| "resume_token_not_used"
|
||||
| "pane_not_usable";
|
||||
detail: string;
|
||||
probe?: NativeResumeProbeEvidence;
|
||||
attemptId?: number;
|
||||
};
|
||||
|
||||
// Only these edge kinds constrain launch order
|
||||
@@ -2166,12 +2169,13 @@ export class RestoreOrchestrator {
|
||||
// Precondition #4: pane is at a usable/idle state — explicitly NOT a
|
||||
// resume-selection prompt and not the "returned to shell" failure mode.
|
||||
if (probe.status !== "resumed") {
|
||||
const evidence = await resumeProbeEvidence(probe, paneContent, this.tmuxAdapter, identity.pane);
|
||||
// The label reads as a shell, but the runtime's lineage was just proven in this pane: name a shell only for a proven idle one.
|
||||
if (probe.code === "returned_to_shell" && !await verifyPaneIdleShell({ target: identity.pane, tmux: this.tmuxAdapter,
|
||||
...(this.listProcesses ? { listProcesses: this.listProcesses } : {}) })) {
|
||||
return { ok: false, code: "pane_not_usable", detail: "Pane is not shown to be resumed: its command reads as a shell, but it is not a proven idle shell (the runtime may be starting or showing a dialog); reconciliation requires resumed." };
|
||||
return { ok: false, code: "pane_not_usable", probe: evidence, attemptId, detail: "Pane is not shown to be resumed: its command reads as a shell, but it is not a proven idle shell (the runtime may be starting or showing a dialog); reconciliation requires resumed." };
|
||||
}
|
||||
return { ok: false, code: "pane_not_usable", detail: `Pane state is ${probe.status} (${probe.code}); reconciliation requires resumed.` };
|
||||
return { ok: false, code: "pane_not_usable", probe: evidence, attemptId, detail: `Pane state is ${probe.status} (${probe.code}); reconciliation requires resumed.` };
|
||||
}
|
||||
|
||||
// All four preconditions hold. Append (never mutate) the audit event.
|
||||
|
||||
@@ -128,6 +128,7 @@ export type HarnessLaunchResult =
|
||||
// conversation doesn't exist; other `retry_fresh` failures (such as a pane back at its shell)
|
||||
// leave it unset.
|
||||
| { ok: false; error: string; recovery?: HarnessLaunchRecovery; evidence?: string; missingConversation?: true;
|
||||
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence;
|
||||
/** The native resume probe's code for the startup gate that stopped the launch (for example `hook_trust_gate`),
|
||||
* when one did. Reporting only: it changes no recovery decision. */
|
||||
gate?: string;
|
||||
|
||||
@@ -377,7 +377,8 @@ interface ClearAttentionDeps {
|
||||
to: "operator_recovered";
|
||||
evidence: { tmux: boolean; fgProcess: string; resumeTokenUsed: boolean; paneState: "usable" };
|
||||
}
|
||||
| { ok: false; code: string; detail: string }
|
||||
| { ok: false; code: string; detail: string; attemptId?: number;
|
||||
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence }
|
||||
>;
|
||||
}
|
||||
|
||||
@@ -426,6 +427,15 @@ export class SeatAttentionReconciler {
|
||||
if (derivedOutcome?.source === "restore.completed") {
|
||||
const reconcile = this.deps.reconcileRestoreOutcome;
|
||||
if (!reconcile) {
|
||||
eventBus.emit({
|
||||
type: "seat.attention_clear_refused",
|
||||
rigId: session.rigId,
|
||||
nodeId: session.nodeId,
|
||||
sessionId: session.id,
|
||||
sessionName,
|
||||
attentionClass: "restore_outcome",
|
||||
code: "strict_reconciler_unavailable",
|
||||
});
|
||||
return {
|
||||
ok: false,
|
||||
code: "not_demonstrably_responsive",
|
||||
@@ -434,6 +444,17 @@ export class SeatAttentionReconciler {
|
||||
}
|
||||
const restored = await reconcile(session.rigId, session.nodeId);
|
||||
if (!restored.ok) {
|
||||
eventBus.emit({
|
||||
type: "seat.attention_clear_refused",
|
||||
rigId: session.rigId,
|
||||
nodeId: session.nodeId,
|
||||
sessionId: session.id,
|
||||
sessionName,
|
||||
attentionClass: "restore_outcome",
|
||||
code: restored.code,
|
||||
...(restored.attemptId !== undefined ? { attemptId: restored.attemptId } : {}),
|
||||
...(restored.probe ? { probe: restored.probe } : {}),
|
||||
});
|
||||
return {
|
||||
ok: false,
|
||||
code: "not_demonstrably_responsive",
|
||||
|
||||
@@ -240,7 +240,8 @@ export type StartupResult = { warnings?: string[] } & (
|
||||
// outcomes so restore-orchestrator's per-node mapping can populate
|
||||
// `attentionEvidence` on the RestoreNodeResult. Internal type only;
|
||||
// not persisted on the failure event.
|
||||
| { ok: false; startupStatus: "attention_required" | "failed"; errors: string[]; evidence?: string; recovery?: string; missingConversation?: true });
|
||||
| { ok: false; startupStatus: "attention_required" | "failed"; errors: string[]; evidence?: string; recovery?: string; missingConversation?: true;
|
||||
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence });
|
||||
|
||||
interface StartupOrchestratorDeps {
|
||||
db: Database.Database;
|
||||
@@ -634,11 +635,11 @@ export class StartupOrchestrator {
|
||||
// unverified fork keeps its unsent context for a verified continuation.
|
||||
return this.fail(input, "attention_required", errors, launchResult.evidence,
|
||||
launchResult.forkIdentityPending ? forkContext ?? false : continuityOutcome === "fresh",
|
||||
undefined, false, "harness_launch", true);
|
||||
undefined, false, "harness_launch", true, launchResult.probe);
|
||||
}
|
||||
|
||||
errors.push(`Harness launch failed: ${launchResult.error}`);
|
||||
return this.fail(input, "failed", errors, undefined, false, launchResult.recovery, launchResult.missingConversation === true, "harness_launch", true);
|
||||
return this.fail(input, "failed", errors, undefined, false, launchResult.recovery, launchResult.missingConversation === true, "harness_launch", true, launchResult.probe);
|
||||
}
|
||||
} catch (err) {
|
||||
errors.push(`Harness launch error: ${(err as Error).message}`);
|
||||
@@ -948,6 +949,7 @@ export class StartupOrchestrator {
|
||||
missingConversation = false,
|
||||
stage?: StartupFailureStage,
|
||||
hasPendingWork?: boolean,
|
||||
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence,
|
||||
): StartupResult {
|
||||
if (status === "attention_required" && typeof pendingContext === "object") {
|
||||
// The reasons above stay visible; this only says what the status means for the fork.
|
||||
@@ -970,8 +972,10 @@ export class StartupOrchestrator {
|
||||
...(hasPendingWork !== undefined ? { hasPendingWork } : {}),
|
||||
requestedResumeToken: input.resumeToken ?? null,
|
||||
requestedResumeType: input.resumeType ?? null,
|
||||
...(probe ? { probe } : {}),
|
||||
});
|
||||
return { ok: false, startupStatus: status, errors, evidence, ...(recovery ? { recovery } : {}),
|
||||
...(probe ? { probe } : {}),
|
||||
...(missingConversation ? { missingConversation: true as const } : {}) };
|
||||
}
|
||||
|
||||
|
||||
@@ -193,6 +193,7 @@ export type RigEvent =
|
||||
// 2026-07-02 founder ruling).
|
||||
| { type: "session.resume_token_captured"; rigId: string; nodeId: string; sessionName: string; sessionId: string; runtime: string; outcome: "captured" | "preserved" | "skipped"; resumeType?: string; provenance?: "adoption"; reason?: "missing_sidecar" | "parse_error" | "probe_timeout" | "process_unreadable" | "invalid_token" | "stale_sidecar" | "higher_rank_present"; redacted: true }
|
||||
| { type: "seat.attention_cleared"; rigId: string; nodeId: string; sessionName: string; from: string; to: "ready"; clearedBy: "evidence" | "operator_attestation"; evidence?: { kind: string; state?: string; reason?: string }; reason?: string; previousError: string | null }
|
||||
| { type: "seat.attention_clear_refused"; rigId: string; nodeId: string; sessionId: string; sessionName: string; attentionClass: "restore_outcome"; code: string; attemptId?: number; probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence }
|
||||
| { type: "rig.imported"; rigId: string; specName: string; specVersion: string }
|
||||
// Package events (cross-rig, no rigId)
|
||||
| { type: "package.validated"; packageName: string; valid: boolean }
|
||||
@@ -233,7 +234,7 @@ export type RigEvent =
|
||||
| { type: "pod.deleted"; rigId: string; podId: string }
|
||||
| { type: "node.startup_pending"; rigId: string; nodeId: string; startupProof?: StartupProofSelection }
|
||||
| { type: "node.startup_ready"; rigId: string; nodeId: string; previousStatus?: "failed" | "attention_required"; submission?: { status: "unverified" | "staged"; reasons: string[]; warning?: string; diagnostics?: StartupSubmissionDiagnostic[] } }
|
||||
| { type: "node.startup_failed"; rigId: string; nodeId: string; error: string; sessionId?: string; freshContextPending?: boolean; forkContextPending?: { parentToken: string; isRestore: boolean }; submissionDiagnostics?: StartupSubmissionDiagnostic[]; stage?: StartupFailureStage; hasPendingWork?: boolean; requestedResumeToken?: string | null; requestedResumeType?: string | null }
|
||||
| { type: "node.startup_failed"; rigId: string; nodeId: string; error: string; sessionId?: string; freshContextPending?: boolean; forkContextPending?: { parentToken: string; isRestore: boolean }; submissionDiagnostics?: StartupSubmissionDiagnostic[]; stage?: StartupFailureStage; hasPendingWork?: boolean; requestedResumeToken?: string | null; requestedResumeType?: string | null; probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence }
|
||||
// OPR.0.4.3.06 — startup proof (challenge-verified orientation). Append-only.
|
||||
// `node.startup_challenged` freezes this launch's challenge ground truth
|
||||
// (challengeId + contractHash; the expected answer is recomputed, never
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
@@ -11,6 +12,11 @@ import { claudePostureFlag } from "../src/adapters/yolo-mode.js";
|
||||
import { resolveReadinessTimeoutMs } from "../src/domain/readiness-timeout.js";
|
||||
import { claudeCommandSessionToken } from "../src/domain/native-process-lineage.js";
|
||||
|
||||
const probeEvidence = expect.objectContaining({
|
||||
status: expect.any(String), code: expect.any(String),
|
||||
capture: { lines: expect.any(Number), bytes: expect.any(Number), sha256: expect.stringMatching(/^[a-f0-9]{64}$/), columns: null, rows: null },
|
||||
});
|
||||
|
||||
function mockTmux(): TmuxAdapter {
|
||||
return {
|
||||
sendText: vi.fn(async () => ({ ok: true as const })),
|
||||
@@ -721,6 +727,21 @@ describe("Claude Code runtime adapter", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("returns diagnostics from the captured login refusal without changing the timeout outcome", async () => {
|
||||
const tmux = mockTmux();
|
||||
const content = "❯ fixture-private-prompt\n● Login expired · Please run /login\n\n❯";
|
||||
vi.mocked(tmux.capturePaneContent).mockResolvedValue(content);
|
||||
vi.mocked(tmux.listPanes).mockResolvedValue([{ id: "%1", index: 0, cwd: "/project", width: 100, height: 30, active: true }]);
|
||||
const adapter = new ClaudeCodeAdapter({ tmux, fsOps: mockFs(), sleep: async () => {} });
|
||||
const result = await adapter.launchHarness(makeBinding(), { name: "dev-impl@test-rig", resumeToken: "abc-123" });
|
||||
expect(result).toEqual({
|
||||
ok: false, error: "Claude resume failed: timed out waiting for Claude to become active",
|
||||
probe: { status: "failed", code: "login_required", loginRefusal: { message: "Login expired · Please run /login", row: "response" },
|
||||
capture: { lines: 4, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"), columns: 100, rows: 30 } },
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain("fixture-private-prompt");
|
||||
});
|
||||
|
||||
it("launchHarness returns retry_fresh when Claude reports no conversation found for the requested resume token", async () => {
|
||||
const tmux = mockTmux();
|
||||
(tmux.getPaneCommand as ReturnType<typeof vi.fn>).mockResolvedValue("zsh");
|
||||
@@ -733,6 +754,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: no conversation found for the requested session",
|
||||
recovery: "retry_fresh",
|
||||
missingConversation: true,
|
||||
@@ -753,6 +775,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
expect(capture).toHaveBeenCalledTimes(17);
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: no conversation found for the requested session",
|
||||
recovery: "retry_fresh",
|
||||
missingConversation: true,
|
||||
@@ -769,6 +792,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: pane returned to shell instead of entering Claude",
|
||||
recovery: "retry_fresh",
|
||||
});
|
||||
@@ -814,6 +838,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
const { result, reads } = await run({ screen: () => "something Claude draws that no probe knows" });
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume did not finish within 6s; Claude may still be starting, so the launch was kept",
|
||||
recovery: "attention_required",
|
||||
evidence: "something Claude draws that no probe knows",
|
||||
@@ -848,6 +873,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
const { result } = await run({ screen: (read) => (read < 22 ? "" : "No conversation found with session ID: abc-123") });
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: no conversation found for the requested session",
|
||||
recovery: "retry_fresh",
|
||||
missingConversation: true,
|
||||
@@ -867,6 +893,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
const { result } = await run({ rows: (read) => (read < 25 ? occupied : [pane]) });
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: pane returned to shell instead of entering Claude",
|
||||
recovery: "retry_fresh",
|
||||
});
|
||||
@@ -877,6 +904,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
const { result, reads } = await run({ rows: (read) => (read < 19 ? occupied : [pane]) });
|
||||
expect(result).toEqual({
|
||||
ok: false,
|
||||
probe: probeEvidence,
|
||||
error: "Claude resume failed: pane returned to shell instead of entering Claude",
|
||||
recovery: "retry_fresh",
|
||||
});
|
||||
@@ -911,9 +939,9 @@ describe("Claude Code runtime adapter", () => {
|
||||
expect((await run({ readinessTimeoutMs, rows: () => [pane] })).result).toMatchObject({ ok: false, recovery: "retry_fresh" });
|
||||
const other = { ...claude, command: "claude --permission-mode acceptEdits --resume some-other-session --name other" };
|
||||
expect((await run({ readinessTimeoutMs, rows: () => [{ ...pane, tpgid: 11 }, launchScript, other] })).result)
|
||||
.toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
.toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect((await run({ readinessTimeoutMs, rows: () => stuck, screen: () => "● Not logged in · Please run /login" })).result)
|
||||
.toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
.toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -930,7 +958,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
});
|
||||
|
||||
describe("a different occupant keeps main's result (timed out, no token): only unknown observations get attention", () => {
|
||||
const timedOut = { ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" };
|
||||
const timedOut = { ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" };
|
||||
const codex = { pid: 13, ppid: 11, pgid: 11, tpgid: 11, executableName: "codex", startedAt, command: "codex resume some-codex-session" };
|
||||
const otherClaude = { ...claude, command: "claude --permission-mode acceptEdits --resume some-other-session --name other" };
|
||||
const booting = [{ ...pane, tpgid: 11 }, launchScript];
|
||||
@@ -1037,7 +1065,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
it("does not wait for a Claude that names another conversation: another occupant, timed out as before", async () => {
|
||||
const other = { ...claude, command: "claude --permission-mode acceptEdits --resume some-other-session --name other" };
|
||||
const { result, reads } = await run({ rows: () => [{ ...pane, tpgid: 11 }, launchScript, other] });
|
||||
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(reads).toBe(17);
|
||||
});
|
||||
|
||||
@@ -1046,7 +1074,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
const lookAlike = { ...claude, executableName: "printf" };
|
||||
const ready = "Restored conversation\n❯\u00a0\n ⏵⏵ bypass permissions on (shift+tab to cycle)";
|
||||
const { result, reads } = await run({ rows: () => [{ ...pane, tpgid: 11 }, launchScript, lookAlike], screen: () => ready });
|
||||
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(reads).toBe(17);
|
||||
});
|
||||
|
||||
@@ -1058,13 +1086,13 @@ describe("Claude Code runtime adapter", () => {
|
||||
rows: (read) => (read < 20 ? [{ ...pane, tpgid: 11 }, launchScript] : [{ ...pane, tpgid: 11 }, launchScript, lookAlike]),
|
||||
screen: (read) => (read < 20 ? "" : ready),
|
||||
});
|
||||
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(reads).toBeLessThan(25);
|
||||
});
|
||||
|
||||
it("does not wait for a Claude that says it needs a login: timed out as before", async () => {
|
||||
const { result, reads } = await run({ screen: () => "● Not logged in · Please run /login" });
|
||||
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(reads).toBe(17);
|
||||
});
|
||||
|
||||
@@ -1072,7 +1100,7 @@ describe("Claude Code runtime adapter", () => {
|
||||
// A trust prompt that shows up at read 22 ends the wait there; it is not a start in progress.
|
||||
const gate = ["Accessing workspace:", "/project", "❯ 1. Yes, I trust this folder", " 2. No, exit"].join("\n");
|
||||
const { result, reads } = await run({ screen: (read) => (read < 22 ? "" : gate) });
|
||||
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
|
||||
expect(reads).toBeLessThan(26);
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { assessNativeResumeProbe } from "../src/domain/native-resume-probe.js";
|
||||
import { resumeProbeEvidence } from "../src/domain/native-resume-probe-evidence.js";
|
||||
|
||||
const pane = { id: "%1", index: 0, cwd: "/", width: 80, height: 24, active: true };
|
||||
const assess = (paneContent: string) => assessNativeResumeProbe({ runtime: "claude-code", paneCommand: "sh", paneContent });
|
||||
|
||||
describe("bounded resume probe evidence (#1480)", () => {
|
||||
it.each(["Not logged in", "Login expired", "OAuth token revoked"])("records the fixed %s refusal and its row kind", async prefix => {
|
||||
for (const [marker, row] of [["●", "response"], ["⏺", "response"], [" ⎿", "result"]] as const) {
|
||||
const message = `${prefix} · Please run /login`;
|
||||
const content = `❯ fixture-secret-do-not-store\r\n${marker} ${message}\r\n\r\n❯\r\n`;
|
||||
const probe = assess(content);
|
||||
expect(probe).toMatchObject({ status: "failed", code: "login_required", loginRefusal: { message, row } });
|
||||
const evidence = await resumeProbeEvidence(probe, content, { listPanes: async () => [pane] }, "%1");
|
||||
expect(evidence).toEqual({ status: "failed", code: "login_required", loginRefusal: { message, row }, capture: {
|
||||
lines: 5, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"), columns: 80, rows: 24,
|
||||
} });
|
||||
expect(JSON.stringify(evidence)).not.toContain("fixture-secret-do-not-store");
|
||||
expect(evidence).not.toHaveProperty("detail");
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
"❯ Not logged in · Please run /login\n● working\n\n❯",
|
||||
"● Not logged in · Please run /login\n\n❯ next\n● working\n\n❯",
|
||||
"● Not logged in · Please run /login with fixture-secret-do-not-store\n\n❯",
|
||||
" ● Not logged in · Please run /login\n\n❯",
|
||||
])("does not assign a refusal to quoted, stale or non-exact text: %s", content => {
|
||||
expect(assess(content).code).not.toBe("login_required");
|
||||
expect(assess(content)).not.toHaveProperty("loginRefusal");
|
||||
});
|
||||
|
||||
it("hashes the exact captured bytes, including CRLF and trailing lines", async () => {
|
||||
const get = (content: string) => resumeProbeEvidence(assess(content), content, { listPanes: async () => [] }, "%1");
|
||||
expect((await get("a\r\nb\n")).capture.sha256).not.toBe((await get("a\nb\n")).capture.sha256);
|
||||
expect((await get("")).capture).toMatchObject({ lines: 0, bytes: 0, columns: null, rows: null });
|
||||
});
|
||||
|
||||
it("keeps the original result if geometry fails, is ambiguous, or belongs to another pane", async () => {
|
||||
const content = "● Login expired · Please run /login";
|
||||
const probe = assess(content);
|
||||
for (const listPanes of [
|
||||
vi.fn(async () => { throw new Error("fixture-secret-do-not-store"); }),
|
||||
vi.fn(async () => [{ ...pane, id: "%2" }]),
|
||||
vi.fn(async () => [{ ...pane, width: NaN, height: -1 }]),
|
||||
]) {
|
||||
expect(await resumeProbeEvidence(probe, content, { listPanes }, "%1")).toMatchObject({
|
||||
status: "failed", code: "login_required", capture: { columns: null, rows: null },
|
||||
});
|
||||
}
|
||||
expect((await resumeProbeEvidence(probe, content, { listPanes: async () => [pane, { ...pane, id: "%2" }] }, "seat")).capture)
|
||||
.toMatchObject({ columns: null, rows: null });
|
||||
});
|
||||
|
||||
it("keeps the probe when the geometry read times out", async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const content = "● Login expired · Please run /login";
|
||||
const result = resumeProbeEvidence(assess(content), content, {
|
||||
listPanes: () => new Promise<never>(() => {}),
|
||||
}, "%1");
|
||||
const returned = vi.fn();
|
||||
void result.then(returned);
|
||||
await vi.advanceTimersByTimeAsync(250);
|
||||
expect(returned).toHaveBeenCalledOnce();
|
||||
expect(await result).toMatchObject({
|
||||
status: "failed", code: "login_required",
|
||||
capture: { columns: null, rows: null, sha256: createHash("sha256").update(content).digest("hex") },
|
||||
});
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["resolve", "reject"] as const)("ignores a geometry read that later %ss after its deadline", async settlement => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
let resolveRead!: (panes: typeof pane[]) => void;
|
||||
let rejectRead!: (error: Error) => void;
|
||||
const read = new Promise<typeof pane[]>((resolve, reject) => { resolveRead = resolve; rejectRead = reject; });
|
||||
const content = "● Login expired · Please run /login";
|
||||
const result = resumeProbeEvidence(assess(content), content, { listPanes: () => read }, "%1");
|
||||
const returned = vi.fn();
|
||||
void result.then(returned);
|
||||
await vi.advanceTimersByTimeAsync(250);
|
||||
expect(returned).toHaveBeenCalledOnce();
|
||||
const evidence = await result;
|
||||
if (settlement === "resolve") resolveRead([pane]);
|
||||
else rejectRead(new Error("fixture-private-late-error"));
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
expect(evidence.capture).toMatchObject({ columns: null, rows: null });
|
||||
expect(returned).toHaveBeenCalledOnce();
|
||||
expect(JSON.stringify(evidence)).not.toContain("fixture-private");
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -35,6 +35,9 @@ import { buildRestorePlanPreview } from "../src/domain/restore-plan-preview.js";
|
||||
import { AppliedLaunchObservationStore } from "../src/domain/applied-launch-observation-store.js";
|
||||
import { observeClaudePermission, observeCodexSandbox, observePiResourceTrust, observeOmpApprovalMode, type AppliedLaunchObservation } from "../src/domain/permission-drift.js";
|
||||
import { SeatIdentityReconciler } from "../src/domain/seat-identity-reconciler.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { SeatAttentionReconciler } from "../src/domain/seat-attention-reconciler.js";
|
||||
import { AgentActivityStore } from "../src/domain/agent-activity-store.js";
|
||||
|
||||
function setupDb(): Database.Database {
|
||||
return createFullTestDb();
|
||||
@@ -3918,6 +3921,50 @@ describe("RestoreOrchestrator", () => {
|
||||
expect(tmux.sendText).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(["available", "error", "timeout"] as const)("persists the strict attention refusal probe with geometry=%s (#1480)", async geometry => {
|
||||
const missingGeometry = geometry !== "available";
|
||||
const tmux = mockTmuxForReconciler();
|
||||
const content = "❯ fixture-private-restore-prompt\n● OAuth token revoked · Please run /login\n\n❯";
|
||||
vi.mocked(tmux.hasSession).mockResolvedValue(true);
|
||||
vi.mocked(tmux.getPaneCommand).mockResolvedValue("sh");
|
||||
let captured = false;
|
||||
vi.mocked(tmux.capturePaneContent).mockImplementation(async () => { captured = true; return content; });
|
||||
// Identity resolves before capture; fail only the later diagnostic geometry read.
|
||||
const listPanes = tmux.listPanes;
|
||||
tmux.listPanes = async (...args) => {
|
||||
if (geometry === "timeout" && captured) return new Promise<never>(() => {});
|
||||
if (geometry === "error" && captured) throw new Error("fixture-private-geometry-error");
|
||||
return listPanes(...args);
|
||||
};
|
||||
const seeded = seedFailedAttempt({ restoreOutcome: "attention_required", withResumeToken: true });
|
||||
sessionRegistry.updateStartupStatus(sessionRegistry.getSessionsForRig(seeded.rig.id)[0]!.id, "ready");
|
||||
const before = db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").get();
|
||||
const attempt = db.prepare("SELECT seq FROM events WHERE type = 'restore.started'").get() as { seq: number };
|
||||
const restore = createOrchestrator({ tmux, listProcesses: async () => managedClaudeRows("tok-abc-123") });
|
||||
const reconciler = new SeatAttentionReconciler({ db, sessionRegistry, eventBus,
|
||||
agentActivityStore: new AgentActivityStore({ db, eventBus }),
|
||||
reconcileRestoreOutcome: (rigId, nodeId) => restore.reconcileNodeRuntimeTruth(rigId, nodeId),
|
||||
});
|
||||
const result = await reconciler.clearAttention(seeded.sessionName, { reason: "operator explanation" });
|
||||
expect(result).toMatchObject({ ok: false, code: "not_demonstrably_responsive",
|
||||
detail: expect.stringContaining("pane_not_usable: Pane state is failed (login_required)") });
|
||||
const events = db.prepare("SELECT payload FROM events WHERE type = 'seat.attention_clear_refused'").all() as { payload: string }[];
|
||||
expect(events).toHaveLength(1);
|
||||
expect(JSON.parse(events[0]!.payload)).toEqual({ type: "seat.attention_clear_refused", rigId: seeded.rig.id, nodeId: seeded.nodeId,
|
||||
sessionId: sessionRegistry.getSessionsForRig(seeded.rig.id)[0]!.id,
|
||||
sessionName: seeded.sessionName, attentionClass: "restore_outcome", code: "pane_not_usable", attemptId: attempt.seq,
|
||||
probe: { status: "failed", code: "login_required",
|
||||
loginRefusal: { message: "OAuth token revoked · Please run /login", row: "response" },
|
||||
capture: { lines: 4, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"),
|
||||
columns: missingGeometry ? null : 80, rows: missingGeometry ? null : 24 } },
|
||||
});
|
||||
expect(events[0]!.payload).not.toContain("fixture-private");
|
||||
expect(db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").get()).toEqual(before);
|
||||
expect(db.prepare("SELECT * FROM events WHERE type IN ('restore.outcome_reconciled', 'seat.attention_cleared')").all()).toHaveLength(0);
|
||||
expect(tmux.sendText).not.toHaveBeenCalled();
|
||||
expect(tmux.sendKeys).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
// #1077 — the seat launched with --resume tok-abc-123 and Claude continued as a new id. The
|
||||
// strict reconciler keeps main's exact current-token path and accepts the launch token only as
|
||||
// the alternative a qualified rotation names: the launched process's own first hook, a resume.
|
||||
|
||||
@@ -555,15 +555,29 @@ describe("SeatAttentionReconciler", () => {
|
||||
});
|
||||
|
||||
it("OPR.0.5.9.14: refuses derived-class clear without strict evidence even with operator attestation", async () => {
|
||||
seedDerivedAttentionSeat("r-derived-refuse", "worker@r-derived-refuse");
|
||||
const { rigId, nodeId, sessionId } = seedDerivedAttentionSeat("r-derived-refuse", "worker@r-derived-refuse");
|
||||
const before = db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").all();
|
||||
const sendVerify = vi.fn(async () => ({ ok: true }));
|
||||
const capture = vi.fn(async (sessionName: string) => ({ ok: true, sessionName, content: "fixture" }));
|
||||
const derivedReconciler = new SeatAttentionReconciler({
|
||||
sessionRegistry, eventBus, agentActivityStore: activityStore, db,
|
||||
sessionRegistry, eventBus, agentActivityStore: activityStore, db, sendVerify, capture,
|
||||
});
|
||||
|
||||
const result = await derivedReconciler.clearAttention("worker@r-derived-refuse", { reason: "operator verified" });
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.code).toBe("not_demonstrably_responsive");
|
||||
expect(result.detail).toContain("strict restore reconciler is unavailable");
|
||||
const refusals = db.prepare("SELECT payload FROM events WHERE type = 'seat.attention_clear_refused'").all() as { payload: string }[];
|
||||
expect(refusals).toHaveLength(1);
|
||||
expect(JSON.parse(refusals[0]!.payload)).toEqual({
|
||||
type: "seat.attention_clear_refused", rigId, nodeId, sessionId,
|
||||
sessionName: "worker@r-derived-refuse", attentionClass: "restore_outcome", code: "strict_reconciler_unavailable",
|
||||
});
|
||||
expect(db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").all()).toEqual(before);
|
||||
expect(db.prepare("SELECT payload FROM events WHERE type = 'seat.attention_cleared'").all()).toHaveLength(0);
|
||||
expect(sendVerify).not.toHaveBeenCalled();
|
||||
expect(capture).not.toHaveBeenCalled();
|
||||
const events = db.prepare("SELECT payload FROM events WHERE type = 'restore.outcome_reconciled'").all();
|
||||
expect(events).toHaveLength(0);
|
||||
});
|
||||
|
||||
@@ -16,6 +16,7 @@ import { AppliedLaunchObservationStore } from "../src/domain/applied-launch-obse
|
||||
import { observeClaudePermission } from "../src/domain/permission-drift.js";
|
||||
import type { SettingsStore } from "../src/domain/user-settings/settings-store.js";
|
||||
import { assessNativeResumeProbe } from "../src/domain/native-resume-probe.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { CodexRuntimeAdapter } from "../src/adapters/codex-runtime-adapter.js";
|
||||
import { ClaudeCodeAdapter } from "../src/adapters/claude-code-adapter.js";
|
||||
import { seatSkillPaths } from "../src/domain/seat-skill-storage.js";
|
||||
@@ -1475,6 +1476,43 @@ describe("StartupOrchestrator", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["available", "error", "timeout"] as const)("persists the real Claude launch probe on startup_failed with geometry=%s (#1480)", async geometry => {
|
||||
const missingGeometry = geometry !== "available";
|
||||
const seed = seedSession();
|
||||
const content = "❯ fixture-private-startup-prompt\n● Login expired · Please run /login\n\n❯";
|
||||
const tmux = mockTmux({
|
||||
getPaneCommand: vi.fn(async () => "claude"),
|
||||
capturePaneContent: vi.fn(async () => content),
|
||||
listPanes: vi.fn(async () => {
|
||||
if (geometry === "timeout") return new Promise<never>(() => {});
|
||||
if (geometry === "error") throw new Error("fixture-private-geometry-error");
|
||||
return [{ id: "%1", index: 0, cwd: "/project", width: 100, height: 30, active: true }];
|
||||
}),
|
||||
});
|
||||
const adapter = new ClaudeCodeAdapter({ tmux, sleep: async () => {}, fsOps: {
|
||||
readFile: () => { throw new Error("fixture file missing"); },
|
||||
writeFile: () => {}, exists: () => false, mkdirp: () => {}, copyFile: () => {}, listFiles: () => [],
|
||||
} });
|
||||
const result = await createOrchestrator(tmux).startNode(makeInput(seed, {
|
||||
adapter, binding: { ...makeBinding(), cwd: "/project" },
|
||||
isRestore: true, resumeToken: "saved-conversation", resumeType: "claude_id", allowFreshFallback: false,
|
||||
}));
|
||||
const probe = {
|
||||
status: "failed", code: "login_required", loginRefusal: { message: "Login expired · Please run /login", row: "response" },
|
||||
capture: { lines: 4, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"), columns: missingGeometry ? null : 100, rows: missingGeometry ? null : 30 },
|
||||
};
|
||||
expect(result).toMatchObject({ ok: false, startupStatus: "failed",
|
||||
errors: ["Harness launch failed: Claude resume failed: timed out waiting for Claude to become active"], probe });
|
||||
expect(sessionRegistry.getSessionsForRig(seed.rigId).find(s => s.id === seed.sessionId)?.startupStatus).toBe("failed");
|
||||
const events = db.prepare("SELECT payload FROM events WHERE type = 'node.startup_failed'").all() as { payload: string }[];
|
||||
expect(events).toHaveLength(1);
|
||||
expect(JSON.parse(events[0]!.payload)).toMatchObject({ rigId: seed.rigId, nodeId: seed.nodeId, sessionId: seed.sessionId,
|
||||
requestedResumeToken: "saved-conversation", stage: "harness_launch", probe });
|
||||
expect(JSON.parse(events[0]!.payload).probe).toEqual(probe);
|
||||
expect(events[0]!.payload).not.toContain("fixture-private");
|
||||
expect(JSON.stringify(result)).not.toContain("fixture-private-startup-prompt");
|
||||
});
|
||||
|
||||
// NS-T04: launchHarness persists resume token
|
||||
it("launchHarness resume token persisted to session", async () => {
|
||||
const seed = seedSession();
|
||||
|
||||
Reference in new issue
Block a user