feat(daemon): record probe evidence on failed Claude launches and refused attention clears (#1499)

A Claude seat whose resume launch failed reported a fixed message, and a refused `clear-attention` returned only the pane's status and code, so nobody could tell which probe result or which login text the decision rested on.

Now `node.startup_failed` and `seat.attention_clear_refused` carry the probe's evidence: its code, which of the fixed login-refusal strings matched and where, the capture's line and byte counts, a SHA-256 of the text the probe read, and the pane size. No pane text or free-form probe detail is stored. Every probe-backed failed return carries it, and a missing strict reconciler records its own refusal without inventing an attempt or evidence. Optional pane geometry has a 250 ms deadline, so a slow or failing read leaves the size empty and the failure is still recorded. Recovery decisions and error messages are unchanged.

Limits: the deadline bounds only the wait; it doesn't cancel the tmux command. The geometry is read after the capture, so it isn't an atomic snapshot. An unexpected reconciler exception keeps its existing error path and isn't recorded as a refusal.

Thanks to @Aschenbath for the PR, and to @AdamWhitehurst for the issue and for checking the PR against his case.

Cost: source +118/-17 (8 files), docs +33 (1 file), tests +237/-10 (5 files). No new refusal or error code; new fields on two existing events.
This commit is contained in:
Nash Aschenbath authored and GitHub committed 2026-10-22 20:45:27 -07:00
1 parent 1c186e7640
commit f9f843bf62
14 files changed
+388 -27

No files matched your search

@@ -88,6 +88,39 @@ Readiness has its own attention codes (`runtime-adapter.ts:74–87`):
When `checkReady` reports one, the readiness wait returns at once and startup
ends `attention_required`.
### Evidence for a failed Claude resume
Previously, a failed Claude launch recorded its error without the probe code or
the captured input's identity, and a refused strict full-restore attention clear
had no corresponding refusal event. Probe-backed Claude launch failures now
carry an optional `probe` in `HarnessLaunchResult` and `node.startup_failed`.
Strict full-restore refusals emit `seat.attention_clear_refused`; a
`pane_not_usable` refusal includes that probe and the restore attempt ID.
A missing strict reconciler emits `strict_reconciler_unavailable` without
inventing an attempt ID or probe evidence, and still refuses the clear.
`native-resume-probe-evidence.ts` records the status, code, a closed-set login
refusal and row kind when matched, and the exact capture's line count, byte
count and SHA-256. Its new fields contain no captured pane text or free-form
probe detail. Existing attention evidence and recovery decisions keep their
previous behavior. MCP approval detection still takes precedence over the
login-refusal check.
Pane dimensions are optional, observed after the capture. Their read has a
250 ms deadline so a hung `listPanes` cannot indefinitely delay the failure
result or its event. On timeout, error, ambiguous pane identity or invalid
dimensions, the unavailable dimensions are `null`; late results are ignored.
The deadline bounds this diagnostic wait, not the underlying tmux command.
`native-resume-probe-evidence.test.ts` covers unavailable and late geometry;
`startup-orchestrator.test.ts` and `restore-orchestrator.test.ts` exercise the
real adapter/orchestrator and SQLite event paths with available, failing and
never-settling geometry reads. `seat-attention-reconciler.test.ts` checks that
a missing strict reconciler records its refusal without changing restore
history or sending input. These use synthetic terminal captures, not a
live Claude session. The stub scenario runtime cannot draw Claude's native
login-refusal UI, and its scenario runner does not bind the restore action.
## 2. The runtime adapters
**5** classes implement `RuntimeAdapter`
@@ -13,6 +13,7 @@ import type {
import { resolveConcreteHint } from "../domain/runtime-adapter.js";
import type { ProjectionPlan, ProjectionEntry } from "../domain/projection-planner.js";
import { assessNativeResumeProbe } from "../domain/native-resume-probe.js";
import { resumeProbeEvidence } from "../domain/native-resume-probe-evidence.js";
import { resolveReadinessTimeoutMs } from "../domain/readiness-timeout.js";
import { subtreeNamesClaudeExecutable, claudeCommandSessionToken, listNativeProcesses, observeClaudeDelivery, observeClaudePaneProcess, observeClaudePaneRuntime, observeCodexPaneProcess, verifyClaudePaneProcess, verifyCodexPaneProcess, verifyPaneIdleShell, witnessedClaudeVersionLabel, type ClaudeLaunchedProcess, type NativeProcessLister, type VersionLabelWitnessWait } from "../domain/native-process-lineage.js";
import { mergeManagedBlock, DEFAULT_CLAUDE_MANAGED_BLOCK_FILE, type ClaudeManagedBlockFile } from "../domain/managed-blocks.js";
@@ -604,6 +605,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
if (probe.code === "no_conversation_found") {
return {
ok: false,
probe: await resumeProbeEvidence(probe, paneContent, this.tmux, tmuxSession),
error: "Claude resume failed: no conversation found for the requested session",
recovery: "retry_fresh",
missingConversation: true,
@@ -620,6 +622,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
if (probe.status === "attention_required") {
return {
ok: false,
probe: await resumeProbeEvidence(probe, paneContent, this.tmux, tmuxSession),
error: probe.detail,
recovery: "attention_required",
evidence: paneContent.split("\n").slice(-12).join("\n"),
@@ -635,7 +638,8 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
}
const enterResult = await this.tmux.sendKeys(tmuxSession, ["Enter"]);
if (!enterResult.ok) {
return { ok: false, error: `Claude trust prompt auto-drive failed: ${enterResult.message}` };
return { ok: false, error: `Claude trust prompt auto-drive failed: ${enterResult.message}`,
probe: await resumeProbeEvidence(probe, paneContent, this.tmux, tmuxSession) };
}
await this.sleep(200);
continue;
@@ -688,10 +692,12 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
if (finalProbe.status === "resumed") {
return { ok: true };
}
const probeEvidence = await resumeProbeEvidence(finalProbe, finalContent, this.tmux, tmuxSession);
if (finalProbe.status === "attention_required" || finalProbe.code === "claude_auto_identity_required") {
return {
ok: false,
probe: probeEvidence,
error: finalProbe.detail,
recovery: "attention_required",
evidence: finalContent.split("\n").slice(-12).join("\n"),
@@ -702,6 +708,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
if (finalProbe.code === "no_conversation_found") {
return {
ok: false,
probe: probeEvidence,
error: "Claude resume failed: no conversation found for the requested session",
recovery: "retry_fresh",
missingConversation: true,
@@ -711,6 +718,7 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
if (await this.paneReturnedToShell(binding, finalCommand)) {
return {
ok: false,
probe: probeEvidence,
error: "Claude resume failed: pane returned to shell instead of entering Claude",
recovery: "retry_fresh",
};
@@ -726,11 +734,12 @@ export class ClaudeCodeAdapter implements RuntimeAdapter {
return {
ok: false,
error: `Claude resume did not finish within ${Math.round(attempts * pollMs / 1000)}s; Claude may still be starting, so the launch was kept`,
probe: probeEvidence,
recovery: "attention_required",
evidence: finalContent.split("\n").slice(-12).join("\n"),
};
}
return { ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" };
return { ok: false, error: "Claude resume failed: timed out waiting for Claude to become active", probe: probeEvidence };
}
private projectEntry(entry: ProjectionEntry, cwd: string, managedBlockFile: ClaudeManagedBlockFile, warnings: string[], seatSkillsRoot?: string, seatSkillsRefused?: string): boolean {
@@ -0,0 +1,56 @@
import { createHash } from "node:crypto";
import type { TmuxAdapter, TmuxPane } from "../adapters/tmux.js";
import type { NativeResumeProbeResult } from "./native-resume-probe.js";
const GEOMETRY_TIMEOUT_MS = 250;
/** Bounded metadata from the exact capture assessed by the probe; never its text or detail. */
export interface NativeResumeProbeEvidence {
status: NativeResumeProbeResult["status"];
code: string;
loginRefusal?: NonNullable<NativeResumeProbeResult["loginRefusal"]>;
capture: {
lines: number;
bytes: number;
sha256: string;
/** Geometry observed after the capture; null when unavailable within the diagnostic deadline. */
columns: number | null;
rows: number | null;
};
}
/** Read geometry best-effort, without re-capturing or changing the probe's decision. */
export async function resumeProbeEvidence(
probe: NativeResumeProbeResult,
content: string,
tmux: Pick<TmuxAdapter, "listPanes">,
target: string,
): Promise<NativeResumeProbeEvidence> {
let columns: number | null = null;
let rows: number | null = null;
let timeout: ReturnType<typeof setTimeout> | undefined;
const dimension = (value: number | undefined) => Number.isSafeInteger(value) && value! > 0 ? value! : null;
try {
// Geometry is optional: a stuck tmux read must not hold up an already-decided failure or its event.
const panes = await Promise.race([
tmux.listPanes(target),
new Promise<TmuxPane[]>(resolve => { timeout = setTimeout(() => resolve([]), GEOMETRY_TIMEOUT_MS); }),
]);
const pane = /^%\d+$/.test(target) ? panes.find(pane => pane.id === target) : panes.length === 1 ? panes[0] : undefined;
columns = dimension(pane?.width);
rows = dimension(pane?.height);
} catch { /* Diagnostics must not replace the original failure when geometry is unavailable. */ }
finally { if (timeout !== undefined) clearTimeout(timeout); }
return {
status: probe.status,
code: probe.code,
...(probe.loginRefusal ? { loginRefusal: probe.loginRefusal } : {}),
capture: {
lines: content.length ? content.split(/\r?\n/).length : 0,
bytes: Buffer.byteLength(content),
sha256: createHash("sha256").update(content).digest("hex"),
columns,
rows,
},
};
}
@@ -21,6 +21,7 @@ export interface NativeResumeProbeResult {
status: NativeResumeProbeStatus;
code: string;
detail: string;
loginRefusal?: { message: ClaudeLoginRefusal; row: "response" | "result" };
}
export interface ProbeShellReadyInput {
@@ -127,11 +128,13 @@ export function assessNativeResumeProbe(
detail: "Claude is waiting for project MCP server approval before the session can become interactive.",
};
}
if (hasCurrentClaudeLoginRefusal(paneContent)) {
const loginRefusal = currentClaudeLoginRefusal(paneContent);
if (loginRefusal) {
return {
status: "failed",
code: "login_required",
detail: "Claude is running but cannot continue until the user logs in.",
loginRefusal,
};
}
if (looksLikeClaudeTui(paneContent)) {
@@ -334,11 +337,12 @@ function looksLikeClaudeResumeSelectionPrompt(paneContent: string): boolean {
// Claude's interactive refusals of a request for a missing, expired or revoked login (code.claude.com/docs/en/errors).
// Only these exact messages count. A 401 reads "Please run /login · API Error: 401 …" with a varying tail, and a
// reply or tool output can start with the same words, so this input can't tell a real one apart.
const CLAUDE_LOGIN_REFUSALS = new Set([
const CLAUDE_LOGIN_REFUSALS = [
"Not logged in · Please run /login",
"Login expired · Please run /login",
"OAuth token revoked · Please run /login",
]);
] as const;
type ClaudeLoginRefusal = typeof CLAUDE_LOGIN_REFUSALS[number];
// A refusal blocks only as Claude's latest response. The persistent "Not logged in · Run /login" footer says only
// that Claude holds no credential of its own, and a gateway can still authenticate its requests. Claude starts a
@@ -346,12 +350,12 @@ const CLAUDE_LOGIN_REFUSALS = new Set([
// row; an indented marker is quoted text. The last ❯ row is the input box only when no response follows it;
// otherwise it is a submitted prompt and no box is drawn. A refusal counts when it is the last response after the
// last submitted prompt, so older scrollback, quoted text and a refusal followed by a newer reply or prompt do not.
function hasCurrentClaudeLoginRefusal(paneContent: string): boolean {
function currentClaudeLoginRefusal(paneContent: string): NativeResumeProbeResult["loginRefusal"] {
const lines = paneContent.split(/\r?\n/);
const isPrompt = (line: string) => /^\s*❯/.test(line);
const isResponse = (line: string) => /^[●⏺]/.test(line) || /^ {2}⎿/.test(line);
const prompts: number[] = [];
const responses: Array<{ at: number; text: string }> = [];
const responses: Array<{ at: number; text: string; row: "response" | "result" }> = [];
for (let at = 0; at < lines.length; at++) {
const line = lines[at]!;
if (isPrompt(line)) prompts.push(at);
@@ -360,13 +364,14 @@ function hasCurrentClaudeLoginRefusal(paneContent: string): boolean {
// Claude wraps its own rows with a short hanging indent; a right-aligned footer is not a continuation.
for (let next = at + 1; next < lines.length && /^ {1,6}\S/.test(lines[next]!) && !isPrompt(lines[next]!)
&& !isResponse(lines[next]!); next++) text += ` ${lines[next]}`;
responses.push({ at, text: text.replace(/\s+/g, " ").trim() });
responses.push({ at, text: text.replace(/\s+/g, " ").trim(), row: /^ {2}⎿/.test(line) ? "result" : "response" });
}
const lastPrompt = prompts.at(-1) ?? -1;
const inputBox = responses.some(({ at }) => at > lastPrompt) ? lines.length : lastPrompt;
const turnStart = prompts.filter((at) => at < inputBox).at(-1) ?? -1;
const latest = responses.filter(({ at }) => at > turnStart && at < inputBox).at(-1)?.text;
return latest !== undefined && CLAUDE_LOGIN_REFUSALS.has(latest);
const latest = responses.filter(({ at }) => at > turnStart && at < inputBox).at(-1);
const message = CLAUDE_LOGIN_REFUSALS.find(message => message === latest?.text);
return message && latest ? { message, row: latest.row } : undefined;
}
function looksLikeClaudeMcpApprovalPrompt(paneContent: string): boolean {
@@ -24,6 +24,7 @@ import type { PiResumeAdapter } from "../adapters/pi-resume.js";
import type { OmpResumeAdapter } from "../adapters/omp-resume.js";
import type { TranscriptStore } from "./transcript-store.js";
import { assessNativeResumeProbe } from "./native-resume-probe.js";
import { resumeProbeEvidence, type NativeResumeProbeEvidence } from "./native-resume-probe-evidence.js";
import { observeClaudeDelivery, observeClaudeResumeLaunch, verifyClaudePaneProcess, verifyPaneIdleShell, type ClaudeLaunchedProcess } from "./native-process-lineage.js";
import { reanchorBuiltinStartupFile, reanchorShippedProjectionEntry } from "./builtin-startup-files.js";
import { seatGuidance, restoreMissingGuidance, missingGuidanceBlocks } from "./restore-guidance.js";
@@ -72,6 +73,8 @@ export type ReconcileNodeResult =
| "resume_token_not_used"
| "pane_not_usable";
detail: string;
probe?: NativeResumeProbeEvidence;
attemptId?: number;
};
// Only these edge kinds constrain launch order
@@ -2166,12 +2169,13 @@ export class RestoreOrchestrator {
// Precondition #4: pane is at a usable/idle state — explicitly NOT a
// resume-selection prompt and not the "returned to shell" failure mode.
if (probe.status !== "resumed") {
const evidence = await resumeProbeEvidence(probe, paneContent, this.tmuxAdapter, identity.pane);
// The label reads as a shell, but the runtime's lineage was just proven in this pane: name a shell only for a proven idle one.
if (probe.code === "returned_to_shell" && !await verifyPaneIdleShell({ target: identity.pane, tmux: this.tmuxAdapter,
...(this.listProcesses ? { listProcesses: this.listProcesses } : {}) })) {
return { ok: false, code: "pane_not_usable", detail: "Pane is not shown to be resumed: its command reads as a shell, but it is not a proven idle shell (the runtime may be starting or showing a dialog); reconciliation requires resumed." };
return { ok: false, code: "pane_not_usable", probe: evidence, attemptId, detail: "Pane is not shown to be resumed: its command reads as a shell, but it is not a proven idle shell (the runtime may be starting or showing a dialog); reconciliation requires resumed." };
}
return { ok: false, code: "pane_not_usable", detail: `Pane state is ${probe.status} (${probe.code}); reconciliation requires resumed.` };
return { ok: false, code: "pane_not_usable", probe: evidence, attemptId, detail: `Pane state is ${probe.status} (${probe.code}); reconciliation requires resumed.` };
}
// All four preconditions hold. Append (never mutate) the audit event.
@@ -128,6 +128,7 @@ export type HarnessLaunchResult =
// conversation doesn't exist; other `retry_fresh` failures (such as a pane back at its shell)
// leave it unset.
| { ok: false; error: string; recovery?: HarnessLaunchRecovery; evidence?: string; missingConversation?: true;
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence;
/** The native resume probe's code for the startup gate that stopped the launch (for example `hook_trust_gate`),
* when one did. Reporting only: it changes no recovery decision. */
gate?: string;
@@ -377,7 +377,8 @@ interface ClearAttentionDeps {
to: "operator_recovered";
evidence: { tmux: boolean; fgProcess: string; resumeTokenUsed: boolean; paneState: "usable" };
}
| { ok: false; code: string; detail: string }
| { ok: false; code: string; detail: string; attemptId?: number;
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence }
>;
}
@@ -426,6 +427,15 @@ export class SeatAttentionReconciler {
if (derivedOutcome?.source === "restore.completed") {
const reconcile = this.deps.reconcileRestoreOutcome;
if (!reconcile) {
eventBus.emit({
type: "seat.attention_clear_refused",
rigId: session.rigId,
nodeId: session.nodeId,
sessionId: session.id,
sessionName,
attentionClass: "restore_outcome",
code: "strict_reconciler_unavailable",
});
return {
ok: false,
code: "not_demonstrably_responsive",
@@ -434,6 +444,17 @@ export class SeatAttentionReconciler {
}
const restored = await reconcile(session.rigId, session.nodeId);
if (!restored.ok) {
eventBus.emit({
type: "seat.attention_clear_refused",
rigId: session.rigId,
nodeId: session.nodeId,
sessionId: session.id,
sessionName,
attentionClass: "restore_outcome",
code: restored.code,
...(restored.attemptId !== undefined ? { attemptId: restored.attemptId } : {}),
...(restored.probe ? { probe: restored.probe } : {}),
});
return {
ok: false,
code: "not_demonstrably_responsive",
@@ -240,7 +240,8 @@ export type StartupResult = { warnings?: string[] } & (
// outcomes so restore-orchestrator's per-node mapping can populate
// `attentionEvidence` on the RestoreNodeResult. Internal type only;
// not persisted on the failure event.
| { ok: false; startupStatus: "attention_required" | "failed"; errors: string[]; evidence?: string; recovery?: string; missingConversation?: true });
| { ok: false; startupStatus: "attention_required" | "failed"; errors: string[]; evidence?: string; recovery?: string; missingConversation?: true;
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence });
interface StartupOrchestratorDeps {
db: Database.Database;
@@ -634,11 +635,11 @@ export class StartupOrchestrator {
// unverified fork keeps its unsent context for a verified continuation.
return this.fail(input, "attention_required", errors, launchResult.evidence,
launchResult.forkIdentityPending ? forkContext ?? false : continuityOutcome === "fresh",
undefined, false, "harness_launch", true);
undefined, false, "harness_launch", true, launchResult.probe);
}
errors.push(`Harness launch failed: ${launchResult.error}`);
return this.fail(input, "failed", errors, undefined, false, launchResult.recovery, launchResult.missingConversation === true, "harness_launch", true);
return this.fail(input, "failed", errors, undefined, false, launchResult.recovery, launchResult.missingConversation === true, "harness_launch", true, launchResult.probe);
}
} catch (err) {
errors.push(`Harness launch error: ${(err as Error).message}`);
@@ -948,6 +949,7 @@ export class StartupOrchestrator {
missingConversation = false,
stage?: StartupFailureStage,
hasPendingWork?: boolean,
probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence,
): StartupResult {
if (status === "attention_required" && typeof pendingContext === "object") {
// The reasons above stay visible; this only says what the status means for the fork.
@@ -970,8 +972,10 @@ export class StartupOrchestrator {
...(hasPendingWork !== undefined ? { hasPendingWork } : {}),
requestedResumeToken: input.resumeToken ?? null,
requestedResumeType: input.resumeType ?? null,
...(probe ? { probe } : {}),
});
return { ok: false, startupStatus: status, errors, evidence, ...(recovery ? { recovery } : {}),
...(probe ? { probe } : {}),
...(missingConversation ? { missingConversation: true as const } : {}) };
}
+2 -1
View File
@@ -193,6 +193,7 @@ export type RigEvent =
// 2026-07-02 founder ruling).
| { type: "session.resume_token_captured"; rigId: string; nodeId: string; sessionName: string; sessionId: string; runtime: string; outcome: "captured" | "preserved" | "skipped"; resumeType?: string; provenance?: "adoption"; reason?: "missing_sidecar" | "parse_error" | "probe_timeout" | "process_unreadable" | "invalid_token" | "stale_sidecar" | "higher_rank_present"; redacted: true }
| { type: "seat.attention_cleared"; rigId: string; nodeId: string; sessionName: string; from: string; to: "ready"; clearedBy: "evidence" | "operator_attestation"; evidence?: { kind: string; state?: string; reason?: string }; reason?: string; previousError: string | null }
| { type: "seat.attention_clear_refused"; rigId: string; nodeId: string; sessionId: string; sessionName: string; attentionClass: "restore_outcome"; code: string; attemptId?: number; probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence }
| { type: "rig.imported"; rigId: string; specName: string; specVersion: string }
// Package events (cross-rig, no rigId)
| { type: "package.validated"; packageName: string; valid: boolean }
@@ -233,7 +234,7 @@ export type RigEvent =
| { type: "pod.deleted"; rigId: string; podId: string }
| { type: "node.startup_pending"; rigId: string; nodeId: string; startupProof?: StartupProofSelection }
| { type: "node.startup_ready"; rigId: string; nodeId: string; previousStatus?: "failed" | "attention_required"; submission?: { status: "unverified" | "staged"; reasons: string[]; warning?: string; diagnostics?: StartupSubmissionDiagnostic[] } }
| { type: "node.startup_failed"; rigId: string; nodeId: string; error: string; sessionId?: string; freshContextPending?: boolean; forkContextPending?: { parentToken: string; isRestore: boolean }; submissionDiagnostics?: StartupSubmissionDiagnostic[]; stage?: StartupFailureStage; hasPendingWork?: boolean; requestedResumeToken?: string | null; requestedResumeType?: string | null }
| { type: "node.startup_failed"; rigId: string; nodeId: string; error: string; sessionId?: string; freshContextPending?: boolean; forkContextPending?: { parentToken: string; isRestore: boolean }; submissionDiagnostics?: StartupSubmissionDiagnostic[]; stage?: StartupFailureStage; hasPendingWork?: boolean; requestedResumeToken?: string | null; requestedResumeType?: string | null; probe?: import("./native-resume-probe-evidence.js").NativeResumeProbeEvidence }
// OPR.0.4.3.06 — startup proof (challenge-verified orientation). Append-only.
// `node.startup_challenged` freezes this launch's challenge ground truth
// (challengeId + contractHash; the expected answer is recomputed, never
@@ -1,3 +1,4 @@
import { createHash } from "node:crypto";
import { execFileSync } from "node:child_process";
import fs from "node:fs";
import os from "node:os";
@@ -11,6 +12,11 @@ import { claudePostureFlag } from "../src/adapters/yolo-mode.js";
import { resolveReadinessTimeoutMs } from "../src/domain/readiness-timeout.js";
import { claudeCommandSessionToken } from "../src/domain/native-process-lineage.js";
const probeEvidence = expect.objectContaining({
status: expect.any(String), code: expect.any(String),
capture: { lines: expect.any(Number), bytes: expect.any(Number), sha256: expect.stringMatching(/^[a-f0-9]{64}$/), columns: null, rows: null },
});
function mockTmux(): TmuxAdapter {
return {
sendText: vi.fn(async () => ({ ok: true as const })),
@@ -721,6 +727,21 @@ describe("Claude Code runtime adapter", () => {
);
});
it("returns diagnostics from the captured login refusal without changing the timeout outcome", async () => {
const tmux = mockTmux();
const content = "❯ fixture-private-prompt\n● Login expired · Please run /login\n\n❯";
vi.mocked(tmux.capturePaneContent).mockResolvedValue(content);
vi.mocked(tmux.listPanes).mockResolvedValue([{ id: "%1", index: 0, cwd: "/project", width: 100, height: 30, active: true }]);
const adapter = new ClaudeCodeAdapter({ tmux, fsOps: mockFs(), sleep: async () => {} });
const result = await adapter.launchHarness(makeBinding(), { name: "dev-impl@test-rig", resumeToken: "abc-123" });
expect(result).toEqual({
ok: false, error: "Claude resume failed: timed out waiting for Claude to become active",
probe: { status: "failed", code: "login_required", loginRefusal: { message: "Login expired · Please run /login", row: "response" },
capture: { lines: 4, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"), columns: 100, rows: 30 } },
});
expect(JSON.stringify(result)).not.toContain("fixture-private-prompt");
});
it("launchHarness returns retry_fresh when Claude reports no conversation found for the requested resume token", async () => {
const tmux = mockTmux();
(tmux.getPaneCommand as ReturnType<typeof vi.fn>).mockResolvedValue("zsh");
@@ -733,6 +754,7 @@ describe("Claude Code runtime adapter", () => {
expect(result).toEqual({
ok: false,
probe: probeEvidence,
error: "Claude resume failed: no conversation found for the requested session",
recovery: "retry_fresh",
missingConversation: true,
@@ -753,6 +775,7 @@ describe("Claude Code runtime adapter", () => {
expect(capture).toHaveBeenCalledTimes(17);
expect(result).toEqual({
ok: false,
probe: probeEvidence,
error: "Claude resume failed: no conversation found for the requested session",
recovery: "retry_fresh",
missingConversation: true,
@@ -769,6 +792,7 @@ describe("Claude Code runtime adapter", () => {
expect(result).toEqual({
ok: false,
probe: probeEvidence,
error: "Claude resume failed: pane returned to shell instead of entering Claude",
recovery: "retry_fresh",
});
@@ -814,6 +838,7 @@ describe("Claude Code runtime adapter", () => {
const { result, reads } = await run({ screen: () => "something Claude draws that no probe knows" });
expect(result).toEqual({
ok: false,
probe: probeEvidence,
error: "Claude resume did not finish within 6s; Claude may still be starting, so the launch was kept",
recovery: "attention_required",
evidence: "something Claude draws that no probe knows",
@@ -848,6 +873,7 @@ describe("Claude Code runtime adapter", () => {
const { result } = await run({ screen: (read) => (read < 22 ? "" : "No conversation found with session ID: abc-123") });
expect(result).toEqual({
ok: false,
probe: probeEvidence,
error: "Claude resume failed: no conversation found for the requested session",
recovery: "retry_fresh",
missingConversation: true,
@@ -867,6 +893,7 @@ describe("Claude Code runtime adapter", () => {
const { result } = await run({ rows: (read) => (read < 25 ? occupied : [pane]) });
expect(result).toEqual({
ok: false,
probe: probeEvidence,
error: "Claude resume failed: pane returned to shell instead of entering Claude",
recovery: "retry_fresh",
});
@@ -877,6 +904,7 @@ describe("Claude Code runtime adapter", () => {
const { result, reads } = await run({ rows: (read) => (read < 19 ? occupied : [pane]) });
expect(result).toEqual({
ok: false,
probe: probeEvidence,
error: "Claude resume failed: pane returned to shell instead of entering Claude",
recovery: "retry_fresh",
});
@@ -911,9 +939,9 @@ describe("Claude Code runtime adapter", () => {
expect((await run({ readinessTimeoutMs, rows: () => [pane] })).result).toMatchObject({ ok: false, recovery: "retry_fresh" });
const other = { ...claude, command: "claude --permission-mode acceptEdits --resume some-other-session --name other" };
expect((await run({ readinessTimeoutMs, rows: () => [{ ...pane, tpgid: 11 }, launchScript, other] })).result)
.toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
.toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect((await run({ readinessTimeoutMs, rows: () => stuck, screen: () => "● Not logged in · Please run /login" })).result)
.toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
.toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
});
});
@@ -930,7 +958,7 @@ describe("Claude Code runtime adapter", () => {
});
describe("a different occupant keeps main's result (timed out, no token): only unknown observations get attention", () => {
const timedOut = { ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" };
const timedOut = { ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" };
const codex = { pid: 13, ppid: 11, pgid: 11, tpgid: 11, executableName: "codex", startedAt, command: "codex resume some-codex-session" };
const otherClaude = { ...claude, command: "claude --permission-mode acceptEdits --resume some-other-session --name other" };
const booting = [{ ...pane, tpgid: 11 }, launchScript];
@@ -1037,7 +1065,7 @@ describe("Claude Code runtime adapter", () => {
it("does not wait for a Claude that names another conversation: another occupant, timed out as before", async () => {
const other = { ...claude, command: "claude --permission-mode acceptEdits --resume some-other-session --name other" };
const { result, reads } = await run({ rows: () => [{ ...pane, tpgid: 11 }, launchScript, other] });
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(reads).toBe(17);
});
@@ -1046,7 +1074,7 @@ describe("Claude Code runtime adapter", () => {
const lookAlike = { ...claude, executableName: "printf" };
const ready = "Restored conversation\n❯\u00a0\n ⏵⏵ bypass permissions on (shift+tab to cycle)";
const { result, reads } = await run({ rows: () => [{ ...pane, tpgid: 11 }, launchScript, lookAlike], screen: () => ready });
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(reads).toBe(17);
});
@@ -1058,13 +1086,13 @@ describe("Claude Code runtime adapter", () => {
rows: (read) => (read < 20 ? [{ ...pane, tpgid: 11 }, launchScript] : [{ ...pane, tpgid: 11 }, launchScript, lookAlike]),
screen: (read) => (read < 20 ? "" : ready),
});
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(reads).toBeLessThan(25);
});
it("does not wait for a Claude that says it needs a login: timed out as before", async () => {
const { result, reads } = await run({ screen: () => "● Not logged in · Please run /login" });
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(reads).toBe(17);
});
@@ -1072,7 +1100,7 @@ describe("Claude Code runtime adapter", () => {
// A trust prompt that shows up at read 22 ends the wait there; it is not a start in progress.
const gate = ["Accessing workspace:", "/project", "❯ 1. Yes, I trust this folder", " 2. No, exit"].join("\n");
const { result, reads } = await run({ screen: (read) => (read < 22 ? "" : gate) });
expect(result).toEqual({ ok: false, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(result).toEqual({ ok: false, probe: probeEvidence, error: "Claude resume failed: timed out waiting for Claude to become active" });
expect(reads).toBeLessThan(26);
});
@@ -0,0 +1,100 @@
import { createHash } from "node:crypto";
import { describe, expect, it, vi } from "vitest";
import { assessNativeResumeProbe } from "../src/domain/native-resume-probe.js";
import { resumeProbeEvidence } from "../src/domain/native-resume-probe-evidence.js";
const pane = { id: "%1", index: 0, cwd: "/", width: 80, height: 24, active: true };
const assess = (paneContent: string) => assessNativeResumeProbe({ runtime: "claude-code", paneCommand: "sh", paneContent });
describe("bounded resume probe evidence (#1480)", () => {
it.each(["Not logged in", "Login expired", "OAuth token revoked"])("records the fixed %s refusal and its row kind", async prefix => {
for (const [marker, row] of [["●", "response"], ["⏺", "response"], [" ⎿", "result"]] as const) {
const message = `${prefix} · Please run /login`;
const content = `❯ fixture-secret-do-not-store\r\n${marker} ${message}\r\n\r\n❯\r\n`;
const probe = assess(content);
expect(probe).toMatchObject({ status: "failed", code: "login_required", loginRefusal: { message, row } });
const evidence = await resumeProbeEvidence(probe, content, { listPanes: async () => [pane] }, "%1");
expect(evidence).toEqual({ status: "failed", code: "login_required", loginRefusal: { message, row }, capture: {
lines: 5, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"), columns: 80, rows: 24,
} });
expect(JSON.stringify(evidence)).not.toContain("fixture-secret-do-not-store");
expect(evidence).not.toHaveProperty("detail");
}
});
it.each([
"❯ Not logged in · Please run /login\n● working\n\n❯",
"● Not logged in · Please run /login\n\n❯ next\n● working\n\n❯",
"● Not logged in · Please run /login with fixture-secret-do-not-store\n\n❯",
" ● Not logged in · Please run /login\n\n❯",
])("does not assign a refusal to quoted, stale or non-exact text: %s", content => {
expect(assess(content).code).not.toBe("login_required");
expect(assess(content)).not.toHaveProperty("loginRefusal");
});
it("hashes the exact captured bytes, including CRLF and trailing lines", async () => {
const get = (content: string) => resumeProbeEvidence(assess(content), content, { listPanes: async () => [] }, "%1");
expect((await get("a\r\nb\n")).capture.sha256).not.toBe((await get("a\nb\n")).capture.sha256);
expect((await get("")).capture).toMatchObject({ lines: 0, bytes: 0, columns: null, rows: null });
});
it("keeps the original result if geometry fails, is ambiguous, or belongs to another pane", async () => {
const content = "● Login expired · Please run /login";
const probe = assess(content);
for (const listPanes of [
vi.fn(async () => { throw new Error("fixture-secret-do-not-store"); }),
vi.fn(async () => [{ ...pane, id: "%2" }]),
vi.fn(async () => [{ ...pane, width: NaN, height: -1 }]),
]) {
expect(await resumeProbeEvidence(probe, content, { listPanes }, "%1")).toMatchObject({
status: "failed", code: "login_required", capture: { columns: null, rows: null },
});
}
expect((await resumeProbeEvidence(probe, content, { listPanes: async () => [pane, { ...pane, id: "%2" }] }, "seat")).capture)
.toMatchObject({ columns: null, rows: null });
});
it("keeps the probe when the geometry read times out", async () => {
vi.useFakeTimers();
try {
const content = "● Login expired · Please run /login";
const result = resumeProbeEvidence(assess(content), content, {
listPanes: () => new Promise<never>(() => {}),
}, "%1");
const returned = vi.fn();
void result.then(returned);
await vi.advanceTimersByTimeAsync(250);
expect(returned).toHaveBeenCalledOnce();
expect(await result).toMatchObject({
status: "failed", code: "login_required",
capture: { columns: null, rows: null, sha256: createHash("sha256").update(content).digest("hex") },
});
} finally {
vi.useRealTimers();
}
});
it.each(["resolve", "reject"] as const)("ignores a geometry read that later %ss after its deadline", async settlement => {
vi.useFakeTimers();
try {
let resolveRead!: (panes: typeof pane[]) => void;
let rejectRead!: (error: Error) => void;
const read = new Promise<typeof pane[]>((resolve, reject) => { resolveRead = resolve; rejectRead = reject; });
const content = "● Login expired · Please run /login";
const result = resumeProbeEvidence(assess(content), content, { listPanes: () => read }, "%1");
const returned = vi.fn();
void result.then(returned);
await vi.advanceTimersByTimeAsync(250);
expect(returned).toHaveBeenCalledOnce();
const evidence = await result;
if (settlement === "resolve") resolveRead([pane]);
else rejectRead(new Error("fixture-private-late-error"));
await vi.advanceTimersByTimeAsync(0);
expect(evidence.capture).toMatchObject({ columns: null, rows: null });
expect(returned).toHaveBeenCalledOnce();
expect(JSON.stringify(evidence)).not.toContain("fixture-private");
} finally {
vi.useRealTimers();
}
});
});
@@ -35,6 +35,9 @@ import { buildRestorePlanPreview } from "../src/domain/restore-plan-preview.js";
import { AppliedLaunchObservationStore } from "../src/domain/applied-launch-observation-store.js";
import { observeClaudePermission, observeCodexSandbox, observePiResourceTrust, observeOmpApprovalMode, type AppliedLaunchObservation } from "../src/domain/permission-drift.js";
import { SeatIdentityReconciler } from "../src/domain/seat-identity-reconciler.js";
import { createHash } from "node:crypto";
import { SeatAttentionReconciler } from "../src/domain/seat-attention-reconciler.js";
import { AgentActivityStore } from "../src/domain/agent-activity-store.js";
function setupDb(): Database.Database {
return createFullTestDb();
@@ -3918,6 +3921,50 @@ describe("RestoreOrchestrator", () => {
expect(tmux.sendText).not.toHaveBeenCalled();
});
it.each(["available", "error", "timeout"] as const)("persists the strict attention refusal probe with geometry=%s (#1480)", async geometry => {
const missingGeometry = geometry !== "available";
const tmux = mockTmuxForReconciler();
const content = "❯ fixture-private-restore-prompt\n● OAuth token revoked · Please run /login\n\n❯";
vi.mocked(tmux.hasSession).mockResolvedValue(true);
vi.mocked(tmux.getPaneCommand).mockResolvedValue("sh");
let captured = false;
vi.mocked(tmux.capturePaneContent).mockImplementation(async () => { captured = true; return content; });
// Identity resolves before capture; fail only the later diagnostic geometry read.
const listPanes = tmux.listPanes;
tmux.listPanes = async (...args) => {
if (geometry === "timeout" && captured) return new Promise<never>(() => {});
if (geometry === "error" && captured) throw new Error("fixture-private-geometry-error");
return listPanes(...args);
};
const seeded = seedFailedAttempt({ restoreOutcome: "attention_required", withResumeToken: true });
sessionRegistry.updateStartupStatus(sessionRegistry.getSessionsForRig(seeded.rig.id)[0]!.id, "ready");
const before = db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").get();
const attempt = db.prepare("SELECT seq FROM events WHERE type = 'restore.started'").get() as { seq: number };
const restore = createOrchestrator({ tmux, listProcesses: async () => managedClaudeRows("tok-abc-123") });
const reconciler = new SeatAttentionReconciler({ db, sessionRegistry, eventBus,
agentActivityStore: new AgentActivityStore({ db, eventBus }),
reconcileRestoreOutcome: (rigId, nodeId) => restore.reconcileNodeRuntimeTruth(rigId, nodeId),
});
const result = await reconciler.clearAttention(seeded.sessionName, { reason: "operator explanation" });
expect(result).toMatchObject({ ok: false, code: "not_demonstrably_responsive",
detail: expect.stringContaining("pane_not_usable: Pane state is failed (login_required)") });
const events = db.prepare("SELECT payload FROM events WHERE type = 'seat.attention_clear_refused'").all() as { payload: string }[];
expect(events).toHaveLength(1);
expect(JSON.parse(events[0]!.payload)).toEqual({ type: "seat.attention_clear_refused", rigId: seeded.rig.id, nodeId: seeded.nodeId,
sessionId: sessionRegistry.getSessionsForRig(seeded.rig.id)[0]!.id,
sessionName: seeded.sessionName, attentionClass: "restore_outcome", code: "pane_not_usable", attemptId: attempt.seq,
probe: { status: "failed", code: "login_required",
loginRefusal: { message: "OAuth token revoked · Please run /login", row: "response" },
capture: { lines: 4, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"),
columns: missingGeometry ? null : 80, rows: missingGeometry ? null : 24 } },
});
expect(events[0]!.payload).not.toContain("fixture-private");
expect(db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").get()).toEqual(before);
expect(db.prepare("SELECT * FROM events WHERE type IN ('restore.outcome_reconciled', 'seat.attention_cleared')").all()).toHaveLength(0);
expect(tmux.sendText).not.toHaveBeenCalled();
expect(tmux.sendKeys).not.toHaveBeenCalled();
});
// #1077 — the seat launched with --resume tok-abc-123 and Claude continued as a new id. The
// strict reconciler keeps main's exact current-token path and accepts the launch token only as
// the alternative a qualified rotation names: the launched process's own first hook, a resume.
@@ -555,15 +555,29 @@ describe("SeatAttentionReconciler", () => {
});
it("OPR.0.5.9.14: refuses derived-class clear without strict evidence even with operator attestation", async () => {
seedDerivedAttentionSeat("r-derived-refuse", "worker@r-derived-refuse");
const { rigId, nodeId, sessionId } = seedDerivedAttentionSeat("r-derived-refuse", "worker@r-derived-refuse");
const before = db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").all();
const sendVerify = vi.fn(async () => ({ ok: true }));
const capture = vi.fn(async (sessionName: string) => ({ ok: true, sessionName, content: "fixture" }));
const derivedReconciler = new SeatAttentionReconciler({
sessionRegistry, eventBus, agentActivityStore: activityStore, db,
sessionRegistry, eventBus, agentActivityStore: activityStore, db, sendVerify, capture,
});
const result = await derivedReconciler.clearAttention("worker@r-derived-refuse", { reason: "operator verified" });
expect(result.ok).toBe(false);
expect(result.code).toBe("not_demonstrably_responsive");
expect(result.detail).toContain("strict restore reconciler is unavailable");
const refusals = db.prepare("SELECT payload FROM events WHERE type = 'seat.attention_clear_refused'").all() as { payload: string }[];
expect(refusals).toHaveLength(1);
expect(JSON.parse(refusals[0]!.payload)).toEqual({
type: "seat.attention_clear_refused", rigId, nodeId, sessionId,
sessionName: "worker@r-derived-refuse", attentionClass: "restore_outcome", code: "strict_reconciler_unavailable",
});
expect(db.prepare("SELECT payload FROM events WHERE type = 'restore.completed'").all()).toEqual(before);
expect(db.prepare("SELECT payload FROM events WHERE type = 'seat.attention_cleared'").all()).toHaveLength(0);
expect(sendVerify).not.toHaveBeenCalled();
expect(capture).not.toHaveBeenCalled();
const events = db.prepare("SELECT payload FROM events WHERE type = 'restore.outcome_reconciled'").all();
expect(events).toHaveLength(0);
});
@@ -16,6 +16,7 @@ import { AppliedLaunchObservationStore } from "../src/domain/applied-launch-obse
import { observeClaudePermission } from "../src/domain/permission-drift.js";
import type { SettingsStore } from "../src/domain/user-settings/settings-store.js";
import { assessNativeResumeProbe } from "../src/domain/native-resume-probe.js";
import { createHash } from "node:crypto";
import { CodexRuntimeAdapter } from "../src/adapters/codex-runtime-adapter.js";
import { ClaudeCodeAdapter } from "../src/adapters/claude-code-adapter.js";
import { seatSkillPaths } from "../src/domain/seat-skill-storage.js";
@@ -1475,6 +1476,43 @@ describe("StartupOrchestrator", () => {
}
});
it.each(["available", "error", "timeout"] as const)("persists the real Claude launch probe on startup_failed with geometry=%s (#1480)", async geometry => {
const missingGeometry = geometry !== "available";
const seed = seedSession();
const content = "❯ fixture-private-startup-prompt\n● Login expired · Please run /login\n\n❯";
const tmux = mockTmux({
getPaneCommand: vi.fn(async () => "claude"),
capturePaneContent: vi.fn(async () => content),
listPanes: vi.fn(async () => {
if (geometry === "timeout") return new Promise<never>(() => {});
if (geometry === "error") throw new Error("fixture-private-geometry-error");
return [{ id: "%1", index: 0, cwd: "/project", width: 100, height: 30, active: true }];
}),
});
const adapter = new ClaudeCodeAdapter({ tmux, sleep: async () => {}, fsOps: {
readFile: () => { throw new Error("fixture file missing"); },
writeFile: () => {}, exists: () => false, mkdirp: () => {}, copyFile: () => {}, listFiles: () => [],
} });
const result = await createOrchestrator(tmux).startNode(makeInput(seed, {
adapter, binding: { ...makeBinding(), cwd: "/project" },
isRestore: true, resumeToken: "saved-conversation", resumeType: "claude_id", allowFreshFallback: false,
}));
const probe = {
status: "failed", code: "login_required", loginRefusal: { message: "Login expired · Please run /login", row: "response" },
capture: { lines: 4, bytes: Buffer.byteLength(content), sha256: createHash("sha256").update(content).digest("hex"), columns: missingGeometry ? null : 100, rows: missingGeometry ? null : 30 },
};
expect(result).toMatchObject({ ok: false, startupStatus: "failed",
errors: ["Harness launch failed: Claude resume failed: timed out waiting for Claude to become active"], probe });
expect(sessionRegistry.getSessionsForRig(seed.rigId).find(s => s.id === seed.sessionId)?.startupStatus).toBe("failed");
const events = db.prepare("SELECT payload FROM events WHERE type = 'node.startup_failed'").all() as { payload: string }[];
expect(events).toHaveLength(1);
expect(JSON.parse(events[0]!.payload)).toMatchObject({ rigId: seed.rigId, nodeId: seed.nodeId, sessionId: seed.sessionId,
requestedResumeToken: "saved-conversation", stage: "harness_launch", probe });
expect(JSON.parse(events[0]!.payload).probe).toEqual(probe);
expect(events[0]!.payload).not.toContain("fixture-private");
expect(JSON.stringify(result)).not.toContain("fixture-private-startup-prompt");
});
// NS-T04: launchHarness persists resume token
it("launchHarness resume token persisted to session", async () => {
const seed = seedSession();