mirror of
https://github.com/mvschwarz/openrig.git
synced 2026-11-12 01:41:39 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2a9c204f2b | ||
|
|
f06edfa410 | ||
|
|
8c371e5041 | ||
|
|
dd7cf38f59 |
No files matched your search
@@ -347,13 +347,8 @@ with `rig up kernel --existing`. A seat you stopped or unclaimed also stays down
|
||||
While the kernel's other seats are running, `rig up kernel --existing` refuses
|
||||
(it restores only a kernel with no live session), so start one stopped seat with
|
||||
`rig launch kernel <logical id>` (`--plan` first). It restores that seat from the
|
||||
kernel snapshot that restore ranks first: the newest usable automatic one (taken
|
||||
before `rig down`, on the periodic schedule, or at a runtime switch), even when a
|
||||
manual one is newer, and otherwise the newest usable one. `--plan` names the
|
||||
snapshot it picked and any newer usable one, and `--snapshot-id <id>` picks one
|
||||
yourself. Under the default restore policy, which the kernel's seats keep, it
|
||||
resumes the seat's saved conversation; a seat it can't resume is reported as
|
||||
waiting for a decision, not as started.
|
||||
kernel's latest restore-usable snapshot and resumes its saved conversation; a seat
|
||||
it can't resume is reported as waiting for a decision, not as started.
|
||||
`rig seat launch <seat> --fresh --reason "…"` instead starts a new conversation
|
||||
with no continuity source: use it only when you want a fresh occupant, since the
|
||||
seat's earlier conversation isn't carried.
|
||||
|
||||
@@ -112,6 +112,33 @@ OpenRig doesn't observe a hosted member between its calls, so it never reads idl
|
||||
- `lastWake`: its latest ring and the result, among its 50 most recent rows.
|
||||
- `rig roster show` and `find` show the last call; the last ring is in `rig_team`.
|
||||
|
||||
### Check-in status
|
||||
|
||||
A connected assistant (a roster member marked `hosted: true`) checks in each time it reads its own pending inbox (through the queue list, as the MCP list tool does
|
||||
when it lists the member's own work). Other calls it makes don't count yet. Its status compares its last check-in with
|
||||
what its roster entry states: a cadence (`check_in_minutes`), or that it checks in when a person asks it
|
||||
(`check_in_on_request: true`):
|
||||
|
||||
| Status | Meaning |
|
||||
|---|---|
|
||||
| `checking-in` | its last check-in is no older than twice its stated cadence |
|
||||
| `check-in-overdue` | its last check-in is older than that: its schedule may be off or waiting for an approval |
|
||||
| `no-check-in-yet` | no check-in recorded for it on this daemon, whatever its entry states |
|
||||
| `on-request` | its entry says it checks in when a person asks it |
|
||||
| `disconnected` | reserved for a recorded revocation or removal of its connection. Nothing records one yet, so a revoked assistant reads overdue (or unknown, with no cadence) |
|
||||
| `unknown` | its entry states neither, its statement can't be used, or its check-ins couldn't be read; the reason says which, and its last check-in is shown. OpenRig assumes no cadence, so this never reads idle or overdue |
|
||||
|
||||
Each place it's shown gives the evidence with it, for example `overdue: last check-in 2026-11-12T13:05:00Z, expected
|
||||
every 60 min; its schedule may be off or waiting for an approval`.
|
||||
|
||||
A stated cadence is its person's statement: neither reading proves the assistant's own schedule ran, or that it picked
|
||||
anything up. It's a status only. Work addressed to a member that isn't checking in still queues and is rung as before, a check-in
|
||||
isn't pickup (pickup is its claim), and no check-in doesn't mean it's idle, failed or safe. `GET /api/queue/hosted-members`
|
||||
gives this daemon's host id and each member on this host with its status, last check-in and stated cadence (with the
|
||||
reason when it's unknown); `rig roster show` and `find` show it as `Check-in:`, for a team with or without terminal seats.
|
||||
When the member has unclaimed work, the standing notice described above states the same status in its `check-ins:`
|
||||
line; when that notice is filed doesn't change.
|
||||
|
||||
## Revoke
|
||||
|
||||
`rig mcp token revoke grok` refuses its next request. The rows it worked, and their answers, stay.
|
||||
|
||||
@@ -92,3 +92,8 @@ A member marked `"hosted": true` is a hosted member: a tool outside the terminal
|
||||
`show` and `find` say it's hosted rather than unmatched. See [Hosted members](hosted-members.md).
|
||||
Its entry can declare `wake.webhook`, the secret names for a trigger OpenRig rings when work lands for it
|
||||
(see [Ring it when work lands](hosted-members.md#ring-it-when-work-lands)).
|
||||
It can also declare `check_in_minutes`, how often it's expected to check its queue (a positive whole number of minutes;
|
||||
an hourly schedule is `60`), or `check_in_on_request: true` when it checks in only when a person asks it. That's what
|
||||
its person set up, not something OpenRig observes. With neither, its check-in status reads unknown: OpenRig doesn't
|
||||
assume a cadence. A value that can't be used doesn't stop the roster loading: that member's check-in
|
||||
status reads unknown, with the reason. See [Check-in status](hosted-members.md#check-in-status).
|
||||
@@ -226,6 +226,18 @@ retry is scheduled. The reasons are:
|
||||
one with a reused `deliveryId`; the result warns older callers that this field no
|
||||
longer deduplicates input. The CLI does not generate a deduplication key.
|
||||
|
||||
The last recorded managed-compaction input refusal appears as `inputReadFailure` in the attempt
|
||||
record and `rig compact --state`: the preparation or compact stage, `composer`
|
||||
(the explicit composer hold) or `draft-aware` check, reader status, confidence,
|
||||
a fixed read-failure reason and cursor flags. An owed restore step reports it in
|
||||
`owedRestore.lastDelivery`. These fields contain no input text, suggestions,
|
||||
terminal frames or exception messages. Confidence and timeout fields are `null`
|
||||
when that reader does not supply them; a known composer read timeout reports
|
||||
`timedOut: true`. A generic `capture_unavailable` does not identify the failed
|
||||
capture operation. Older records lack this diagnostic. It neither changes the
|
||||
refusal nor retries an attempt; missing metadata cannot establish which check
|
||||
caused a historical refusal.
|
||||
|
||||
Clearing a draft, changing modes or restarting does not flush saved messages or
|
||||
callbacks. To try again, send a current request: it rechecks the recipient and input.
|
||||
A caller abort before paste cancels input; after a write may have started, the
|
||||
|
||||
@@ -5,6 +5,10 @@ import { fileURLToPath } from "node:url";
|
||||
import { execFile } from "node:child_process";
|
||||
import { ConfigStore } from "../config-store.js";
|
||||
import { mcpPresenceFilePath, readPresence } from "../mcp-presence.js";
|
||||
import { CHECK_IN_DISPLAY, describeCheckIn, type CheckInStatus } from "@openrig/daemon/connection-contract";
|
||||
import { getDaemonStatus, getDaemonUrl } from "../daemon-lifecycle.js";
|
||||
import { DaemonClient } from "../client.js";
|
||||
import { realDeps } from "./daemon.js";
|
||||
import { parseRoster, readRosters, type Roster, type RosterMember, type RosterWake } from "@openrig/daemon/rosters";
|
||||
|
||||
// One roster parser, shared with the daemon (a hosted member's wake route reads the same files).
|
||||
@@ -17,6 +21,23 @@ export interface RosterInventory {
|
||||
sources: Row[];
|
||||
/** This host's id, from the local daemon's own nodes; null when it has none to read it from. */
|
||||
localHostId?: string | null;
|
||||
/** The local daemon's hosted members with their check-in status, and that daemon's own host id (the host whose
|
||||
* check-ins these are); null when it couldn't be read. */
|
||||
checkIns?: { hostId: string | null; members: Row[] } | null;
|
||||
}
|
||||
|
||||
/** Each hosted member's check-in status, from the local daemon. Null when the daemon isn't running or doesn't answer
|
||||
* (an older daemon has no such read): every hosted member's status then reads unknown. */
|
||||
export async function readHostedCheckIns(): Promise<RosterInventory["checkIns"]> {
|
||||
try {
|
||||
const status = await getDaemonStatus(realDeps());
|
||||
if (status.state !== "running") return null;
|
||||
const res = await new DaemonClient(getDaemonUrl(status)).get<unknown>("/api/queue/hosted-members");
|
||||
return res.status < 400 && object(res.data) && Array.isArray(res.data.members)
|
||||
? { hostId: text(res.data.hostId), members: res.data.members.filter(object) } : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
const object = (v: unknown): v is Row => !!v && typeof v === "object" && !Array.isArray(v);
|
||||
const text = (v: unknown): string | null => typeof v === "string" && v.trim() ? v : null;
|
||||
@@ -49,7 +70,30 @@ export async function readRosterInventory(): Promise<RosterInventory> {
|
||||
sources.push({ scope, status: "unavailable", error: result.stderr.trim() || (error as Error).message });
|
||||
}
|
||||
}
|
||||
return { readAt: new Date().toISOString(), nodes, sources, localHostId };
|
||||
return { readAt: new Date().toISOString(), nodes, sources, localHostId, checkIns: await readHostedCheckIns() };
|
||||
}
|
||||
|
||||
function hostedCheckIn(member: RosterMember, inventory: RosterInventory): Row {
|
||||
const unknown = (reason: string) => ({ status: "unknown", lastCheckInAt: null, expectedMinutes: member.check_in_minutes ?? null,
|
||||
onRequest: member.check_in_on_request === true, reason });
|
||||
// The daemon names its own host with its check-ins, so a team with no terminal seat still matches its members.
|
||||
if (!inventory.checkIns) return unknown("this host's daemon gave no check-in status");
|
||||
if (!inventory.checkIns.hostId || member.host !== inventory.checkIns.hostId) return unknown("it's listed on another host, whose check-ins this host doesn't hold");
|
||||
const row = inventory.checkIns.members.find((m) => m.seat === member.seat);
|
||||
return row && typeof row.status === "string" && row.status in CHECK_IN_DISPLAY ? row
|
||||
: unknown("this host's daemon gave no status for it");
|
||||
}
|
||||
|
||||
/** A connected assistant's check-in in one human line with its evidence, in the daemon's own words
|
||||
* (`describeCheckIn`): "overdue: last check-in 13:05, expected every 60 min; its schedule may be off or waiting for an
|
||||
* approval". A status this CLI doesn't know reads unknown. */
|
||||
export function formatCheckIn(checkIn: Row): string {
|
||||
const known = typeof checkIn.status === "string" && checkIn.status in CHECK_IN_DISPLAY;
|
||||
return describeCheckIn({ status: known ? checkIn.status as CheckInStatus : "unknown",
|
||||
lastCheckInAt: typeof checkIn.lastCheckInAt === "string" ? checkIn.lastCheckInAt : null,
|
||||
expectedMinutes: typeof checkIn.expectedMinutes === "number" ? checkIn.expectedMinutes : null,
|
||||
onRequest: checkIn.onRequest === true,
|
||||
...(known && typeof checkIn.reason === "string" ? { reason: checkIn.reason } : {}) });
|
||||
}
|
||||
|
||||
export function enrichRosterMember(member: RosterMember, inventory: RosterInventory) {
|
||||
@@ -93,6 +137,9 @@ export function enrichRosterMember(member: RosterMember, inventory: RosterInvent
|
||||
// is in rig_team.
|
||||
...(member.hosted ? { lastCallAt: inventory.localHostId && member.host === inventory.localHostId
|
||||
? readPresence(mcpPresenceFilePath())[member.seat]?.lastCallAt ?? null : null } : {}),
|
||||
// Whether it's checking its queue, for a member on this host; another host's member, or a status this daemon
|
||||
// couldn't give, reads unknown.
|
||||
...(member.hosted ? { checkIn: hostedCheckIn(member, inventory) } : {}),
|
||||
} };
|
||||
}
|
||||
|
||||
@@ -134,7 +181,7 @@ export function rosterCommand(deps: Deps = {}): Command {
|
||||
else {
|
||||
const members = (verb === "show" ? (result.roster as Row).members : result.matches) as Array<ReturnType<typeof enrichRosterMember> & { roster?: { id: string } }>;
|
||||
if (verb === "show") { const r = result.roster as Roster; console.log(`${r.name} — ${r.purpose}\nCurator: ${r.curator.seat} (${r.curator.host})`); }
|
||||
for (const m of members) console.log(`${m.seat} (${m.host})${m.roster ? ` [${m.roster.id}]` : ""}\n ${m.engagement.join(", ")}: ${m.capabilities.join(", ")}\n When: ${m.use_when}\n Why: ${m.why}${m.caveat ? `\n Caveat: ${m.caveat}` : ""}\n ${m.live.status}: runtime ${m.live.runtime ?? "unknown"}; configured model ${m.live.configuredModel ?? "unknown"}; last observed model ${formatModelObservation(m.live.modelObservation)}\n Activity: ${m.live.agentActivity.state}; sampled ${m.live.agentActivity.sampledAt ?? "unknown"}${m.hosted ? `\n Last call: ${m.live.lastCallAt ?? "unknown"}` : ""}\n ${m.live.reason}`);
|
||||
for (const m of members) console.log(`${m.seat} (${m.host})${m.roster ? ` [${m.roster.id}]` : ""}\n ${m.engagement.join(", ")}: ${m.capabilities.join(", ")}\n When: ${m.use_when}\n Why: ${m.why}${m.caveat ? `\n Caveat: ${m.caveat}` : ""}\n ${m.live.status}: runtime ${m.live.runtime ?? "unknown"}; configured model ${m.live.configuredModel ?? "unknown"}; last observed model ${formatModelObservation(m.live.modelObservation)}\n Activity: ${m.live.agentActivity.state}; sampled ${m.live.agentActivity.sampledAt ?? "unknown"}${m.hosted ? `\n Last call: ${m.live.lastCallAt ?? "unknown"}\n Check-in: ${formatCheckIn(m.live.checkIn ?? {})}` : ""}\n ${m.live.reason}`);
|
||||
if (!members.length) console.log("No matching members.");
|
||||
const obs = result.observations as { sources: Row[] };
|
||||
for (const source of obs.sources) if (source.status !== "ok") console.error(`Observation gap (${source.scope}): ${source.warning ?? source.error ?? source.status}`);
|
||||
|
||||
@@ -5,7 +5,7 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { execFile, execFileSync } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { enrichRosterMember, parseRoster, readRosters, rosterCommand, type RosterInventory } from "../src/commands/roster.js";
|
||||
import { enrichRosterMember, formatCheckIn, parseRoster, readRosters, rosterCommand, type RosterInventory } from "../src/commands/roster.js";
|
||||
|
||||
const member = { seat: "editor@studio", host: "host-a", capabilities: ["colour grading"], engagement: ["consult"], use_when: "Choose a look", why: "Knows the footage", caveat: "Advice only" };
|
||||
const authored = { version: 1, id: "production", name: "Production", purpose: "Make a film", curator: { seat: "lead@studio", host: "host-a" }, updated_at: "2026-09-01", members: [member] };
|
||||
@@ -133,3 +133,55 @@ it("does not choose the first duplicate route when activity observations conflic
|
||||
expect(enrichRosterMember(member, { readAt: "2026-10-01T00:00:02Z", nodes: [idle, unclocked], sources: [] }).live.agentActivity)
|
||||
.toEqual({ state: "idle", reason: null, sampledAt: null });
|
||||
});
|
||||
|
||||
it("keeps any positive whole check-in cadence; one that can't be used still loads, with its reason and no cadence", () => {
|
||||
const hourly = { ...member, seat: "bot@team", hosted: true, check_in_minutes: 60 };
|
||||
expect(parseRoster({ ...authored, members: [hourly, { ...hourly, seat: "fast@team", check_in_minutes: 1 }] }).members)
|
||||
.toEqual([hourly, { ...hourly, seat: "fast@team", check_in_minutes: 1 }]);
|
||||
for (const bad of [{ ...hourly, check_in_minutes: 0 }, { ...hourly, check_in_minutes: 1.5 }, { ...hourly, check_in_minutes: "60" },
|
||||
{ ...member, check_in_minutes: 60 }]) {
|
||||
const [loaded] = parseRoster({ ...authored, members: [bad] }).members;
|
||||
expect(loaded).not.toHaveProperty("check_in_minutes");
|
||||
expect(loaded!.check_in_problem).toMatch(/check_in_minutes/);
|
||||
}
|
||||
});
|
||||
it("shows a hosted member's check-in status from this host's daemon; another host's member, or no answer, reads unknown", () => {
|
||||
const bot = { ...member, seat: "bot@team", host: "host-a", hosted: true, check_in_minutes: 60 };
|
||||
const checkIns = [{ seat: "bot@team", rung: false, status: "check-in-overdue", lastCheckInAt: "2026-11-12T09:00:00.000Z", expectedMinutes: 60, reporter: "openrig-observed" }];
|
||||
const here = { ...inventory([]), localHostId: "host-a", checkIns: { hostId: "host-a", members: checkIns } };
|
||||
expect(enrichRosterMember(bot, here).live.checkIn).toMatchObject({ status: "check-in-overdue", lastCheckInAt: "2026-11-12T09:00:00.000Z" });
|
||||
expect(formatCheckIn(enrichRosterMember(bot, here).live.checkIn!))
|
||||
.toBe("overdue: last check-in 2026-11-12T09:00:00.000Z, expected every 60 min; its schedule may be off or waiting for an approval");
|
||||
for (const [missing, reason] of [[{ ...here, checkIns: null }, "this host's daemon gave no check-in status"],
|
||||
[{ ...here, checkIns: { hostId: "host-b", members: checkIns } }, "it's listed on another host, whose check-ins this host doesn't hold"],
|
||||
[{ ...here, checkIns: { hostId: null, members: checkIns } }, "it's listed on another host, whose check-ins this host doesn't hold"],
|
||||
[{ ...here, checkIns: { hostId: "host-a", members: [{ ...checkIns[0], status: "invented" }] } }, "this host's daemon gave no status for it"]] as const) {
|
||||
expect(enrichRosterMember(bot, missing).live.checkIn).toEqual({ status: "unknown", lastCheckInAt: null, expectedMinutes: 60, onRequest: false, reason });
|
||||
}
|
||||
// The roster line is the daemon's own sentence for each status.
|
||||
expect(formatCheckIn({ status: "unknown", lastCheckInAt: null, expectedMinutes: null, reason: "its check-ins couldn't be read" }))
|
||||
.toBe("unknown (its check-ins couldn't be read)");
|
||||
expect(formatCheckIn({ status: "no-check-in-yet", lastCheckInAt: null, expectedMinutes: null })).toBe("no check-in yet; no check-in cadence declared");
|
||||
expect(formatCheckIn({ status: "no-check-in-yet", lastCheckInAt: null, expectedMinutes: 60 })).toBe("no check-in yet; expected every 60 min");
|
||||
expect(formatCheckIn({ status: "on-request", lastCheckInAt: "2026-11-12T09:00:00.000Z", expectedMinutes: null, onRequest: true }))
|
||||
.toBe("on request: it checks its queue when a person asks it; last check-in 2026-11-12T09:00:00.000Z");
|
||||
expect(formatCheckIn({ status: "invented", lastCheckInAt: null, expectedMinutes: null })).toBe("unknown (its check-in status couldn't be read)");
|
||||
expect(enrichRosterMember(member, here).live).not.toHaveProperty("checkIn");
|
||||
});
|
||||
it("a team with no terminal seat still shows its members' check-ins: the daemon names its own host with them", () => {
|
||||
const bot = { ...member, seat: "bot@team", host: "host-a", hosted: true, check_in_minutes: 60 };
|
||||
const fresh = { seat: "bot@team", rung: false, status: "checking-in", lastCheckInAt: "2026-11-12T09:55:00.000Z", expectedMinutes: 60, reporter: "openrig-observed" };
|
||||
const noSeats: RosterInventory = { ...inventory([]), localHostId: null, checkIns: { hostId: "host-a", members: [fresh] } };
|
||||
expect(enrichRosterMember(bot, noSeats).live.checkIn).toMatchObject({ status: "checking-in", lastCheckInAt: fresh.lastCheckInAt });
|
||||
});
|
||||
it("keeps check_in_on_request: true; any other value, both statements at once, or a member not marked hosted loads with its reason", () => {
|
||||
const asked = { ...member, seat: "bot@team", hosted: true, check_in_on_request: true };
|
||||
expect(parseRoster({ ...authored, members: [asked] }).members).toEqual([asked]);
|
||||
for (const [bad, reason] of [[{ ...asked, check_in_on_request: false }, /isn't true/], [{ ...asked, check_in_minutes: 60 }, /both/],
|
||||
[{ ...member, check_in_on_request: true }, /connected assistant/]] as const) {
|
||||
const [loaded] = parseRoster({ ...authored, members: [bad] }).members;
|
||||
expect(loaded).not.toHaveProperty("check_in_on_request");
|
||||
expect(loaded).not.toHaveProperty("check_in_minutes");
|
||||
expect(loaded!.check_in_problem).toMatch(reason);
|
||||
}
|
||||
});
|
||||
@@ -31,6 +31,10 @@
|
||||
"types": "./dist/rosters-surface.d.ts",
|
||||
"import": "./dist/rosters-surface.js"
|
||||
},
|
||||
"./connection-contract": {
|
||||
"types": "./dist/connection-contract-surface.d.ts",
|
||||
"import": "./dist/connection-contract-surface.js"
|
||||
},
|
||||
"./gateway-protocol": {
|
||||
"types": "./dist/gateway-protocol-surface.d.ts",
|
||||
"import": "./dist/gateway-protocol-surface.js"
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { InputReadDiagnostic } from "../domain/input-read-diagnostic.js";
|
||||
import { DeliveryGuardError, type SeatDeliveryGuard } from "../domain/seat-delivery-guard.js";
|
||||
import { writeFile as fsWriteFile, unlink as fsUnlink } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
@@ -77,7 +78,7 @@ export function defaultTmuxFileOps(): TmuxFileOps {
|
||||
|
||||
export type TmuxResult =
|
||||
| { ok: true; warnings?: string[]; stopPhase?: StopPhase }
|
||||
| { ok: false; code: string; message: string; stopPhase?: StopPhase };
|
||||
| { ok: false; code: string; message: string; stopPhase?: StopPhase; inputReadFailure?: InputReadDiagnostic };
|
||||
|
||||
/** Furthest external stop boundary reached; an attempted command can have an uncertain effect. */
|
||||
export type StopPhase = "none" | "detach_attempted" | "detached" | "kill_attempted";
|
||||
@@ -246,7 +247,7 @@ function isTmuxTransportAbsentError(err: unknown): boolean {
|
||||
}
|
||||
|
||||
function classifyWriteError(err: unknown): TmuxResult {
|
||||
if (err instanceof DeliveryGuardError) return { ok: false, code: err.code, message: err.message };
|
||||
if (err instanceof DeliveryGuardError) return { ok: false, code: err.code, message: err.message, ...(err.inputReadFailure ? { inputReadFailure: err.inputReadFailure } : {}) };
|
||||
if (!(err instanceof Error)) {
|
||||
return { ok: false, code: "unknown", message: String(err) };
|
||||
}
|
||||
@@ -600,7 +601,8 @@ export class TmuxAdapter {
|
||||
return guard.input(identity, () => write(pane, () => guard.checkInput(identity), bound.nodeId));
|
||||
});
|
||||
} catch (error) {
|
||||
return { ok: false, code: (error as { code?: string }).code ?? "guard_target_unknown", message: String((error as Error).message) };
|
||||
return { ok: false, code: (error as { code?: string }).code ?? "guard_target_unknown", message: String((error as Error).message),
|
||||
...(error instanceof DeliveryGuardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}) };
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
// The `@openrig/daemon/connection-contract` surface: the shared outcome and status words, so the CLI and the daemon
|
||||
// read them from one place.
|
||||
export * from "./domain/connection-contract.js";
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { InputReadDiagnostic } from "./input-read-diagnostic.js";
|
||||
import type { TerminalActor, TerminalPolicyStore } from "./terminal-policy.js";
|
||||
import { TerminalPolicyRefusal } from "./terminal-policy.js";
|
||||
import { DeliveryGuardError } from "./seat-delivery-guard.js";
|
||||
@@ -425,6 +426,7 @@ type PendingPostCompactStage = "turn_boundary" | "restore_prompt" | "compliance_
|
||||
|
||||
/** Safe delivery diagnostics: never retain pane contents or an exception message. */
|
||||
export interface RestoreDeliveryAttempt {
|
||||
inputReadFailure?: InputReadDiagnostic;
|
||||
stage: PendingPostCompactStage;
|
||||
at: string;
|
||||
ok: boolean;
|
||||
@@ -461,6 +463,7 @@ export interface ObservedCompaction {
|
||||
detail: string;
|
||||
}
|
||||
export interface PreparationState {
|
||||
inputReadFailure?: InputReadDiagnostic & { stage: "preparation" | "compact" };
|
||||
/** The reading that held this preparation send; never includes draft text. */
|
||||
inputBox?: SendResult["heldInputBoxReading"];
|
||||
policyActor?: TerminalActor;
|
||||
@@ -565,6 +568,7 @@ interface PreparationFailureRun {
|
||||
}
|
||||
|
||||
interface CompactionRecord {
|
||||
inputReadFailure?: PreparationState["inputReadFailure"];
|
||||
preparationFailures?: PreparationFailureRun;
|
||||
inputBox?: PreparationState["inputBox"];
|
||||
/** The daemon process that last wrote it. */
|
||||
@@ -1000,6 +1004,7 @@ export class ClaudeCompactionEnforcer {
|
||||
} catch (error) {
|
||||
this.recordRestoreFailure(input.sessionName, stage, record, {
|
||||
ok: false, reason: error instanceof DeliveryGuardError ? error.code : "send_failed",
|
||||
...(error instanceof DeliveryGuardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}),
|
||||
}, error instanceof Error ? error.name : typeof error);
|
||||
if (error instanceof DeliveryGuardError) return { triggered: false, reason: error.code === "typing_guard_enabled" ? "typing_guard_enabled" : "guard_target_unknown" };
|
||||
throw error;
|
||||
@@ -1027,7 +1032,7 @@ export class ClaudeCompactionEnforcer {
|
||||
private recordRestoreFailure(
|
||||
session: string, stage: PendingPostCompactStage | undefined,
|
||||
record: { file: string; record: CompactionRecord } | undefined,
|
||||
result: Pick<SendResult, "ok" | "reason" | "outcome" | "activity" | "heldInputBox">, errorClass?: string,
|
||||
result: Pick<SendResult, "ok" | "reason" | "outcome" | "activity" | "heldInputBox" | "inputReadFailure">, errorClass?: string,
|
||||
): void {
|
||||
if (!stage || !record || this.records.get(session) !== record || this.pendingPostCompactRestore.get(session) !== stage) return;
|
||||
const prior = record.record.lastDelivery;
|
||||
@@ -1042,6 +1047,7 @@ export class ClaudeCompactionEnforcer {
|
||||
const lastDelivery: RestoreDeliveryAttempt = {
|
||||
stage, at: new Date(this.now()).toISOString(), ok: result.ok, reason: result.reason ?? "send_failed",
|
||||
...(result.outcome ? { outcome: result.outcome } : {}),
|
||||
...(result.inputReadFailure ? { inputReadFailure: result.inputReadFailure } : {}),
|
||||
...(result.activity ? { activity: { state: result.activity.state, reason: result.activity.reason, evidenceSource: result.activity.evidenceSource } } : {}),
|
||||
...(errorClass ? { errorClass } : {}), failures: waiting ? 0 : (prior?.failures ?? 0) + 1,
|
||||
...(held ? { inputBox: held } : {}), ...(unreadable ? { unreadable } : {}),
|
||||
@@ -1598,7 +1604,7 @@ export class ClaudeCompactionEnforcer {
|
||||
if (error instanceof DeliveryGuardError) {
|
||||
const written = (error as { inputWritten?: boolean }).inputWritten === true;
|
||||
const attempted = (error as { inputAttempted?: boolean }).inputAttempted === true;
|
||||
prep = { ok: false, sent: written, reason: error.code,
|
||||
prep = { ok: false, sent: written, reason: error.code, inputReadFailure: error.inputReadFailure,
|
||||
inputPhase: written ? "written" as const : attempted ? "attempted" as const : "not_reached" as const };
|
||||
} else prep = null;
|
||||
}
|
||||
@@ -1611,7 +1617,8 @@ export class ClaudeCompactionEnforcer {
|
||||
// Neither a failed response nor its loss proves no paste/Enter. Never replay uncertainty.
|
||||
attempt.delivery = prep?.ok && prep.outcome !== "retained" ? "delivered" : notSent ? "not_sent" : "uncertain";
|
||||
attempt.inputBox = prep?.heldInputBoxReading;
|
||||
this.persist(input.sessionName, attempt.attemptId, { delivery: attempt.delivery, sends: attempt.sends, inputBox: attempt.inputBox });
|
||||
if (prep?.inputReadFailure) attempt.inputReadFailure = { stage: "preparation", ...prep.inputReadFailure };
|
||||
this.persist(input.sessionName, attempt.attemptId, { delivery: attempt.delivery, sends: attempt.sends, inputBox: attempt.inputBox, inputReadFailure: attempt.inputReadFailure });
|
||||
// The replacement request went out (or may have): the ended attempt's gate is used up.
|
||||
if (!notSent && attempt.afterEnded !== undefined && this.pendingPreCompactPrep.get(input.sessionName) === attempt) this.lastEnded.delete(input.sessionName);
|
||||
if (this.pendingPreCompactPrep.get(input.sessionName) !== attempt || attempt.controller.signal.aborted) return;
|
||||
@@ -1661,7 +1668,7 @@ export class ClaudeCompactionEnforcer {
|
||||
return this.sessionTransport.send(input.sessionName, composed.command,
|
||||
{ openrigNotice: true, allowDraftWhileIdle: true, beforeWrite, ...(attempt.deadlineAt !== null ? { waitForIdleMs: attempt.prepared ? idleWait : Math.max(1, Math.min(idleWait, this.compactDeadline(attempt)! - this.now())), readinessFromPaneOnly: true } : {}) });
|
||||
};
|
||||
let result: Pick<SendResult, "ok" | "reason" | "outcome" | "sent" | "inputPhase" | "activity" | "heldInputBox">;
|
||||
let result: Pick<SendResult, "ok" | "reason" | "outcome" | "sent" | "inputPhase" | "activity" | "heldInputBox" | "inputReadFailure">;
|
||||
let guardRefused = false;
|
||||
try {
|
||||
const guard = this.sessionTransport.deliveryGuard;
|
||||
@@ -1672,11 +1679,16 @@ export class ClaudeCompactionEnforcer {
|
||||
const attempted = (error as { inputAttempted?: boolean } | null)?.inputAttempted === true;
|
||||
guardRefused = error instanceof DeliveryGuardError && !written && !attempted;
|
||||
result = { ok: false, reason: error instanceof DeliveryGuardError ? error.code : "delivery_uncertain",
|
||||
...(error instanceof DeliveryGuardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}),
|
||||
...(written ? { sent: true, inputPhase: "written" as const } : attempted ? { inputPhase: "attempted" as const } : {}) };
|
||||
} finally {
|
||||
delete attempt.compactInFlight;
|
||||
}
|
||||
if (!result.ok || result.outcome === "retained") {
|
||||
if (result.inputReadFailure) {
|
||||
attempt.inputReadFailure = { stage: "compact", ...result.inputReadFailure };
|
||||
this.persist(input.sessionName, attempt.attemptId, { inputReadFailure: attempt.inputReadFailure });
|
||||
}
|
||||
const phase = result.inputPhase;
|
||||
// A write that was issued, or text already in the input, is never "nothing sent".
|
||||
const writeAttempted = phase === "attempted" || phase === "written" || result.sent === true;
|
||||
@@ -1927,7 +1939,7 @@ export class ClaudeCompactionEnforcer {
|
||||
compact: "not_sent", sends: record.sends,
|
||||
deadlineAt: record.deadlineAt ?? (record.delivery === "not_sent" ? null : record.updatedAt + AUTO_PREP_WAIT_MS_DEFAULT),
|
||||
policyWasEnabled: true, preparationRoot: record.preparationRoot, controller: new AbortController(),
|
||||
compactInstruction: record.compactInstruction, inputBox: record.inputBox,
|
||||
compactInstruction: record.compactInstruction, inputBox: record.inputBox, inputReadFailure: record.inputReadFailure,
|
||||
// A held attempt comes back held, with its recorded hold (never stamped again) and its map's age and usage.
|
||||
...(held ? { prepared: true as const, preparedAt: record.preparedAt ?? record.updatedAt, preparedReason: record.preparedReason ?? "map_complete_seat_never_idle" } : {}),
|
||||
...(held ? { mapReadyAt: record.mapReadyAt ?? record.preparedAt ?? record.updatedAt } : record.mapReadyAt !== undefined ? { mapReadyAt: record.mapReadyAt } : {}),
|
||||
|
||||
@@ -42,6 +42,102 @@ export interface ReportedFact<F extends string> {
|
||||
reporter: Reporter;
|
||||
}
|
||||
|
||||
/** Whether a member outside any terminal is checking its queue. It's a status, never a refusal: work addressed to the
|
||||
* member queues and is rung exactly as before, a check-in isn't pickup (pickup is a claim), and no check-in isn't
|
||||
* idle, failure or safe. */
|
||||
export type CheckInStatus =
|
||||
/** Its last check-in is no older than twice its stated cadence (`CHECK_IN_OVERDUE_CADENCES`). */
|
||||
| "checking-in"
|
||||
/** Its last check-in is older than twice its stated cadence: its schedule may be off or waiting for an approval. */
|
||||
| "check-in-overdue"
|
||||
/** No check-in recorded for it. */
|
||||
| "no-check-in-yet"
|
||||
/** Its setup says it checks its queue when a person asks it (`check_in_on_request`), not on a schedule. */
|
||||
| "on-request"
|
||||
/** Its connection was revoked or removed, as recorded. Never inferred from silence. Nothing records a revocation yet,
|
||||
* so nothing emits it: a revoked member stops checking in and reads overdue. */
|
||||
| "disconnected"
|
||||
/** Its setup states no cadence, its stated one can't be used, or its check-ins couldn't be read (the reason says
|
||||
* which). Never idle and never overdue: with no stated cadence, OpenRig assumes none. */
|
||||
| "unknown";
|
||||
|
||||
/** How each check-in status reads to a person. */
|
||||
export const CHECK_IN_DISPLAY: Readonly<Record<CheckInStatus, string>> = {
|
||||
"checking-in": "checking in",
|
||||
"check-in-overdue": "overdue",
|
||||
"no-check-in-yet": "no check-in yet",
|
||||
"on-request": "on request",
|
||||
"disconnected": "disconnected",
|
||||
"unknown": "unknown",
|
||||
};
|
||||
|
||||
/** The one rule for a stated cadence: a last check-in no older than this many cadences reads checking in, and an older
|
||||
* one overdue. One missed run, plus its schedule's own jitter, isn't overdue. A cadence is its person's statement, so
|
||||
* neither reading proves a vendor schedule or a pickup. */
|
||||
export const CHECK_IN_OVERDUE_CADENCES = 2;
|
||||
|
||||
export interface MemberCheckIn {
|
||||
status: CheckInStatus;
|
||||
/** Its own latest read of its pending inbox, as OpenRig observed it. */
|
||||
lastCheckInAt: string | null;
|
||||
/** The cadence its person stated at setup; null when none was. */
|
||||
expectedMinutes: number | null;
|
||||
/** Its setup says it checks in when a person asks it. */
|
||||
onRequest: boolean;
|
||||
reporter: Reporter;
|
||||
/** Why the status is unknown, when it is. */
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
/** A connected assistant's check-in status from what its setup states (a cadence, or on request) and its last observed
|
||||
* check-in (`seen`, `never`, or `unknown` when the record can't be read). First match wins: disconnected, unknown (an
|
||||
* unusable statement or an unreadable record), no check-in yet, on request, then checking in or overdue by its cadence,
|
||||
* else unknown because it states none. Nothing passes `revokedAt` yet, so nothing reads disconnected. */
|
||||
export function memberCheckInStatus(input: {
|
||||
checkInMinutes: number | null | undefined;
|
||||
/** Its setup says it checks in on request rather than on a schedule. */
|
||||
onRequest?: boolean;
|
||||
/** Why the stated cadence can't be used, from the roster. */
|
||||
cadenceProblem?: string | null;
|
||||
checkIn: { state: "seen"; at: string } | { state: "never" } | { state: "unknown" };
|
||||
revokedAt?: string | null;
|
||||
now: Date;
|
||||
}): MemberCheckIn {
|
||||
const expectedMinutes = input.checkInMinutes ?? null;
|
||||
const lastCheckInAt = input.checkIn.state === "seen" ? input.checkIn.at : null;
|
||||
const onRequest = input.onRequest === true;
|
||||
const unknown = (reason: string): MemberCheckIn => ({ status: "unknown", lastCheckInAt, expectedMinutes, onRequest, reporter: "openrig-observed", reason });
|
||||
const known = (status: CheckInStatus): MemberCheckIn => ({ status, lastCheckInAt, expectedMinutes, onRequest, reporter: "openrig-observed" });
|
||||
if (input.revokedAt) return known("disconnected");
|
||||
if (input.cadenceProblem) return unknown(input.cadenceProblem);
|
||||
if (input.checkIn.state === "unknown") return unknown("its check-ins couldn't be read");
|
||||
if (input.checkIn.state === "never") return known("no-check-in-yet");
|
||||
if (onRequest) return known("on-request");
|
||||
if (expectedMinutes === null) return unknown(NO_CADENCE_STATED);
|
||||
const age = input.now.getTime() - Date.parse(input.checkIn.at);
|
||||
if (!Number.isFinite(age)) return unknown("its last check-in's time couldn't be read");
|
||||
return known(age <= expectedMinutes * CHECK_IN_OVERDUE_CADENCES * 60_000 ? "checking-in" : "check-in-overdue");
|
||||
}
|
||||
|
||||
/** Why a connected assistant whose setup states neither a cadence nor on request reads unknown. */
|
||||
export const NO_CADENCE_STATED = "no check-in cadence declared";
|
||||
|
||||
/** A connected assistant's check-in in one line for a person, with its evidence: "overdue: last check-in 13:05,
|
||||
* expected every 60 min; its schedule may be off or waiting for an approval". */
|
||||
export function describeCheckIn(c: Pick<MemberCheckIn, "status" | "lastCheckInAt" | "expectedMinutes" | "onRequest" | "reason">): string {
|
||||
const last = c.lastCheckInAt ? `last check-in ${c.lastCheckInAt}` : null;
|
||||
const expected = c.expectedMinutes !== null ? `expected every ${c.expectedMinutes} min`
|
||||
: c.onRequest ? "checks in when a person asks it" : NO_CADENCE_STATED;
|
||||
switch (c.status) {
|
||||
case "checking-in": return `checking in: ${last}, ${expected}`;
|
||||
case "check-in-overdue": return `overdue: ${last}, ${expected}; its schedule may be off or waiting for an approval`;
|
||||
case "no-check-in-yet": return `no check-in yet; ${expected}`;
|
||||
case "on-request": return `on request: it checks its queue when a person asks it${last ? `; ${last}` : ""}`;
|
||||
case "disconnected": return "disconnected: its connection was revoked or removed";
|
||||
case "unknown": return `unknown (${c.reason ?? "its check-in status couldn't be read"})${last ? `; ${last}` : ""}`;
|
||||
}
|
||||
}
|
||||
|
||||
/** The route a wake took. `none` means no route can carry it (a pull-only member, an address nothing holds). */
|
||||
export type WakeRoute = "terminal" | "gateway" | "hosted-trigger" | "none";
|
||||
|
||||
|
||||
@@ -95,7 +95,8 @@ export class DraftAwareDelivery {
|
||||
: guardError ? error.message : "The delivery attempt ended without a reliable write result. Repeating the request may duplicate input.";
|
||||
result = { ok: false, sessionName: session, outcome: "failed", inputPhase,
|
||||
...(inputWritten || inputPhase === "not_reached" ? { sent: inputWritten } : {}),
|
||||
reason: guardError ? error.code : "delivery_indeterminate", error: detail };
|
||||
reason: guardError ? error.code : "delivery_indeterminate", error: detail,
|
||||
...(guardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}) };
|
||||
}
|
||||
if (inputWritten) result = { ...result, sent: true };
|
||||
const preInputRefusal = ["draft_input_busy", "draft_input_unknown", "draft_input_changed", "draft_wake_superseded",
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
/** Reporting only: never retain composer text, suggestions, frames, hints or exception messages. */
|
||||
export interface InputReadDiagnostic {
|
||||
check: "composer" | "draft-aware";
|
||||
status: "empty" | "text" | "typed" | "likely-suggestion" | "typed-with-suggestion" | "unknown" | "no-input-box";
|
||||
confidence: "high" | "medium" | "low" | "unknown" | null;
|
||||
reason: string;
|
||||
cursor: { present: boolean; valid: boolean | null; atEnd: boolean | null };
|
||||
/** null means this reader does not report whether a timeout occurred. */
|
||||
timedOut: boolean | null;
|
||||
}
|
||||
|
||||
const reasons = new Set(["capture_unavailable", "pane_in_mode", "invalid_cursor", "incomplete_frame",
|
||||
"ambiguous_composer", "unrecognized_composer", "read_timeout", "read_budget_spent", "cursor_moved",
|
||||
"cursor_unavailable", "cursor_outside", "unrecognized_input_box", "input_not_clear",
|
||||
"paste_controls", "paste_long", "paste_viewport"]);
|
||||
|
||||
/** Copy only fixed vocabulary and cursor flags. Unknown reader reasons remain content-free. */
|
||||
export function inputReadDiagnostic(check: InputReadDiagnostic["check"], reading: {
|
||||
state?: string; status?: string; confidence?: string; unknownReason?: string;
|
||||
cursor?: { x: number; y: number; width: number; height: number } | null;
|
||||
cursorAtEnd?: boolean; evidence?: string[];
|
||||
} | null, reason?: string): InputReadDiagnostic {
|
||||
const evidence = reading?.evidence ?? [];
|
||||
const timeout = evidence.includes("input box reads timed out");
|
||||
const derived = reason ?? reading?.unknownReason ?? (timeout ? "read_timeout"
|
||||
: evidence.includes("input box budget spent") ? "read_budget_spent"
|
||||
: evidence.includes("cursor moved during the capture") ? "cursor_moved"
|
||||
: evidence.includes("cursor unavailable") ? "cursor_unavailable"
|
||||
: evidence.some(e => e === "cursor outside the input box" || e === "cursor seen once outside the input box") ? "cursor_outside"
|
||||
: reading?.status === "no-input-box" ? "unrecognized_input_box"
|
||||
: reading?.status === "unknown" || !reading ? "capture_unavailable" : "input_not_clear");
|
||||
const status = reading?.status ?? reading?.state;
|
||||
const confidence = reading?.confidence;
|
||||
const cursor = reading?.cursor;
|
||||
return {
|
||||
check,
|
||||
status: ["empty", "text", "typed", "likely-suggestion", "typed-with-suggestion", "unknown", "no-input-box"].includes(status ?? "")
|
||||
? status as InputReadDiagnostic["status"] : "unknown",
|
||||
confidence: ["high", "medium", "low", "unknown"].includes(confidence ?? "")
|
||||
? confidence as InputReadDiagnostic["confidence"] : null,
|
||||
reason: reasons.has(derived) ? derived : "capture_unavailable",
|
||||
cursor: { present: !!cursor, valid: cursor ? [cursor.x, cursor.y, cursor.width, cursor.height].every(Number.isInteger)
|
||||
&& cursor.x >= 0 && cursor.y >= 0 && cursor.width > cursor.x && cursor.height > cursor.y : null,
|
||||
atEnd: typeof reading?.cursorAtEnd === "boolean" ? reading.cursorAtEnd : null },
|
||||
timedOut: timeout ? true : null,
|
||||
};
|
||||
}
|
||||
@@ -85,7 +85,8 @@ import { parseReplyToChoice, parseReplyToThreadMissing, formatReplyToChoice, des
|
||||
import { classifyDestination, type DestinationClass } from "./gateway/destination-resolver.js";
|
||||
import { join } from "node:path";
|
||||
import { SettingsStore } from "./user-settings/settings-store.js";
|
||||
import { hostedWakeFor, isHostedMemberAt, type RosterWake } from "./rosters.js";
|
||||
import { hostedMembersAt, hostedWakeFor, isHostedMemberAt, type RosterMember, type RosterWake } from "./rosters.js";
|
||||
import { memberCheckInStatus, type MemberCheckIn } from "./connection-contract.js";
|
||||
import { hostedRingMayHaveArrived, ringHostedWake } from "./hosted-wake.js";
|
||||
import { effectiveGateRoles } from "./gate-predicate.js";
|
||||
import {
|
||||
@@ -651,6 +652,16 @@ function defaultIsHostedMember(destination: string): boolean {
|
||||
}
|
||||
}
|
||||
|
||||
/** The hosted members the authored rosters list on this host. Empty when the rosters or this host's id can't be read. */
|
||||
function defaultHostedMembers(): RosterMember[] {
|
||||
try {
|
||||
const root = new SettingsStore().resolveOne("workspace.root").value;
|
||||
return typeof root === "string" && root ? hostedMembersAt(join(root, "rosters"), getSelfHostId()) : [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/** A session's latest read of its own pending inbox: when it was seen, never seen, or unknown (the record can't be read). */
|
||||
export type InboxCheckIn = { state: "seen"; at: string } | { state: "never" } | { state: "unknown" };
|
||||
|
||||
@@ -1098,6 +1109,7 @@ export class QueueRepository {
|
||||
private readonly hostedWakeFn: (destination: string) => RosterWake | null;
|
||||
/** Whether the rosters list an address as a hosted member, trigger or not. */
|
||||
private readonly hostedMemberFn: (destination: string) => boolean;
|
||||
private readonly hostedMembersFn: () => RosterMember[];
|
||||
private readonly permissionDecider?: (ask: QueueItem, requester: string) => { session: string; describe?: string } | null;
|
||||
private readonly hostedWakeFetch: typeof fetch | undefined;
|
||||
/** When this process last wrote each session's inbox check-in, so a seat that polls often writes at most once a minute. */
|
||||
@@ -1151,6 +1163,7 @@ export class QueueRepository {
|
||||
sharedChannels?: () => ReadonlyArray<{ address: string }>;
|
||||
hostedWake?: (destination: string) => RosterWake | null;
|
||||
isHostedMember?: (destination: string) => boolean;
|
||||
hostedMembers?: () => RosterMember[];
|
||||
hostedWakeFetch?: typeof fetch;
|
||||
/** Who decides a runner's permission ask after its person answers, while the runner waits (startup wires the
|
||||
* receiving-agent order, `permissionDeciderFor`). Absent, or null for an ask: the ask closes with the answer, as
|
||||
@@ -1174,6 +1187,7 @@ export class QueueRepository {
|
||||
this.sharedChannelsFn = opts?.sharedChannels ?? (() => sharedChannelAddresses());
|
||||
this.hostedWakeFn = opts?.hostedWake ?? defaultHostedWake;
|
||||
this.hostedMemberFn = opts?.isHostedMember ?? defaultIsHostedMember;
|
||||
this.hostedMembersFn = opts?.hostedMembers ?? defaultHostedMembers;
|
||||
this.permissionDecider = opts?.permissionDecider;
|
||||
this.hostedWakeFetch = opts?.hostedWakeFetch;
|
||||
this.hasTargetRepoColumn = detectQueueColumn(db, "target_repo");
|
||||
@@ -5271,6 +5285,24 @@ export class QueueRepository {
|
||||
}
|
||||
}
|
||||
|
||||
/** Each hosted member on this host with its check-in status: its stated cadence against its own inbox reads. A status
|
||||
* only; nothing here changes how work reaches it. */
|
||||
hostedMemberCheckIns(now: Date = new Date()): Array<{ seat: string; rung: boolean } & MemberCheckIn> {
|
||||
return this.hostedMembersFn().map((m) => ({ seat: m.seat, rung: m.wake !== undefined,
|
||||
...memberCheckInStatus({ checkInMinutes: m.check_in_minutes, onRequest: m.check_in_on_request, cadenceProblem: m.check_in_problem,
|
||||
checkIn: this.lastInboxCheckIn(m.seat), now }) }));
|
||||
}
|
||||
|
||||
/** One member's check-in status, as `hostedMemberCheckIns` gives it. A seat the rosters don't list here (or can't be
|
||||
* read for) states no cadence, so it reads by its check-ins alone. */
|
||||
hostedMemberCheckIn(seat: string, now: Date = new Date()): MemberCheckIn {
|
||||
let member: RosterMember | undefined;
|
||||
try { member = this.hostedMembersFn().find((m) => this.isSameSession(m.seat, seat)); } catch { member = undefined; }
|
||||
// The roster's spelling of the seat, as the list read uses, so both read the same check-ins.
|
||||
return memberCheckInStatus({ checkInMinutes: member?.check_in_minutes, onRequest: member?.check_in_on_request,
|
||||
cadenceProblem: member?.check_in_problem, checkIn: this.lastInboxCheckIn(member?.seat ?? seat), now });
|
||||
}
|
||||
|
||||
/** This session's latest recorded check-in under any of its names. */
|
||||
lastInboxCheckIn(session: string): InboxCheckIn {
|
||||
try {
|
||||
|
||||
@@ -30,11 +30,11 @@ import { resolveOperatorSeat, resolveRecoveryAgentSeat, resolveSessionNodeId } f
|
||||
import { nodeExecutorStoppedReason } from "./node-inventory.js";
|
||||
import { isBoundTerminalSeat } from "./terminal-seat.js";
|
||||
import type Database from "better-sqlite3";
|
||||
import { createLinkedSuccessorLookup, deriveCrossHostSuccessorId, hasLinkedSuccessor, matchesRecordedHandoffTarget, QueueRepositoryError, type InboxCheckIn, type QueueItem, type QueueRepository } from "./queue-repository.js";
|
||||
import { createLinkedSuccessorLookup, deriveCrossHostSuccessorId, hasLinkedSuccessor, matchesRecordedHandoffTarget, QueueRepositoryError, type QueueItem, type QueueRepository } from "./queue-repository.js";
|
||||
import { stalledPickupFinding } from "./queue-pickup.js";
|
||||
import { SettingsStore } from "./user-settings/settings-store.js";
|
||||
import { loadHostRegistry } from "./hosts/hosts-registry-reader.js";
|
||||
import { readWakeOutcome } from "./connection-contract.js";
|
||||
import { describeCheckIn, readWakeOutcome, type MemberCheckIn } from "./connection-contract.js";
|
||||
|
||||
export const STUCK_SWEEP_INTERVAL_KEY = "queue.stuck_sweep_interval_seconds";
|
||||
export const DEFAULT_STUCK_SWEEP_INTERVAL_SECONDS = 300;
|
||||
@@ -621,20 +621,36 @@ function unclaimedBacklogText(seat: string, rows: BacklogRow[], thresholdMinutes
|
||||
* and what to do. Its work's age runs from each row's own time, which a check-in never resets. Built from the rows' fixed
|
||||
* facts and the member's check-in state, never a current check-in's exact time, so a pass rewrites it only when the rows
|
||||
* or that state change. */
|
||||
function polledBacklogText(member: string, rows: BacklogRow[], thresholdMinutes: number, checkIn: InboxCheckIn,
|
||||
function polledBacklogText(member: string, rows: BacklogRow[], thresholdMinutes: number, checkIn: MemberCheckIn,
|
||||
now: Date): { summary: string; body: string } {
|
||||
const { sorted, listed, count } = backlogListing(rows);
|
||||
const recent = checkIn.state === "seen" && now.getTime() - Date.parse(checkIn.at) <= thresholdMinutes * 60_000;
|
||||
const checkIns = recent
|
||||
? `checking in: it read its own inbox within the last ${thresholdMinutes} min`
|
||||
: checkIn.state === "seen"
|
||||
? `no check-in observed since ${checkIn.at}`
|
||||
: checkIn.state === "never" ? "no check-in observed yet" : "unknown: its check-ins couldn't be read";
|
||||
// The member's shared check-in status, in the same words `rig roster` uses (`describeCheckIn`). Only the words follow
|
||||
// it; when this notice is filed doesn't. Where that line would carry the time of a check-in that keeps moving (checking
|
||||
// in, on request, unknown), the notice says instead whether the member read its inbox within the threshold, so a
|
||||
// check-in rewrites the notice only when it changes that, never its time alone. An overdue or never-seen member's line
|
||||
// is fixed until it checks in, so it keeps its evidence.
|
||||
const at = checkIn.lastCheckInAt;
|
||||
const lately = at !== null && now.getTime() - Date.parse(at) <= thresholdMinutes * 60_000;
|
||||
const recency = at === null ? "" : lately ? `; it read its own inbox within the last ${thresholdMinutes} min` : `; no check-in since ${at}`;
|
||||
const recent = checkIn.status === "checking-in" || ((checkIn.status === "on-request" || checkIn.status === "unknown") && lately);
|
||||
const checkIns = ((): string => {
|
||||
switch (checkIn.status) {
|
||||
case "checking-in": return `checking in: its last check-in is within twice its expected ${checkIn.expectedMinutes} min`;
|
||||
case "on-request": return `on request: it checks its queue when a person asks it${recency}`;
|
||||
case "unknown": return `unknown (${checkIn.reason ?? "its check-in status couldn't be read"})${recency}`;
|
||||
case "check-in-overdue":
|
||||
case "no-check-in-yet":
|
||||
case "disconnected":
|
||||
return describeCheckIn(checkIn);
|
||||
}
|
||||
})();
|
||||
const why = recent
|
||||
? "it's checking in, but this work is still unclaimed. Check whether it can do this work, or route it to another seat"
|
||||
: checkIn.state === "unknown"
|
||||
? "nobody has claimed these rows, and whether it's reading its queue isn't known. Ask its owner, or route the work to another seat"
|
||||
: "nobody has claimed these rows, and no read of its own inbox has been observed in that time. Ask its owner to check it's running, or route the work to another seat";
|
||||
? "it's reading its own inbox, but this work is still unclaimed. Check whether it can do this work, or route it to another seat"
|
||||
: checkIn.status === "check-in-overdue"
|
||||
? "nobody has claimed these rows, and it hasn't checked in as often as its setup expects. Ask its owner to check its schedule or a pending approval, or route the work to another seat"
|
||||
: checkIn.status === "unknown" && at === null
|
||||
? "nobody has claimed these rows, and whether it's reading its queue isn't known. Ask its owner, or route the work to another seat"
|
||||
: "nobody has claimed these rows, and no read of its own inbox has been observed in that time. Ask its owner to check it's running, or route the work to another seat";
|
||||
return {
|
||||
summary: `Stuck sweep: unclaimed-backlog on ${member} (${count})`,
|
||||
body:
|
||||
@@ -1495,7 +1511,7 @@ export async function runStuckSweep(deps: StuckSweepDeps): Promise<StuckSweepRes
|
||||
// A polling member's notice says whether it's checking in; an agent seat's names the route to add instead.
|
||||
const polled = rows.length > 0 ? rows.some(polledUnclaimed) : pollsForWork(seat);
|
||||
const backlogText = (standsFor: BacklogRow[]) => polled
|
||||
? polledBacklogText(seat, standsFor, ageMinutes, deps.queueRepo.lastInboxCheckIn(seat), now)
|
||||
? polledBacklogText(seat, standsFor, ageMinutes, deps.queueRepo.hostedMemberCheckIn(seat, now), now)
|
||||
: unclaimedBacklogText(seat, standsFor, ageMinutes, node);
|
||||
// A polling member's notice goes where its rows' findings went: the person, else their creator when they share one,
|
||||
// else the operator agent. With none, it stays with the member, which reads it when it next checks.
|
||||
|
||||
@@ -26,6 +26,14 @@ export interface RosterMember extends Reference {
|
||||
hosted?: boolean;
|
||||
/** A hosted member's trigger, rung when a row lands for it. */
|
||||
wake?: RosterWake;
|
||||
/** How often a hosted member is expected to check its queue, in minutes, as its person set it up (an hourly schedule
|
||||
* is 60). Stated, not observed: absent means it checks only when a person prompts it. */
|
||||
check_in_minutes?: number;
|
||||
/** Its person set it to check in when asked, not on a schedule. */
|
||||
check_in_on_request?: true;
|
||||
/** Why an authored `check_in_minutes` can't be used. The roster still loads; the member's check-in status reads
|
||||
* unknown with this reason. */
|
||||
check_in_problem?: string;
|
||||
}
|
||||
export interface Roster {
|
||||
version: 1;
|
||||
@@ -65,9 +73,19 @@ export function parseRoster(value: unknown): Roster {
|
||||
|| (m.hosted !== undefined && typeof m.hosted !== "boolean")) {
|
||||
throw new Error("Each member needs seat, host, capabilities, engagement, use_when and why (optional caveat, and hosted: true or false)");
|
||||
}
|
||||
// A cadence that can't be used never stops the roster loading: the member's check-in status reads unknown instead.
|
||||
const cadence = m.check_in_minutes, onRequest = m.check_in_on_request;
|
||||
const cadenceProblem = cadence === undefined && onRequest === undefined ? null
|
||||
: m.hosted !== true ? "check_in_minutes and check_in_on_request are for a connected assistant (a member marked hosted: true)"
|
||||
: onRequest !== undefined && onRequest !== true ? `check_in_on_request ${JSON.stringify(onRequest)} isn't true`
|
||||
: onRequest === true && cadence !== undefined ? "it states both check_in_minutes and check_in_on_request"
|
||||
: cadence !== undefined && (!Number.isInteger(cadence) || (cadence as number) <= 0) ? `check_in_minutes ${JSON.stringify(cadence)} isn't a positive whole number of minutes`
|
||||
: null;
|
||||
return { seat: m.seat, host: m.host, capabilities: m.capabilities, engagement: m.engagement,
|
||||
use_when: m.use_when as string, why: m.why as string, ...(m.caveat === undefined ? {} : { caveat: m.caveat as string }),
|
||||
...(m.hosted === true ? { hosted: true } : {}), ...(m.wake === undefined ? {} : { wake: parseWake(m.wake) }) };
|
||||
...(m.hosted === true ? { hosted: true } : {}), ...(m.wake === undefined ? {} : { wake: parseWake(m.wake) }),
|
||||
...(cadenceProblem ? { check_in_problem: cadenceProblem }
|
||||
: { ...(cadence === undefined ? {} : { check_in_minutes: cadence as number }), ...(onRequest === true ? { check_in_on_request: true as const } : {}) }) };
|
||||
});
|
||||
return { version: 1, id: value.id as string, name: value.name as string, purpose: value.purpose as string,
|
||||
curator: { seat: value.curator.seat, host: value.curator.host }, updated_at: value.updated_at as string, members };
|
||||
@@ -103,6 +121,17 @@ export function hostedWakeFor(destination: string, folder: string, selfHostId: s
|
||||
return null;
|
||||
}
|
||||
|
||||
/** The hosted members the rosters list on this host, one per seat: where two rosters list the same seat, the first by
|
||||
* file name wins. Empty when this host's id is unknown. */
|
||||
export function hostedMembersAt(folder: string, selfHostId: string | null): RosterMember[] {
|
||||
if (!selfHostId) return [];
|
||||
const bySeat = new Map<string, RosterMember>();
|
||||
for (const roster of readRosters(folder).rosters) {
|
||||
for (const m of roster.members) if (m.hosted === true && m.host === selfHostId && !bySeat.has(m.seat)) bySeat.set(m.seat, m);
|
||||
}
|
||||
return [...bySeat.values()];
|
||||
}
|
||||
|
||||
/** Whether a roster on this host lists this address as a hosted member, with or without a trigger. */
|
||||
export function isHostedMemberAt(destination: string, folder: string, selfHostId: string | null): boolean {
|
||||
if (!selfHostId) return false;
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { inputReadDiagnostic, type InputReadDiagnostic } from "./input-read-diagnostic.js";
|
||||
import { hasOtherSeatHistory, isStoppedSeatPredecessor, resolveCurrentSessionAddress } from "./session-registry.js";
|
||||
import { AsyncLocalStorage } from "node:async_hooks";
|
||||
import { randomUUID } from "node:crypto";
|
||||
@@ -28,7 +29,7 @@ interface Lease {
|
||||
}
|
||||
|
||||
export class DeliveryGuardError extends Error {
|
||||
constructor(readonly code: string, message: string) { super(message); }
|
||||
constructor(readonly code: string, message: string, readonly inputReadFailure?: InputReadDiagnostic) { super(message); }
|
||||
|
||||
// Hono's error protocol also preserves this typed refusal on lifecycle routes.
|
||||
getResponse(): Response {
|
||||
@@ -440,14 +441,14 @@ export class SeatDeliveryGuard {
|
||||
this.checkInput(name);
|
||||
return observed;
|
||||
};
|
||||
const unknown = () => new DeliveryGuardError("draft_input_unknown", "The current input cannot be read. Automatic delivery is held.");
|
||||
const unknown = (reading: ComposerInput | null) => new DeliveryGuardError("draft_input_unknown", "The current input cannot be read. Automatic delivery is held.", inputReadDiagnostic("draft-aware", reading));
|
||||
let input = await read();
|
||||
// Claude may still be redrawing this lease's own paste: one bounded re-read before refusing its Enter.
|
||||
if (submit && lease.staged && (!input || input.state === "unknown")) { await this.sleep(OWN_INPUT_REREAD_MS); input = await read(); }
|
||||
if (!input || input.state === "unknown") {
|
||||
// No Enter was sent, so the paste stays there unsubmitted, and the next delivery may recognize it as OpenRig's.
|
||||
if (submit && lease.staged) this.leftovers.set(target.nodeId, { target: lease.target, humanEpoch: lease.humanEpoch, ...lease.staged });
|
||||
throw unknown();
|
||||
throw unknown(input);
|
||||
}
|
||||
if (leftover === "recognize" && !submit && this.ownsLeftover(target, input)) { lease.ownLeftover = "recognized"; return; }
|
||||
if (typeof leftover === "object" && lease.ownLeftover === "recognized") {
|
||||
@@ -466,7 +467,7 @@ export class SeatDeliveryGuard {
|
||||
lease.ownLeftover = "submitted";
|
||||
await this.sleep(OWN_INPUT_REREAD_MS);
|
||||
const after = await read();
|
||||
if (!after || after.state === "unknown") throw unknown();
|
||||
if (!after || after.state === "unknown") throw unknown(after);
|
||||
input = after;
|
||||
}
|
||||
}
|
||||
@@ -489,7 +490,7 @@ export class SeatDeliveryGuard {
|
||||
: uncertainty === "long"
|
||||
? "OpenRig couldn't confirm ownership of Codex's collapsed long paste. Send a shorter message; nothing was pasted. Turning this seat's typing guard off explicitly disables draft protection."
|
||||
: "OpenRig couldn't confirm the complete message would stay visible in this Codex viewport. Open a larger terminal or shorten the message, then retry this request; nothing was pasted.";
|
||||
throw new DeliveryGuardError("draft_input_unknown", message);
|
||||
throw new DeliveryGuardError("draft_input_unknown", message, inputReadDiagnostic("draft-aware", input, `paste_${uncertainty}`));
|
||||
}
|
||||
lease.inputFrame = input.frame;
|
||||
}
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { inputReadDiagnostic, type InputReadDiagnostic } from "./input-read-diagnostic.js";
|
||||
import { expectedConversation, sameConversation, resolveExpectedConversation, type ExpectedConversation } from "./expected-conversation.js";
|
||||
import type { ContextUsageStore } from "./context-usage-store.js";
|
||||
import { physicalSessionName } from "./session-registry.js";
|
||||
@@ -2176,6 +2177,7 @@ export interface SendResult {
|
||||
heldInputBox?: InputBoxAnnotation["status"];
|
||||
/** The existing preserve-input reading, without the draft or suggestion text. No extra capture. */
|
||||
heldInputBoxReading?: Pick<InputBoxAnnotation, "status" | "confidence" | "evidence" | "cursor">;
|
||||
inputReadFailure?: InputReadDiagnostic;
|
||||
}
|
||||
|
||||
/** One reading of a Claude Code or Codex input box: `exact` (the sent text, word boundaries kept), `ambiguous`
|
||||
@@ -2823,6 +2825,7 @@ export class SessionTransport {
|
||||
sent: written ? true : attempted ? undefined : false,
|
||||
inputPhase: written ? "written" : attempted ? "attempted" : "not_reached",
|
||||
reason: (error as { code?: string }).code ?? "expected_identity_unavailable", error: (error as Error).message,
|
||||
...(error instanceof DeliveryGuardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}),
|
||||
...(receiptUnconfirmed ? { warning: "Conditional input receipt could not be finalized; inspect before retrying." } : {}) };
|
||||
}
|
||||
}
|
||||
@@ -2944,17 +2947,20 @@ export class SessionTransport {
|
||||
if ((error as { inputWritten?: boolean }).inputWritten === true) {
|
||||
// Only a throw before the Enter key reached the adapter positively means Enter was never sent.
|
||||
const enter = (error as { enterAttempted?: boolean }).enterAttempted === true ? "the submission (Enter) outcome is unconfirmed" : "Enter was not sent";
|
||||
return { ok: false, sessionName, sent: true, outcome: "failed", inputPhase: "written", reason: (error as { code?: string }).code ?? "submit_refused",
|
||||
return { ok: false, sessionName, sent: true, outcome: "failed", inputPhase: "written",
|
||||
...(error instanceof DeliveryGuardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}), reason: (error as { code?: string }).code ?? "submit_refused",
|
||||
error: `Text was written to '${sessionName}' but ${enter}: ${(error as Error).message}` };
|
||||
}
|
||||
// A throw after the text write was issued: whether it landed is unknown, so it is not "nothing sent" either.
|
||||
if ((error as { inputAttempted?: boolean }).inputAttempted === true) {
|
||||
return { ok: false, sessionName, outcome: "failed", inputPhase: "attempted", reason: (error as { code?: string }).code ?? "delivery_indeterminate",
|
||||
return { ok: false, sessionName, outcome: "failed", inputPhase: "attempted",
|
||||
...(error instanceof DeliveryGuardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}), reason: (error as { code?: string }).code ?? "delivery_indeterminate",
|
||||
error: `A text write to '${sessionName}' was issued, but its result was lost: ${(error as Error).message}` };
|
||||
}
|
||||
const noInputCode = error instanceof OutboxHandlerError && error.code === "retained_quota_full" ? "retained_quota_full" as const
|
||||
: error instanceof DeliveryGuardError && error.code === "terminal_seat" ? "terminal_seat" as const : undefined;
|
||||
return { ok: false, sessionName, sent: false, reason: (error as { code?: string }).code ?? "guard_unavailable", error: (error as Error).message,
|
||||
...(error instanceof DeliveryGuardError && error.inputReadFailure ? { inputReadFailure: error.inputReadFailure } : {}),
|
||||
...(noInputCode ? { noInputCode, inputPhase: "not_reached" as const } : {}) };
|
||||
}
|
||||
}
|
||||
@@ -3350,7 +3356,7 @@ export class SessionTransport {
|
||||
}
|
||||
if (!submitResult.ok && !enterIssued && submitResult.code === CALLER_CANCELLED && cancel?.aborted) return callerCancelled(sessionName);
|
||||
if (!submitResult.ok) {
|
||||
return { ok: false, sessionName, inputPhase: enterIssued ? "attempted" : "not_reached", ...(!enterIssued ? { sent: false } : {}), reason: submitResult.code?.startsWith("expected_identity_") ? submitResult.code : "submit_failed", outcome: "failed", error: enterIssued
|
||||
return { ok: false, sessionName, ...(submitResult.inputReadFailure ? { inputReadFailure: submitResult.inputReadFailure } : {}), inputPhase: enterIssued ? "attempted" : "not_reached", ...(!enterIssued ? { sent: false } : {}), reason: submitResult.code?.startsWith("expected_identity_") ? submitResult.code : "submit_failed", outcome: "failed", error: enterIssued
|
||||
? `submitOnly: Enter reported a failure for '${sessionName}'; submission is unconfirmed: ${submitResult.message}`
|
||||
: `submitOnly: Enter was not sent to '${sessionName}': ${submitResult.message}` };
|
||||
}
|
||||
@@ -3700,7 +3706,7 @@ export class SessionTransport {
|
||||
heldInputBoxReading = { status, confidence, evidence, cursor };
|
||||
}
|
||||
throw new DeliveryGuardError(box?.status === "typed" ? "draft_input_busy" : "draft_input_unknown",
|
||||
`Managed input waits for an empty composer (it reads ${box?.status ?? "unknown"}). Your draft was left untouched; inspect the pane before retrying.`);
|
||||
`Managed input waits for an empty composer (it reads ${box?.status ?? "unknown"}). Your draft was left untouched; inspect the pane before retrying.`, inputReadDiagnostic("composer", box));
|
||||
}
|
||||
if (!clear) pastedOverInputBox = box?.status ?? "unknown";
|
||||
} : undefined;
|
||||
@@ -3735,6 +3741,7 @@ export class SessionTransport {
|
||||
ok: false,
|
||||
sessionName,
|
||||
reason: policyRefusal ? textResult.code : "send_failed",
|
||||
...(textResult.inputReadFailure ? { inputReadFailure: textResult.inputReadFailure } : {}),
|
||||
outcome: "failed",
|
||||
// The adapter's draft checks run before the paste (only file/buffer preparation precedes them), and its
|
||||
// after-paste observation cannot fail the write, so a draft_* receipt proves the target input was never reached.
|
||||
@@ -3803,6 +3810,7 @@ export class SessionTransport {
|
||||
ok: false,
|
||||
sessionName,
|
||||
reason: policyRefusal ? submitResult.code : "submit_failed",
|
||||
...(submitResult.inputReadFailure ? { inputReadFailure: submitResult.inputReadFailure } : {}),
|
||||
outcome: "failed",
|
||||
inputPhase: "written",
|
||||
error: policyRefusal
|
||||
|
||||
@@ -3,7 +3,6 @@ import { performance } from "node:perf_hooks";
|
||||
import { Worker } from "node:worker_threads";
|
||||
import type { MiddlewareHandler } from "hono";
|
||||
|
||||
export const SLOW_OPERATION_BARRIER_TIMEOUT_MS = 250;
|
||||
export const SLOW_OPERATION_THRESHOLD_MS = 250;
|
||||
export const SLOW_OPERATION_ROTATION_BYTES = 1024 * 1024;
|
||||
export const SLOW_OPERATION_ROTATION_COUNT = 3;
|
||||
@@ -89,7 +88,6 @@ interface SlowOpRecorderOptions {
|
||||
maxBytes?: number;
|
||||
rotationCount?: number;
|
||||
slowThresholdMs?: number;
|
||||
barrierTimeoutMs?: number;
|
||||
}
|
||||
|
||||
interface Span {
|
||||
@@ -141,13 +139,7 @@ const WORKER_SOURCE = String.raw`
|
||||
ok = false;
|
||||
error = caught instanceof Error ? caught.message : String(caught);
|
||||
}
|
||||
if (message.signal) {
|
||||
const state = new Int32Array(message.signal);
|
||||
Atomics.store(state, 0, ok ? 1 : 2);
|
||||
Atomics.notify(state, 0);
|
||||
} else if (message.id) {
|
||||
parentPort.postMessage({ id: message.id, ok, error });
|
||||
}
|
||||
if (message.id) parentPort.postMessage({ id: message.id, ok, error });
|
||||
});
|
||||
`;
|
||||
|
||||
@@ -157,7 +149,6 @@ export class SlowOpRecorder implements SlowOperationInstrumentation {
|
||||
private readonly maxBytes: number;
|
||||
private readonly rotationCount: number;
|
||||
private readonly slowThresholdMs: number;
|
||||
private readonly barrierTimeoutMs: number;
|
||||
private readonly pending = new Map<string, { resolve: () => void; reject: (error: unknown) => void }>();
|
||||
private degraded: SlowOperationSnapshot = { healthy: true };
|
||||
private degradedHandler?: (snapshot: Required<Pick<SlowOperationSnapshot, "reason" | "site">>) => void;
|
||||
@@ -181,7 +172,6 @@ export class SlowOpRecorder implements SlowOperationInstrumentation {
|
||||
this.maxBytes = options.maxBytes ?? SLOW_OPERATION_ROTATION_BYTES;
|
||||
this.rotationCount = options.rotationCount ?? SLOW_OPERATION_ROTATION_COUNT;
|
||||
this.slowThresholdMs = options.slowThresholdMs ?? SLOW_OPERATION_THRESHOLD_MS;
|
||||
this.barrierTimeoutMs = options.barrierTimeoutMs ?? SLOW_OPERATION_BARRIER_TIMEOUT_MS;
|
||||
this.worker = new Worker(WORKER_SOURCE, { eval: true, execArgv: [] });
|
||||
this.worker.unref();
|
||||
this.worker.on("message", (message: { id?: string; ok?: boolean }) => {
|
||||
@@ -220,27 +210,10 @@ export class SlowOpRecorder implements SlowOperationInstrumentation {
|
||||
|
||||
beginSyncSpan(site: string): Span {
|
||||
const span: Span = { spanId: randomUUID(), site, startedAt: performance.now() };
|
||||
const signal = new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT);
|
||||
const state = new Int32Array(signal);
|
||||
try {
|
||||
this.worker.postMessage({
|
||||
type: "append",
|
||||
signal,
|
||||
logPath: this.logPath,
|
||||
maxBytes: this.maxBytes,
|
||||
rotationCount: this.rotationCount,
|
||||
record: { v: 1, ts: new Date().toISOString(), spanId: span.spanId, phase: "begin", site },
|
||||
});
|
||||
} catch {
|
||||
// Synchronous postMessage failure means the Worker is gone: degrade
|
||||
// through the terminal transition but still return the span so the
|
||||
// wrapped operation runs to completion (its value/error stays exact).
|
||||
this.handleTerminalFailure("recorder_post_failed", site);
|
||||
return span;
|
||||
}
|
||||
const wait = Atomics.wait(state, 0, 0, this.barrierTimeoutMs);
|
||||
if (wait === "timed-out") this.markDegraded("begin_barrier_timeout", site);
|
||||
else if (Atomics.load(state, 0) !== 1) this.markDegraded("begin_barrier_failed", site);
|
||||
// Queue before the operation without waiting on worker startup or fsync. The
|
||||
// worker can write even while the main thread is wedged; only flush/close
|
||||
// acknowledge durability. A crash before the worker appends can lose begin.
|
||||
this.appendAsync({ v: 1, ts: new Date().toISOString(), spanId: span.spanId, phase: "begin", site });
|
||||
return span;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { execFile } from "node:child_process";
|
||||
import type { TmuxAdapter } from "../adapters/tmux.js";
|
||||
import { runSyncSite } from "./sync-site-wrap.js";
|
||||
import { runAsyncSite } from "./sync-site-wrap.js";
|
||||
|
||||
export interface TmuxOptionDefaultsDeps {
|
||||
tmuxAdapter: TmuxAdapter;
|
||||
@@ -17,11 +17,11 @@ export interface TmuxOptionDefaultsDeps {
|
||||
*/
|
||||
platform?: NodeJS.Platform;
|
||||
/**
|
||||
* OPR.0.4.6.02 S1 — cheap `command -v <bin>` probe for the linux
|
||||
* copy-command fallback chain. Defaults to a real shell probe; injectable
|
||||
* OPR.0.4.6.02 S1 — asynchronous `command -v <bin>` probe for the linux
|
||||
* copy-command fallback chain. Defaults to a bounded asynchronous shell probe; injectable
|
||||
* for deterministic tests.
|
||||
*/
|
||||
hasCommand?: (bin: string) => boolean;
|
||||
hasCommand?: (bin: string) => boolean | Promise<boolean>;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -44,7 +44,7 @@ export class TmuxOptionDefaultsApplier {
|
||||
private tmuxAdapter: TmuxAdapter;
|
||||
private readTmuxOptionDefaults: () => { statusBar: boolean };
|
||||
private platform: NodeJS.Platform;
|
||||
private hasCommand: (bin: string) => boolean;
|
||||
private hasCommand: (bin: string) => boolean | Promise<boolean>;
|
||||
private serverDefaultsAsserted = false;
|
||||
|
||||
constructor(deps: TmuxOptionDefaultsDeps) {
|
||||
@@ -120,7 +120,14 @@ export class TmuxOptionDefaultsApplier {
|
||||
warnings.push(`tmux "set-clipboard" server option not set: ${clip.message}`);
|
||||
}
|
||||
|
||||
const copyCommand = resolveCopyCommand(this.platform, this.hasCommand);
|
||||
let copyCommand: string | null;
|
||||
try {
|
||||
copyCommand = await resolveCopyCommand(this.platform, this.hasCommand);
|
||||
} catch {
|
||||
this.serverDefaultsAsserted = false;
|
||||
warnings.push("Clipboard lookup interrupted (250 ms limit); copy-command left unchanged (OSC 52 if unset). Will retry on the next fresh session.");
|
||||
return;
|
||||
}
|
||||
if (copyCommand !== null) {
|
||||
const cc = await this.tmuxAdapter.setServerOption("copy-command", copyCommand);
|
||||
if (!cc.ok) {
|
||||
@@ -141,14 +148,14 @@ export class TmuxOptionDefaultsApplier {
|
||||
* Pure: platform + a `command -v` probe in, the command string (or null)
|
||||
* out. One table, no side effects — the applier decides whether to write.
|
||||
*/
|
||||
export function resolveCopyCommand(
|
||||
export async function resolveCopyCommand(
|
||||
platform: NodeJS.Platform,
|
||||
hasCommand: (bin: string) => boolean,
|
||||
): string | null {
|
||||
hasCommand: (bin: string) => boolean | Promise<boolean>,
|
||||
): Promise<string | null> {
|
||||
if (platform === "darwin") return "pbcopy";
|
||||
if (platform === "linux") {
|
||||
if (hasCommand("wl-copy")) return "wl-copy";
|
||||
if (hasCommand("xclip")) return "xclip -selection clipboard -i";
|
||||
if (await hasCommand("wl-copy")) return "wl-copy";
|
||||
if (await hasCommand("xclip")) return "xclip -selection clipboard -i";
|
||||
return null;
|
||||
}
|
||||
return null;
|
||||
@@ -156,16 +163,18 @@ export function resolveCopyCommand(
|
||||
|
||||
/**
|
||||
* OPR.0.4.6.02 S1 — default `command -v <bin>` probe (POSIX shell builtin).
|
||||
* Returns true when the binary resolves on PATH. Cheap + best-effort; any
|
||||
* failure (spawn error, non-shell env) reads as absent.
|
||||
* Returns true when the binary resolves on PATH. A killed probe rejects so
|
||||
* the applier can warn and retry; other failures read as absent.
|
||||
*/
|
||||
function defaultHasCommand(bin: string): boolean {
|
||||
async function defaultHasCommand(bin: string): Promise<boolean> {
|
||||
try {
|
||||
const r = runSyncSite("tmux_options.command_v", () =>
|
||||
spawnSync(`command -v ${bin}`, { shell: true, stdio: "ignore" })
|
||||
);
|
||||
return r.status === 0;
|
||||
} catch {
|
||||
return await runAsyncSite("tmux_options.command_v", () => new Promise<boolean>((resolve, reject) => {
|
||||
execFile("/bin/sh", ["-c", 'command -v "$1"', "--", bin], {
|
||||
timeout: 250, killSignal: "SIGKILL", maxBuffer: 1024,
|
||||
}, error => error?.killed ? reject(error) : resolve(!error));
|
||||
}));
|
||||
} catch (error) {
|
||||
if ((error as { killed?: boolean } | null)?.killed) throw error;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -1441,6 +1441,11 @@ export function queueRoutes(): Hono {
|
||||
return c.json(items);
|
||||
});
|
||||
|
||||
// Each connected assistant on this host and whether it's checking its queue, with this daemon's own host id: the check-ins
|
||||
// are this host's, so a reader matches members to it without needing a terminal seat to learn it. A status only:
|
||||
// work addressed to an assistant that isn't checking in still queues and is rung as before. Precedes /:qitemId.
|
||||
app.get("/hosted-members", (c) => c.json({ hostId: getSelfHostId(), members: getRepo(c).hostedMemberCheckIns() }));
|
||||
|
||||
// GET /undelivered — surfaces pending create-path nudge failures plus active
|
||||
// human-notification episodes whose gateway delivery ledger failed or never posted.
|
||||
// Rig-scoped + bounded + compact-by-default, mirroring /overdue.
|
||||
|
||||
@@ -3,6 +3,9 @@
|
||||
// per poll, filling the seat's held quota. Real SessionTransport, real draft-aware delivery and outbox on an
|
||||
// in-memory database; the tmux adapter records commands instead of running them.
|
||||
import Database from "better-sqlite3";
|
||||
import { Hono } from "hono";
|
||||
import { inputReadDiagnostic } from "../src/domain/input-read-diagnostic.js";
|
||||
import { compactionRoutes } from "../src/routes/compaction.js";
|
||||
import stringWidth from "string-width";
|
||||
import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
@@ -83,7 +86,11 @@ async function fixture(opts: { pane: string | null; mode?: "off" | "draft-aware"
|
||||
const effects = () => ({ compactPastes: writes.filter((w) => w.startsWith("paste:/compact")).length, prepPastes: writes.filter((w) => w.startsWith("paste:") && !w.startsWith("paste:/compact")).length, enters: writes.filter((w) => w === "enter").length });
|
||||
const compactRows = (state?: string) => (db.prepare("SELECT outbox_id AS id, delivery_state AS state FROM outbox_entries WHERE body LIKE '/compact%'").all() as Array<{ id: string; state: string }>)
|
||||
.filter((r) => !state || r.state === state);
|
||||
return { db, e, transport, tmux, writes, compactRows, rows, effects, stageMap, body: () => body,
|
||||
return { db, e, transport, tmux, writes, compactRows, rows, effects, stageMap, recover: () => {
|
||||
const next = new ClaudeCompactionEnforcer({ resolveClaudeCompactionPolicy: () => policy } as any, transport,
|
||||
{ openrigHome: home, now: () => clock, resolveOccupantGeneration: () => "g1" });
|
||||
next.recoverSeat(input); return next;
|
||||
}, body: () => body,
|
||||
onStage: (fn: () => void) => { onStage = fn; }, activity: (s: string) => { activity = s; }, draft: (text: string) => { body = text; } };
|
||||
}
|
||||
|
||||
@@ -361,3 +368,95 @@ it("the managed idle wait treats only a recognized draft according to the ordina
|
||||
expect(await f.e.triggerManualCompact(input, { operatorInitiated: true, skipMap: true })).toMatchObject({ triggered: true });
|
||||
expect(f.effects()).toMatchObject({ prepPastes: 1, compactPastes: 1, enters: 2 });
|
||||
});
|
||||
|
||||
|
||||
it.each(["preparation", "compact"] as const)("records the %s refusal through the existing transport without content or input", async stage => {
|
||||
const f = await fixture({ pane: "%1", mode: stage === "preparation" ? "draft-aware" : "off" });
|
||||
const send = f.transport.send.bind(f.transport);
|
||||
vi.spyOn(f.transport, "send").mockImplementation(async (s, text, opts) => {
|
||||
if (stage === "preparation") {
|
||||
vi.mocked(f.tmux.captureComposerSnapshot).mockResolvedValue({ ...frame("PRIVATE_INPUT https://private.invalid/gateway"), inMode: true });
|
||||
} else if (text.startsWith("/compact")) {
|
||||
vi.spyOn(f.transport as any, "captureInputBox").mockResolvedValue({ status: "unknown", confidence: "unknown", cursor: null,
|
||||
evidence: ["input box reads timed out"], input: { text: "PRIVATE_INPUT" }, hint: "https://private.invalid/gateway" });
|
||||
return send(s, text, { ...opts, preserveInput: true });
|
||||
}
|
||||
return send(s, text, opts);
|
||||
});
|
||||
await f.e.triggerManualCompact(input, { operatorInitiated: true, skipMap: true });
|
||||
const state = f.e.getPreparationState(seat)!;
|
||||
const expected = stage === "preparation"
|
||||
? { stage, check: "draft-aware", status: "unknown", confidence: null, reason: "pane_in_mode", timedOut: null,
|
||||
cursor: { present: true, valid: true, atEnd: false } }
|
||||
: { stage, check: "composer", status: "unknown", confidence: "unknown", reason: "read_timeout", timedOut: true,
|
||||
cursor: { present: false, valid: null, atEnd: null } };
|
||||
expect(state).toMatchObject({ inputReadFailure: expected, compact: "not_sent" });
|
||||
expect(f.effects()).toEqual({ compactPastes: 0, prepPastes: stage === "preparation" ? 0 : 1, enters: stage === "preparation" ? 0 : 1 });
|
||||
const record = readFileSync(join(dirname(state.mapPath), COMPACTION_RECORD_FILE), "utf8");
|
||||
expect(JSON.parse(record)).toMatchObject({ inputReadFailure: expected });
|
||||
const app = new Hono();
|
||||
app.use("*", async (c, next) => { c.set("compactionEnforcer" as never, f.e as never); c.set("db" as never, f.db as never); await next(); });
|
||||
app.route("/api/compaction", compactionRoutes());
|
||||
const response = await app.request(`/api/compaction/state?session=${seat}`);
|
||||
const body = await response.text();
|
||||
expect(response.status).toBe(200);
|
||||
expect(JSON.parse(body)).toMatchObject({ preparation: { inputReadFailure: expected } });
|
||||
expect(record + body).not.toMatch(/PRIVATE_INPUT|private\.invalid/);
|
||||
});
|
||||
|
||||
it("distinguishes an opted-in composer timeout from the separate draft-aware check", async () => {
|
||||
const f = await fixture({ pane: "%1", mode: "off" });
|
||||
const send = f.transport.send.bind(f.transport);
|
||||
vi.spyOn(f.transport as any, "captureInputBox").mockResolvedValue({ status: "unknown", confidence: "unknown", cursor: null,
|
||||
evidence: ["input box reads timed out"], input: { text: "PRIVATE_INPUT" }, suggestion: { text: "PRIVATE_SUGGESTION" }, hint: "https://private.invalid/gateway" });
|
||||
vi.spyOn(f.transport, "send").mockImplementation((s, text, opts) => send(s, text, { ...opts, preserveInput: true }));
|
||||
await f.e.triggerManualCompact(input, { operatorInitiated: true, skipMap: true });
|
||||
const state = f.e.getPreparationState(seat)!;
|
||||
expect(state).toMatchObject({ delivery: "not_sent", inputReadFailure: { stage: "preparation", check: "composer", reason: "read_timeout", timedOut: true,
|
||||
status: "unknown", confidence: "unknown", cursor: { present: false, valid: null, atEnd: null } } });
|
||||
const record = readFileSync(join(dirname(state.mapPath), COMPACTION_RECORD_FILE), "utf8");
|
||||
expect(record).not.toMatch(/PRIVATE_INPUT|PRIVATE_SUGGESTION|private\.invalid/);
|
||||
expect(f.writes).toEqual([]);
|
||||
});
|
||||
|
||||
|
||||
it("retains a held completed map's compact diagnostic across record recovery", async () => {
|
||||
const f = await fixture({ pane: "%1", mode: "off" });
|
||||
await f.e.maybeAutoCompact(input); f.stageMap();
|
||||
const send = f.transport.send.bind(f.transport);
|
||||
vi.spyOn(f.transport as any, "captureInputBox").mockResolvedValue({ status: "unknown", confidence: "unknown", cursor: null, evidence: ["cursor unavailable"] });
|
||||
vi.spyOn(f.transport, "send").mockImplementation((s, text, opts) => send(s, text, { ...opts, preserveInput: true }));
|
||||
expect(await f.e.maybeAutoCompact(input)).toMatchObject({ triggered: false, reason: "prepared_awaiting_window" });
|
||||
const state = f.e.getPreparationState(seat)!;
|
||||
expect(state).toMatchObject({ prepared: true, compact: "not_sent", inputReadFailure: { stage: "compact", check: "composer", reason: "cursor_unavailable", timedOut: null } });
|
||||
const recordPath = join(dirname(state.mapPath), COMPACTION_RECORD_FILE);
|
||||
const record = JSON.parse(readFileSync(recordPath, "utf8"));
|
||||
// A new enforcer in this process deliberately ignores this process's live record; model a predecessor daemon.
|
||||
writeFileSync(recordPath, JSON.stringify({ ...record, daemon: "previous-daemon" }));
|
||||
expect(f.recover().getPreparationState(seat)?.inputReadFailure).toEqual(state.inputReadFailure);
|
||||
expect(f.effects()).toEqual({ compactPastes: 0, prepPastes: 1, enters: 1 });
|
||||
});
|
||||
|
||||
it("records the same diagnostic on an owed restore step without advancing it", async () => {
|
||||
const f = await fixture({ pane: "%1", mode: "off" });
|
||||
await f.e.triggerManualCompact(input, { operatorInitiated: true, skipMap: true });
|
||||
await f.e.maybeAutoCompact(input);
|
||||
expect(f.e.getOwedRestore(seat)?.stage).toBe("restore_prompt");
|
||||
const before = f.effects(), send = f.transport.send.bind(f.transport);
|
||||
vi.spyOn(f.transport as any, "captureInputBox").mockResolvedValue({ status: "unknown", confidence: "unknown", cursor: null, evidence: ["cursor unavailable"] });
|
||||
vi.spyOn(f.transport, "send").mockImplementation((s, text, opts) => send(s, text, { ...opts, preserveInput: true }));
|
||||
await f.e.maybeAutoCompact(input);
|
||||
expect(f.e.getOwedRestore(seat)).toMatchObject({ stage: "restore_prompt", lastDelivery: { stage: "restore_prompt", reason: "draft_input_unknown",
|
||||
inputReadFailure: { check: "composer", reason: "cursor_unavailable", timedOut: null } } });
|
||||
expect(f.effects()).toEqual(before);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ state: "unknown", unknownReason: "invalid_cursor", cursor: { x: -1, y: 0, width: 80, height: 24 }, reason: "invalid_cursor", valid: false },
|
||||
{ state: "unknown", unknownReason: "ambiguous_composer", reason: "ambiguous_composer", valid: null },
|
||||
{ state: "unknown", unknownReason: "PRIVATE_INPUT https://private.invalid/gateway", reason: "capture_unavailable", valid: null },
|
||||
])("diagnostic vocabulary excludes content and reports existing cursor validity ($reason)", ({ reason, valid, ...reading }) => {
|
||||
const safe = inputReadDiagnostic("draft-aware", { ...reading, rows: ["PRIVATE_INPUT"], suggestionRows: ["PRIVATE_SUGGESTION"], frame: "https://private.invalid/gateway" } as any);
|
||||
expect(safe).toMatchObject({ reason, cursor: { valid } });
|
||||
expect(JSON.stringify(safe)).not.toMatch(/PRIVATE_INPUT|PRIVATE_SUGGESTION|private\.invalid|rows|frame/);
|
||||
});
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { readWakeOutcome } from "../src/domain/connection-contract.js";
|
||||
import { describeCheckIn, memberCheckInStatus, readWakeOutcome } from "../src/domain/connection-contract.js";
|
||||
import { NUDGE_FAIL_PREFIX } from "../src/domain/policies/parked-owner-consumer.js";
|
||||
|
||||
describe("readWakeOutcome: every recorded nudge result reads as a delivery fact", () => {
|
||||
@@ -51,3 +51,73 @@ describe("readWakeOutcome: every recorded nudge result reads as a delivery fact"
|
||||
expect(readWakeOutcome(undefined)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("memberCheckInStatus: a hosted member's check-in status", () => {
|
||||
const now = new Date("2026-11-12T12:00:00.000Z");
|
||||
const seenAgo = (minutes: number) => ({ state: "seen" as const, at: new Date(now.getTime() - minutes * 60_000).toISOString() });
|
||||
const status = (checkInMinutes: number | null, checkIn: Parameters<typeof memberCheckInStatus>[0]["checkIn"], revokedAt?: string) =>
|
||||
memberCheckInStatus({ checkInMinutes, checkIn, now, ...(revokedAt ? { revokedAt } : {}) }).status;
|
||||
|
||||
it("with a stated cadence, one rule: checking in up to exactly twice it, overdue a moment after", () => {
|
||||
const at = (ms: number) => ({ state: "seen" as const, at: new Date(now.getTime() - ms).toISOString() });
|
||||
expect(status(60, seenAgo(5))).toBe("checking-in");
|
||||
expect(status(60, seenAgo(90))).toBe("checking-in"); // past one cadence is not yet overdue
|
||||
expect(status(60, at(2 * 60 * 60_000))).toBe("checking-in");
|
||||
expect(status(60, at(2 * 60 * 60_000 + 1))).toBe("check-in-overdue");
|
||||
expect(status(1, at(2 * 60_000 + 1))).toBe("check-in-overdue"); // any positive cadence works the same way
|
||||
});
|
||||
|
||||
it("a stated cadence that can't be used reads unknown with its reason, ahead of every other reading", () => {
|
||||
expect(memberCheckInStatus({ checkInMinutes: null, cadenceProblem: "check_in_minutes \"60\" isn't a positive whole number of minutes",
|
||||
checkIn: seenAgo(5), now })).toMatchObject({ status: "unknown", reason: expect.stringContaining("check_in_minutes") });
|
||||
expect(memberCheckInStatus({ checkInMinutes: 60, checkIn: { state: "unknown" }, now })).toMatchObject({ status: "unknown", reason: "its check-ins couldn't be read" });
|
||||
});
|
||||
|
||||
it("with no stated cadence it reads unknown with its last check-in, never on request, idle or overdue", () => {
|
||||
expect(memberCheckInStatus({ checkInMinutes: null, checkIn: seenAgo(5), now }))
|
||||
.toMatchObject({ status: "unknown", reason: "no check-in cadence declared", lastCheckInAt: seenAgo(5).at });
|
||||
expect(status(null, seenAgo(10_000))).toBe("unknown");
|
||||
});
|
||||
|
||||
it("on request only when its setup says so, whatever its last check-in's age", () => {
|
||||
const onRequest = (checkIn: Parameters<typeof memberCheckInStatus>[0]["checkIn"]) =>
|
||||
memberCheckInStatus({ checkInMinutes: null, onRequest: true, checkIn, now });
|
||||
expect(onRequest(seenAgo(5)).status).toBe("on-request");
|
||||
expect(onRequest(seenAgo(10_000))).toMatchObject({ status: "on-request", onRequest: true });
|
||||
expect(onRequest({ state: "never" }).status).toBe("no-check-in-yet");
|
||||
expect(memberCheckInStatus({ checkInMinutes: undefined, checkIn: seenAgo(5), now })).toMatchObject({ lastCheckInAt: seenAgo(5).at, expectedMinutes: null });
|
||||
});
|
||||
|
||||
it("no recorded check-in reads never, with or without a cadence; an unreadable record reads unknown, never idle or overdue", () => {
|
||||
expect(status(60, { state: "never" })).toBe("no-check-in-yet");
|
||||
expect(memberCheckInStatus({ checkInMinutes: null, checkIn: { state: "never" }, now })).toMatchObject({ status: "no-check-in-yet", expectedMinutes: null });
|
||||
expect(status(60, { state: "unknown" })).toBe("unknown");
|
||||
expect(status(60, { state: "seen", at: "not a time" })).toBe("unknown");
|
||||
});
|
||||
|
||||
it("disconnected comes only from a recorded revocation, ahead of everything else; silence never produces it", () => {
|
||||
expect(status(60, seenAgo(5), "2026-11-12T11:00:00.000Z")).toBe("disconnected");
|
||||
expect(status(60, seenAgo(100_000))).toBe("check-in-overdue");
|
||||
expect(status(null, { state: "never" })).not.toBe("disconnected");
|
||||
});
|
||||
});
|
||||
|
||||
describe("describeCheckIn: one line, with its evidence", () => {
|
||||
const base = { lastCheckInAt: "2026-11-12T13:05:00.000Z", expectedMinutes: 60, onRequest: false };
|
||||
it("overdue carries its last check-in, its expected cadence and the suspected reason", () => {
|
||||
expect(describeCheckIn({ ...base, status: "check-in-overdue" }))
|
||||
.toBe("overdue: last check-in 2026-11-12T13:05:00.000Z, expected every 60 min; its schedule may be off or waiting for an approval");
|
||||
expect(describeCheckIn({ ...base, status: "checking-in" })).toBe("checking in: last check-in 2026-11-12T13:05:00.000Z, expected every 60 min");
|
||||
});
|
||||
it("names what setup says when there's no check-in, and gives an unknown its reason and last check-in", () => {
|
||||
expect(describeCheckIn({ ...base, status: "no-check-in-yet", lastCheckInAt: null })).toBe("no check-in yet; expected every 60 min");
|
||||
expect(describeCheckIn({ ...base, status: "no-check-in-yet", lastCheckInAt: null, expectedMinutes: null, onRequest: true }))
|
||||
.toBe("no check-in yet; checks in when a person asks it");
|
||||
expect(describeCheckIn({ ...base, status: "no-check-in-yet", lastCheckInAt: null, expectedMinutes: null }))
|
||||
.toBe("no check-in yet; no check-in cadence declared");
|
||||
expect(describeCheckIn({ ...base, status: "on-request", expectedMinutes: null, onRequest: true }))
|
||||
.toBe("on request: it checks its queue when a person asks it; last check-in 2026-11-12T13:05:00.000Z");
|
||||
expect(describeCheckIn({ ...base, status: "unknown", expectedMinutes: null, reason: "no check-in cadence declared" }))
|
||||
.toBe("unknown (no check-in cadence declared); last check-in 2026-11-12T13:05:00.000Z");
|
||||
});
|
||||
});
|
||||
@@ -11,7 +11,7 @@ import { ALL_MIGRATIONS } from "../src/db/all-migrations.js";
|
||||
import { QueueRepository, type QueueItem } from "../src/domain/queue-repository.js";
|
||||
import { EventBus } from "../src/domain/event-bus.js";
|
||||
import { queueRoutes } from "../src/routes/queue.js";
|
||||
import type { RosterWake } from "../src/domain/rosters.js";
|
||||
import type { RosterMember, RosterWake } from "../src/domain/rosters.js";
|
||||
import { createStuckSweepStatus, runStuckSweep, STUCK_SWEEP_FINDING_TAG, unclaimedBacklogTag } from "../src/domain/queue-stuck-sweep.js";
|
||||
|
||||
const MEMBER = "helper-a@hub";
|
||||
@@ -23,12 +23,13 @@ describe("stuck sweep: a hosted member that polls for its work", () => {
|
||||
let repo: QueueRepository;
|
||||
|
||||
function makeRepo(opts: { hostedWake?: (d: string) => RosterWake | null; resolveSessionAddresses?: (n: string) => string[];
|
||||
validateRig?: (d: string) => boolean } = {}): QueueRepository {
|
||||
validateRig?: (d: string) => boolean; hostedMembers?: () => RosterMember[] } = {}): QueueRepository {
|
||||
return new QueueRepository(db, bus, {
|
||||
validateRig: opts.validateRig ?? (() => true),
|
||||
loadHumanRegistry: () => ({ ok: true, entities: [] }),
|
||||
isHostedMember: (d) => d === MEMBER,
|
||||
hostedWake: opts.hostedWake ?? (() => null),
|
||||
hostedMembers: opts.hostedMembers ?? (() => []),
|
||||
...(opts.resolveSessionAddresses ? { resolveSessionAddresses: opts.resolveSessionAddresses } : {}),
|
||||
});
|
||||
}
|
||||
@@ -114,16 +115,88 @@ describe("stuck sweep: a hosted member that polls for its work", () => {
|
||||
expect(more).toEqual([]);
|
||||
expect(notice!.destinationSession).toBe(PERSON);
|
||||
expect(notice!.sourceSession).toBe(MEMBER);
|
||||
expect(notice!.body).toContain(`check-ins: no check-in observed since ${at.toISOString()}`);
|
||||
expect(notice!.body).toContain(`check-ins: unknown (no check-in cadence declared); no check-in since ${at.toISOString()}`);
|
||||
expect(notice!.body).toContain("no read of its own inbox has been observed in that time. Ask its owner to check it's running, or route the work to another seat");
|
||||
for (const row of rows) expect(notice!.body).toContain(`- ${row.qitemId}, unclaimed since ${row.tsCreated}, from lead@hub: work for ${MEMBER}`);
|
||||
expect(notice!.body).not.toMatch(/dead/i);
|
||||
});
|
||||
|
||||
/** The member as its roster lists it, with the cadence its person stated at setup (or none). */
|
||||
const asListed = (checkInMinutes?: number): RosterMember[] => [{ seat: MEMBER, host: "hub", capabilities: ["research"],
|
||||
engagement: ["delegate"], use_when: "research", why: "it researches", hosted: true,
|
||||
...(checkInMinutes === undefined ? {} : { check_in_minutes: checkInMinutes }) }];
|
||||
|
||||
it("a stated cadence decides only the notice's check-in line, never when the notice is filed: overdue past twice it, checking in within it", async () => {
|
||||
const at = new Date(Date.now() - 3 * 3_600_000);
|
||||
repo = makeRepo({ hostedMembers: () => asListed(60) });
|
||||
repo.recordInboxCheckIn(MEMBER, at);
|
||||
await work(MEMBER, { ageMinutes: 61 });
|
||||
await sweep();
|
||||
expect(notices()[0]!.body).toContain(`check-ins: overdue: last check-in ${at.toISOString()}, expected every 60 min; its schedule may be off or waiting for an approval`);
|
||||
expect(notices()[0]!.body).toContain("why: nobody has claimed these rows, and it hasn't checked in as often as its setup expects.");
|
||||
|
||||
// The same three-hour-old check-in is on time for a member expected every four hours, though older than the
|
||||
// unclaimed age that judges a member with no stated cadence.
|
||||
db.close();
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
bus = new EventBus(db);
|
||||
repo = makeRepo({ hostedMembers: () => asListed(240) });
|
||||
seedSeat("lead@hub");
|
||||
repo.recordInboxCheckIn(MEMBER, at);
|
||||
await work(MEMBER, { ageMinutes: 61 });
|
||||
await sweep();
|
||||
expect(notices()[0]!.body).toContain("check-ins: checking in: its last check-in is within twice its expected 240 min");
|
||||
});
|
||||
|
||||
it("the notice says what the hosted-members read says: an unusable cadence reads unknown with its reason, even with a fresh check-in", async () => {
|
||||
const member = { ...asListed()[0]!, check_in_problem: "check_in_minutes \"60\" isn't a positive whole number of minutes" };
|
||||
repo = makeRepo({ hostedMembers: () => [member] });
|
||||
repo.recordInboxCheckIn(MEMBER, new Date(Date.now() - 5 * 60_000));
|
||||
await work(MEMBER, { ageMinutes: 61 });
|
||||
await sweep();
|
||||
expect(repo.hostedMemberCheckIns()[0]).toMatchObject({ status: "unknown", reason: member.check_in_problem });
|
||||
expect(notices()[0]!.body).toContain(`check-ins: unknown (${member.check_in_problem}); it read its own inbox within the last 60 min`);
|
||||
expect(notices()[0]!.body).not.toContain("checking in");
|
||||
});
|
||||
|
||||
it("check-in status is a status, never a refusal: work for an overdue or never-seen member queues and is recorded as before", async () => {
|
||||
repo = makeRepo({ hostedMembers: () => asListed(60) });
|
||||
repo.recordInboxCheckIn(MEMBER, new Date(Date.now() - 5 * 3_600_000));
|
||||
const overdue = await repo.create({ sourceSession: "lead@hub", destinationSession: MEMBER, body: "work", summary: "work" });
|
||||
const unlisted = makeRepo();
|
||||
const baseline = await unlisted.create({ sourceSession: "lead@hub", destinationSession: MEMBER, body: "work", summary: "work" });
|
||||
expect(repo.getById(overdue.qitemId)).toMatchObject({ state: "pending", destinationSession: MEMBER });
|
||||
expect(repo.getById(overdue.qitemId)!.lastNudgeResult).toBe(unlisted.getById(baseline.qitemId)!.lastNudgeResult);
|
||||
expect(repo.hostedMemberCheckIns().map((m) => m.status)).toEqual(["check-in-overdue"]);
|
||||
});
|
||||
|
||||
it("the hosted-members read gives each member's status, last check-in and stated cadence", async () => {
|
||||
const members = [...asListed(60), { ...asListed()[0]!, seat: "helper-b@hub", check_in_on_request: true as const },
|
||||
{ ...asListed(30)[0]!, seat: "helper-c@hub" }, { ...asListed()[0]!, seat: "helper-d@hub" }];
|
||||
repo = makeRepo({ hostedMembers: () => members });
|
||||
const at = new Date(Date.now() - 10 * 60_000);
|
||||
repo.recordInboxCheckIn(MEMBER, at);
|
||||
repo.recordInboxCheckIn("helper-b@hub", at);
|
||||
repo.recordInboxCheckIn("helper-d@hub", at);
|
||||
const res = await appFor(repo).request("/api/queue/hosted-members");
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json() as { hostId: unknown; members: unknown };
|
||||
expect(body).toHaveProperty("hostId");
|
||||
expect(body.members).toEqual([
|
||||
{ seat: MEMBER, rung: false, status: "checking-in", lastCheckInAt: at.toISOString(), expectedMinutes: 60, onRequest: false, reporter: "openrig-observed" },
|
||||
{ seat: "helper-b@hub", rung: false, status: "on-request", lastCheckInAt: at.toISOString(), expectedMinutes: null, onRequest: true, reporter: "openrig-observed" },
|
||||
{ seat: "helper-c@hub", rung: false, status: "no-check-in-yet", lastCheckInAt: null, expectedMinutes: 30, onRequest: false, reporter: "openrig-observed" },
|
||||
// States neither a cadence nor on request: unknown, with its last check-in, never on request, idle or overdue.
|
||||
{ seat: "helper-d@hub", rung: false, status: "unknown", lastCheckInAt: at.toISOString(), expectedMinutes: null, onRequest: false,
|
||||
reporter: "openrig-observed", reason: "no check-in cadence declared" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("a member that never checked in says so", async () => {
|
||||
await work(MEMBER, { ageMinutes: 61 });
|
||||
await sweep();
|
||||
expect(notices()[0]!.body).toContain("check-ins: no check-in observed yet");
|
||||
expect(notices()[0]!.body).toContain("check-ins: no check-in yet; no check-in cadence declared");
|
||||
});
|
||||
|
||||
it("polling without claiming: old work still surfaces, and the member's check-ins neither reset its age nor rewrite the notice", async () => {
|
||||
@@ -132,8 +205,8 @@ describe("stuck sweep: a hosted member that polls for its work", () => {
|
||||
await poll();
|
||||
await sweep();
|
||||
const [notice] = notices();
|
||||
expect(notice!.body).toContain("check-ins: checking in: it read its own inbox within the last 60 min");
|
||||
expect(notice!.body).toContain("it's checking in, but this work is still unclaimed");
|
||||
expect(notice!.body).toContain("check-ins: unknown (no check-in cadence declared); it read its own inbox within the last 60 min");
|
||||
expect(notice!.body).toContain("it's reading its own inbox, but this work is still unclaimed");
|
||||
expect(notice!.body).toContain(`- ${old.qitemId}, unclaimed since ${old.tsCreated}`);
|
||||
expect(notice!.body).not.toContain(young.qitemId);
|
||||
expect(perRow()).toEqual([]);
|
||||
@@ -186,7 +259,7 @@ describe("stuck sweep: a hosted member that polls for its work", () => {
|
||||
expect(errors.mock.calls.filter((call) => String(call[0]).includes("inbox check-in not recorded"))).toHaveLength(1);
|
||||
expect(repo.lastInboxCheckIn(MEMBER)).toEqual({ state: "unknown" });
|
||||
await sweep();
|
||||
expect(notices()[0]!.body).toContain("check-ins: unknown: its check-ins couldn't be read");
|
||||
expect(notices()[0]!.body).toContain("check-ins: unknown (its check-ins couldn't be read)");
|
||||
});
|
||||
|
||||
it("after a restart the stale check-in is still known, and the first new read records at once", async () => {
|
||||
|
||||
@@ -5,7 +5,7 @@ import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import type { Worker } from "node:worker_threads";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { Hono } from "hono";
|
||||
import { createDaemon } from "../src/startup.js";
|
||||
import { createSlowOpRequestMiddleware } from "../src/domain/slow-op-recorder.js";
|
||||
@@ -67,12 +67,37 @@ afterEach(() => {
|
||||
});
|
||||
|
||||
describe("SlowOpRecorder locked instrumentation contract", () => {
|
||||
it("returns promptly while begin acknowledgements are delayed", async () => {
|
||||
const mod = await loadRecorderModule();
|
||||
if (!mod) throw new Error("recorder module missing");
|
||||
const logPath = path.join(tempDir(), "pending.jsonl");
|
||||
const recorder = new mod.SlowOpRecorder({ logPath });
|
||||
const worker: Worker = (recorder as any).worker;
|
||||
const post = worker.postMessage;
|
||||
const held: unknown[] = [];
|
||||
worker.postMessage = message => {
|
||||
if (message?.record?.phase === "begin") held.push(message);
|
||||
else post.call(worker, message);
|
||||
};
|
||||
try {
|
||||
const start = performance.now();
|
||||
const values = [1, 2, 3].map(value => recorder.runSync("test.delayed.begin", () => value));
|
||||
const elapsedMs = performance.now() - start;
|
||||
expect(values).toEqual([1, 2, 3]);
|
||||
expect(held).toHaveLength(3);
|
||||
expect(elapsedMs).toBeLessThan(100);
|
||||
} finally {
|
||||
worker.postMessage = post;
|
||||
for (const message of held) post.call(worker, message);
|
||||
await recorder.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("pins the absolute boundary, rotation, filename, and secret-safe record contract", async () => {
|
||||
const mod = await loadRecorderModule();
|
||||
expect(mod, "slow-op-recorder production module is missing").not.toBeNull();
|
||||
if (!mod) return;
|
||||
|
||||
expect(mod.SLOW_OPERATION_BARRIER_TIMEOUT_MS).toBe(250);
|
||||
expect(mod.SLOW_OPERATION_THRESHOLD_MS).toBe(250);
|
||||
expect(mod.SLOW_OPERATION_ROTATION_BYTES).toBe(1024 * 1024);
|
||||
expect(mod.SLOW_OPERATION_ROTATION_COUNT).toBe(3);
|
||||
@@ -85,9 +110,6 @@ describe("SlowOpRecorder locked instrumentation contract", () => {
|
||||
process.env.OPENRIG_TEST_SECRET = secret;
|
||||
try {
|
||||
const value = recorder.runSync("test.sync.boundary", () => {
|
||||
const records = fs.readFileSync(logPath, "utf8");
|
||||
expect(records).toContain('"phase":"begin"');
|
||||
expect(records).toContain('"site":"test.sync.boundary"');
|
||||
return 41;
|
||||
});
|
||||
expect(value).toBe(41);
|
||||
@@ -107,88 +129,34 @@ describe("SlowOpRecorder locked instrumentation contract", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("leaves a durable open span after the begin barrier and exposes fail-visible degradation", async () => {
|
||||
it("flushes a queued open span and exposes acknowledged write failure in health", async () => {
|
||||
const mod = await loadRecorderModule();
|
||||
expect(mod, "slow-op-recorder production module is missing").not.toBeNull();
|
||||
if (!mod) return;
|
||||
|
||||
if (!mod) throw new Error("recorder module missing");
|
||||
const dir = tempDir();
|
||||
const logPath = path.join(dir, "slow-operations.jsonl");
|
||||
const logPath = path.join(dir, "open.jsonl");
|
||||
const recorder = new mod.SlowOpRecorder({ logPath });
|
||||
const span = recorder.beginSyncSpan("test.sync.wedge");
|
||||
expect(span).toBeTruthy();
|
||||
const records = await readRecords(logPath);
|
||||
expect(records).toHaveLength(1);
|
||||
expect(records[0]).toMatchObject({ phase: "begin", site: "test.sync.wedge" });
|
||||
expect(recorder.snapshot()).toMatchObject({ healthy: true });
|
||||
recorder.beginSyncSpan("test.sync.wedge");
|
||||
await recorder.flush();
|
||||
expect(await readRecords(logPath)).toMatchObject([{ phase: "begin", site: "test.sync.wedge" }]);
|
||||
await recorder.close();
|
||||
|
||||
const degraded = new mod.SlowOpRecorder({
|
||||
logPath: path.join(dir, "degraded.jsonl"),
|
||||
barrierTimeoutMs: 0,
|
||||
});
|
||||
const degraded = new mod.SlowOpRecorder({ logPath: dir });
|
||||
const oldNoKernel = process.env.OPENRIG_NO_KERNEL;
|
||||
process.env.OPENRIG_NO_KERNEL = "1";
|
||||
const daemon = await createTestDaemon({
|
||||
dbPath: ":memory:",
|
||||
tmuxExec: async () => "",
|
||||
cmuxExec: async () => "",
|
||||
slowOpRecorder: degraded,
|
||||
} as never);
|
||||
const daemon = await createTestDaemon({ dbPath: ":memory:", tmuxExec: async () => "", cmuxExec: async () => "", slowOpRecorder: degraded } as never);
|
||||
try {
|
||||
const worker: Worker = (degraded as any).worker;
|
||||
const postMessage = worker.postMessage;
|
||||
let intercepted = 0;
|
||||
let withheldSignal: SharedArrayBuffer | undefined;
|
||||
try {
|
||||
worker.postMessage = (message, transferList) => {
|
||||
if (message?.type === "append"
|
||||
&& message.record?.phase === "begin"
|
||||
&& message.record?.site === "test.sync.degraded"
|
||||
&& message.signal instanceof SharedArrayBuffer) {
|
||||
intercepted++;
|
||||
const { signal, ...withoutSignal } = message;
|
||||
withheldSignal = signal;
|
||||
// Append through the real worker, but withhold this acknowledgement.
|
||||
return postMessage.call(worker, withoutSignal, transferList);
|
||||
}
|
||||
return postMessage.call(worker, message, transferList);
|
||||
};
|
||||
expect(degraded.runSync("test.sync.degraded", () => "continued")).toBe("continued");
|
||||
} finally {
|
||||
worker.postMessage = postMessage;
|
||||
}
|
||||
expect(intercepted).toBe(1);
|
||||
expect(worker.postMessage).toBe(postMessage);
|
||||
expect(withheldSignal).toBeInstanceOf(SharedArrayBuffer);
|
||||
expect(new Int32Array(withheldSignal!)[0]).toBe(0);
|
||||
expect(degraded.snapshot()).toMatchObject({
|
||||
healthy: false,
|
||||
reason: "begin_barrier_timeout",
|
||||
site: "test.sync.degraded",
|
||||
});
|
||||
|
||||
const healthResponse = await daemon.app.request("/healthz");
|
||||
expect(await healthResponse.json()).toMatchObject({
|
||||
status: "ok",
|
||||
slowOperations: {
|
||||
healthy: false,
|
||||
reason: "begin_barrier_timeout",
|
||||
site: "test.sync.degraded",
|
||||
},
|
||||
});
|
||||
expect(degraded.runSync("test.sync.degraded", () => "continued")).toBe("continued");
|
||||
await expect(degraded.flush()).rejects.toBeInstanceOf(mod.SlowOpRecorderWriteError);
|
||||
expect(degraded.snapshot()).toMatchObject({ healthy: false, reason: "recorder_write_failed", site: "recorder.worker" });
|
||||
const response = await daemon.app.request("/healthz");
|
||||
expect(await response.json()).toMatchObject({ status: "ok", slowOperations: { healthy: false, reason: "recorder_write_failed" } });
|
||||
const observations = daemon.deps.streamStore!.list();
|
||||
expect(observations).toHaveLength(1);
|
||||
expect(observations[0]!.body).toContain("begin_barrier_timeout");
|
||||
expect(observations[0]!.body).toContain("test.sync.degraded");
|
||||
await degraded.flush();
|
||||
const degradedRecords = await readRecords(path.join(dir, "degraded.jsonl"));
|
||||
expect(degradedRecords.filter((record) => record.phase === "begin"
|
||||
&& record.site === "test.sync.degraded")).toHaveLength(1);
|
||||
expect(observations[0]!.body).toContain("recorder_write_failed");
|
||||
} finally {
|
||||
daemon.eventLoopMonitor.stop();
|
||||
daemon.db.close();
|
||||
await degraded.close();
|
||||
await degraded.close().catch(() => {});
|
||||
if (oldNoKernel === undefined) delete process.env.OPENRIG_NO_KERNEL;
|
||||
else process.env.OPENRIG_NO_KERNEL = oldNoKernel;
|
||||
}
|
||||
@@ -211,20 +179,17 @@ describe("SlowOpRecorder locked instrumentation contract", () => {
|
||||
});
|
||||
expect(value).toBe("continued");
|
||||
expect(ran).toBe(true);
|
||||
expect(recorder.snapshot()).toEqual({
|
||||
healthy: false,
|
||||
reason: "begin_barrier_failed",
|
||||
site: "test.sync.observer_failure",
|
||||
});
|
||||
await expect(recorder.flush()).rejects.toBeInstanceOf(mod.SlowOpRecorderWriteError);
|
||||
expect(recorder.snapshot()).toEqual({ healthy: false, reason: "recorder_write_failed", site: "recorder.worker" });
|
||||
} finally {
|
||||
// This recorder's log path is intentionally unwritable (to force the
|
||||
// begin-barrier failure above), so close() now correctly rejects on the
|
||||
// asynchronously acknowledged write failure above), so close() now correctly rejects on the
|
||||
// latched lost-write durability — tolerate it in teardown.
|
||||
await recorder.close().catch(() => {});
|
||||
}
|
||||
});
|
||||
|
||||
it("durably records an open span before a main-thread sync wedge is killed", async () => {
|
||||
it("writes a queued begin while the main thread is wedged, before a later kill", async () => {
|
||||
const dir = tempDir();
|
||||
const logPath = path.join(dir, "crash.jsonl");
|
||||
fs.writeFileSync(logPath, "", { mode: 0o600 });
|
||||
@@ -278,6 +243,7 @@ describe("SlowOpRecorder locked instrumentation contract", () => {
|
||||
});
|
||||
});
|
||||
expect(callbackEvidence).toEqual({ isMainThread: true, threadId: 0 });
|
||||
await vi.waitFor(async () => expect(await readRecords(logPath)).toMatchObject([{ phase: "begin", site: "test.sync.crash" }]));
|
||||
child.kill("SIGKILL");
|
||||
await once(child, "exit");
|
||||
|
||||
@@ -351,7 +317,7 @@ describe("SlowOpRecorder locked instrumentation contract", () => {
|
||||
expect(JSON.stringify(requests)).not.toContain("must-not-appear");
|
||||
});
|
||||
|
||||
it("wraps all nine accepted sync calls without moving them off the main thread", () => {
|
||||
it("keeps the accepted slow-operation site labels", () => {
|
||||
const srcRoot = path.resolve(import.meta.dirname, "../src");
|
||||
for (const [relativePath, sites] of expectedSites) {
|
||||
const source = fs.readFileSync(path.join(srcRoot, relativePath), "utf8");
|
||||
@@ -496,7 +462,7 @@ describe("SlowOpRecorder locked instrumentation contract", () => {
|
||||
expect((await app.request("/definitely-missing")).status).toBe(404);
|
||||
});
|
||||
|
||||
it("drives the real tmux command-v spawnSync site through a hermetic daemon", async () => {
|
||||
it("drives the asynchronous tmux command-v site through a hermetic daemon", async () => {
|
||||
const mod = await loadRecorderModule();
|
||||
expect(mod, "slow-op-recorder production module is missing").not.toBeNull();
|
||||
if (!mod) return;
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { TmuxAdapter } from "../src/adapters/tmux.js";
|
||||
import { TmuxOptionDefaultsApplier } from "../src/domain/tmux-option-defaults.js";
|
||||
|
||||
const execute = vi.hoisted(() => vi.fn());
|
||||
vi.mock("node:child_process", () => ({ execFile: execute }));
|
||||
beforeEach(() => { execute.mockReset(); });
|
||||
|
||||
describe("clipboard discovery deadline", () => {
|
||||
it.each([null, "custom-copy"])("warns and retries a killed default probe, preserving %j", async initial => {
|
||||
let copyCommand = initial;
|
||||
const setServerOption = vi.fn(async (name: string, value: string) => {
|
||||
if (name === "copy-command") copyCommand = value;
|
||||
return { ok: true };
|
||||
});
|
||||
const adapter = { setSessionOption: async () => ({ ok: true }), setServerOption } as unknown as TmuxAdapter;
|
||||
execute.mockImplementationOnce((_file, _args, options, callback) => {
|
||||
expect(options).toMatchObject({ timeout: 250, killSignal: "SIGKILL" });
|
||||
callback(Object.assign(new Error("probe deadline"), { killed: true }));
|
||||
}).mockImplementationOnce((_file, _args, _options, callback) => callback(null));
|
||||
const applier = new TmuxOptionDefaultsApplier({ tmuxAdapter: adapter, platform: "linux" });
|
||||
const warnings = await applier.applyToFreshSession("first@test");
|
||||
expect(warnings).toHaveLength(1);
|
||||
expect(warnings[0]).toMatch(/clipboard.*250 ms/i);
|
||||
expect(warnings[0]).toMatch(/retry/i);
|
||||
expect(copyCommand).toBe(initial);
|
||||
expect(execute).toHaveBeenCalledTimes(1);
|
||||
expect(await applier.applyToFreshSession("next@test")).toEqual([]);
|
||||
expect(copyCommand).toBe("wl-copy");
|
||||
expect(execute).toHaveBeenCalledTimes(2);
|
||||
expect(await applier.applyToFreshSession("later@test")).toEqual([]);
|
||||
expect(execute).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("memoizes confirmed absence without a deadline warning", async () => {
|
||||
execute.mockImplementation((_file, _args, _options, callback) => callback(Object.assign(new Error("absent"), { code: 1 })));
|
||||
const setServerOption = vi.fn(async () => ({ ok: true }));
|
||||
const adapter = { setSessionOption: async () => ({ ok: true }), setServerOption } as unknown as TmuxAdapter;
|
||||
const applier = new TmuxOptionDefaultsApplier({ tmuxAdapter: adapter, platform: "linux" });
|
||||
expect(await applier.applyToFreshSession("first@test")).toEqual([]);
|
||||
expect(await applier.applyToFreshSession("next@test")).toEqual([]);
|
||||
expect(execute).toHaveBeenCalledTimes(2);
|
||||
expect(setServerOption).not.toHaveBeenCalledWith("copy-command", expect.anything());
|
||||
});
|
||||
});
|
||||
@@ -23,6 +23,24 @@ function mockAdapter(overrides?: {
|
||||
}
|
||||
|
||||
describe("TmuxOptionDefaultsApplier", () => {
|
||||
it("awaits an asynchronous clipboard lookup without holding unrelated work", async () => {
|
||||
const { adapter, setServerOption } = mockAdapter();
|
||||
let release!: (present: boolean) => void;
|
||||
const pending = new Promise<boolean>(resolve => { release = resolve; });
|
||||
const probe = vi.fn((bin: string) => bin === "wl-copy" ? pending : Promise.resolve(true));
|
||||
let finished = false;
|
||||
const applier = new TmuxOptionDefaultsApplier({ tmuxAdapter: adapter, platform: "linux", hasCommand: probe });
|
||||
const applied = applier.applyToFreshSession("async-clipboard@test").then(value => { finished = true; return value; });
|
||||
try {
|
||||
await new Promise(resolve => setImmediate(resolve));
|
||||
expect(finished).toBe(false);
|
||||
expect(setServerOption).not.toHaveBeenCalledWith("copy-command", expect.anything());
|
||||
} finally { release(false); }
|
||||
expect(await applied).toEqual([]);
|
||||
expect(probe.mock.calls.map(([bin]) => bin)).toEqual(["wl-copy", "xclip"]);
|
||||
expect(setServerOption).toHaveBeenCalledWith("copy-command", "xclip -selection clipboard -i");
|
||||
});
|
||||
|
||||
it("sets mouse on + status off (default) on the given session via SESSION scope only", async () => {
|
||||
const { adapter, setSessionOption, setServerOption } = mockAdapter();
|
||||
// default reader (omitted) → statusBar false.
|
||||
@@ -109,20 +127,20 @@ describe("TmuxOptionDefaultsApplier", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveCopyCommand (pure per-platform table)", () => {
|
||||
it("darwin → pbcopy", () => {
|
||||
expect(resolveCopyCommand("darwin", () => false)).toBe("pbcopy");
|
||||
describe("resolveCopyCommand (pure per-platform table)", async () => {
|
||||
it("darwin → pbcopy", async () => {
|
||||
expect(await resolveCopyCommand("darwin", () => false)).toBe("pbcopy");
|
||||
});
|
||||
it("linux → wl-copy when present", () => {
|
||||
expect(resolveCopyCommand("linux", (b) => b === "wl-copy")).toBe("wl-copy");
|
||||
it("linux → wl-copy when present", async () => {
|
||||
expect(await resolveCopyCommand("linux", (b) => b === "wl-copy")).toBe("wl-copy");
|
||||
});
|
||||
it("linux → xclip when wl-copy absent but xclip present", () => {
|
||||
expect(resolveCopyCommand("linux", (b) => b === "xclip")).toBe("xclip -selection clipboard -i");
|
||||
it("linux → xclip when wl-copy absent but xclip present", async () => {
|
||||
expect(await resolveCopyCommand("linux", (b) => b === "xclip")).toBe("xclip -selection clipboard -i");
|
||||
});
|
||||
it("linux → null when neither present (OSC 52 fallback)", () => {
|
||||
expect(resolveCopyCommand("linux", () => false)).toBeNull();
|
||||
it("linux → null when neither present (OSC 52 fallback)", async () => {
|
||||
expect(await resolveCopyCommand("linux", () => false)).toBeNull();
|
||||
});
|
||||
it("other platforms → null", () => {
|
||||
expect(resolveCopyCommand("win32", () => true)).toBeNull();
|
||||
it("other platforms → null", async () => {
|
||||
expect(await resolveCopyCommand("win32", () => true)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { AttentionRead, AttentionItem } from "@openrig/daemon/attention";
|
||||
import type { Action, FleetSnapshot, ViewState } from "../types.js";
|
||||
import { listItem, wrapDetailLines, type ContentLine } from "../detail.js";
|
||||
import { displayTime } from "../time.js";
|
||||
|
||||
/** The bounded, confirmed-delivery or bridge-acceptance projection served by S02; no client classification. */
|
||||
export interface DeliveredHumanUpdates {
|
||||
@@ -11,6 +12,8 @@ export interface DeliveredHumanUpdates {
|
||||
truncated: boolean;
|
||||
}
|
||||
|
||||
const BRIDGE_ACCEPTED_SCOPE = " · accepted by bridge";
|
||||
|
||||
function isBridgeAcceptedUpdate(q: DeliveredHumanUpdates["items"][number]): boolean {
|
||||
return q.deliveryOutcome === "accepted" || q.deliveryReceipt.startsWith("relay-owner-notification-accepted");
|
||||
}
|
||||
@@ -38,13 +41,13 @@ export function composeHumanUpdates(attention: AttentionRead | null, updates: De
|
||||
const baseScope = projects.length === 1 ? `project ${projects[0]} (queue tag)` : "instance · project unknown";
|
||||
const item: AttentionItem = { id: `human-update:${q.qitemId}`, kind: "update", summary: q.summary || q.body.trim().split(/\r?\n/).find(Boolean) || (accepted ? "Accepted by bridge" : "Delivered update"),
|
||||
recipient: q.destinationSession, urgency: "update", unblocks: null, at: q.deliveredAt,
|
||||
scope: accepted ? `${baseScope} · accepted by bridge` : baseScope, project: null,
|
||||
scope: accepted ? `${baseScope}${BRIDGE_ACCEPTED_SCOPE}` : baseScope, project: null,
|
||||
source: `/api/queue/${encodeURIComponent(q.qitemId)}` };
|
||||
if (!read.items.some(current => current.id === `queue-update:${q.qitemId}`)) read.items.push(item);
|
||||
if (wanted === item.id && read.detail?.item.id !== item.id) {
|
||||
read.detailError = null;
|
||||
read.detail = { item, lines: [q.body, ...(q.humanDetail ? ["Supplemental detail:", q.humanDetail] : []),
|
||||
`From: ${q.sourceSession}`, accepted ? `Accepted by bridge: ${q.deliveredAt}` : `Delivered: ${q.deliveredAt}`, `Delivery receipt: ${q.deliveryReceipt}`, `Queue: ${q.qitemId}`, `Evidence: ${q.evidenceRef ?? "none recorded"}`],
|
||||
`From: ${q.sourceSession}`, `Delivery receipt: ${q.deliveryReceipt}`, `Queue: ${q.qitemId}`, `Evidence: ${q.evidenceRef ?? "none recorded"}`],
|
||||
files: q.evidenceRef?.startsWith("/") ? [{ label: "Update evidence", path: q.evidenceRef }] : [] };
|
||||
}
|
||||
}
|
||||
@@ -71,7 +74,10 @@ export function attentionLines(state: ViewState, snap: FleetSnapshot, width: num
|
||||
if (d.item.recipient) lines.push({ text: `To: ${d.item.recipient}` });
|
||||
if (d.item.id.startsWith("human-update:") || d.item.id.startsWith("queue-update:")) lines.push({ text: "No action needed" });
|
||||
if (d.item.unblocks) lines.push({ text: `Unblocks: ${d.item.unblocks}` });
|
||||
lines.push({ text: `Scope: ${d.item.scope}` }, { text: `Observed: ${d.item.at ?? "unknown"}` });
|
||||
// Times are drawn in the person's zone (ui.timezone), as the header's are; a delivered update's time is when it was
|
||||
// delivered or accepted by the bridge.
|
||||
const when = !d.item.id.startsWith("human-update:") ? "Observed" : d.item.scope.endsWith(BRIDGE_ACCEPTED_SCOPE) ? "Accepted by bridge" : "Delivered";
|
||||
lines.push({ text: `Scope: ${d.item.scope}` }, { text: `${when}: ${displayTime(d.item.at, state.timeZone)}` });
|
||||
lines.push(...questionLines(d.item).map(text => ({ text })));
|
||||
lines.push(...d.lines.map(text => ({ text })));
|
||||
for (const f of d.files) lines.push(listItem(f.label, { type: "attention-source", path: f.path }));
|
||||
@@ -124,10 +130,10 @@ export function attentionLines(state: ViewState, snap: FleetSnapshot, width: num
|
||||
if (i.humanQuestions?.length) lines.push({ text: ` Questions: ${i.humanQuestions.length}, ${unansweredCount(i)} unanswered` });
|
||||
if (i.id.startsWith("human-update:") || i.id.startsWith("queue-update:")) lines.push({ text: " No action needed" });
|
||||
if (i.unblocks) lines.push({ text: ` Unblocks: ${i.unblocks}` });
|
||||
lines.push({ text: ` ${i.scope} · ${i.at ?? "time unknown"}` });
|
||||
lines.push({ text: ` ${i.scope} · ${displayTime(i.at, state.timeZone)}` });
|
||||
}
|
||||
}
|
||||
lines.push({ text: "" }, { text: `Read at ${read.readAt}` }, ...read.sources.map(s => ({ text: `${s.source}: ${s.state} · ${s.detail}` })));
|
||||
lines.push({ text: "" }, { text: `Read at ${displayTime(read.readAt, state.timeZone)}` }, ...read.sources.map(s => ({ text: `${s.source}: ${s.state} · ${s.detail}` })));
|
||||
}
|
||||
lines.push({ text: "" }, { text: "Viewing is not approval. Each request's detail says whether it went to Slack and where to answer it." });
|
||||
return wrapDetailLines(lines, width);
|
||||
|
||||
@@ -50,6 +50,24 @@ it("renders an unsent local FYI without inventing an approval or a delivery rece
|
||||
expect(text).not.toContain("Delivery receipt:");
|
||||
});
|
||||
|
||||
it("draws Feed times in the person's zone, as the header does, and redraws them when the zone changes", () => {
|
||||
const item = current(), attention = read([item]);
|
||||
attention.detail = { item, lines: ["Current queue body"], files: [] };
|
||||
const merged = composeHumanUpdates(attention, delivered("other"), item.id)!;
|
||||
const draw = (timeZone: string, attentionOpen: string | null) => attentionLines(
|
||||
{ ...createViewState({ instanceId: "owned-fixture", timeZone }).get(), section: "needs" as const, attentionOpen },
|
||||
{ ...emptySnapshot(), attentionRead: merged }, 140).map(l => l.text).join("\n");
|
||||
const london = draw("Europe/London", null);
|
||||
expect(london).toContain("instance · 2026-10-01 13:00:00 GMT+1");
|
||||
expect(london).toContain("instance · project unknown · 2026-10-08 13:00:00 GMT+1");
|
||||
expect(london).toContain("Read at 2026-10-09 13:00:00 GMT+1");
|
||||
expect(london).not.toMatch(/\d{4}-\d{2}-\d{2}T\d{2}:/);
|
||||
// The same read, drawn after the zone setting arrives: nothing is re-read, and nothing keeps the earlier zone.
|
||||
const angeles = draw("America/Los_Angeles", item.id);
|
||||
expect(angeles).toContain("Observed: 2026-10-01 05:00:00 PDT");
|
||||
expect(angeles).not.toMatch(/\d{4}-\d{2}-\d{2}T\d{2}:|GMT\+1/);
|
||||
});
|
||||
|
||||
it("does not retain delivered history beyond a successful empty receipt window when only current updates fail", () => {
|
||||
const prior = composeHumanUpdates(read([]), delivered())!;
|
||||
const failed = read([]);
|
||||
@@ -155,8 +173,9 @@ it("labels relay bridge acceptance as accepted by the bridge rather than Deliver
|
||||
],
|
||||
};
|
||||
const openedBridge = composeHumanUpdates(read([]), bridgeUpdates, "human-update:bridge-fyi")!;
|
||||
const bridgeDetailLines = openedBridge.detail?.lines.join("\n") ?? "";
|
||||
expect(bridgeDetailLines).toContain("Accepted by bridge: 2026-10-08T14:00:00Z");
|
||||
const detailState = (id: string) => ({ ...createViewState({ instanceId: "owned-fixture", timeZone: "UTC" }).get(), section: "needs" as const, attentionOpen: id });
|
||||
const bridgeDetailLines = attentionLines(detailState("human-update:bridge-fyi"), { ...emptySnapshot(), attentionRead: openedBridge }, 140).map(l => l.text).join("\n");
|
||||
expect(bridgeDetailLines).toContain("Accepted by bridge: 2026-10-08 14:00:00 UTC");
|
||||
expect(bridgeDetailLines).not.toContain("Delivered:");
|
||||
expect(openedBridge.sources.find(s => s.source === "delivered updates")?.detail).toContain("accepted by the bridge");
|
||||
expect(openedBridge.sources.find(s => s.source === "delivered updates")?.detail).not.toContain("confirmed delivered updates");
|
||||
@@ -168,8 +187,8 @@ it("labels relay bridge acceptance as accepted by the bridge rather than Deliver
|
||||
|
||||
// Slack delivered wording remains unchanged
|
||||
const openedSlack = composeHumanUpdates(read([]), delivered("slack-fyi"), "human-update:slack-fyi")!;
|
||||
const slackDetailLines = openedSlack.detail?.lines.join("\n") ?? "";
|
||||
expect(slackDetailLines).toContain("Delivered: 2026-10-08T12:00:00Z");
|
||||
const slackDetailLines = attentionLines(detailState("human-update:slack-fyi"), { ...emptySnapshot(), attentionRead: openedSlack }, 140).map(l => l.text).join("\n");
|
||||
expect(slackDetailLines).toContain("Delivered: 2026-10-08 12:00:00 UTC");
|
||||
expect(slackDetailLines).not.toContain("Accepted by bridge:");
|
||||
expect(openedSlack.sources.find(s => s.source === "delivered updates")?.detail).toBe(
|
||||
"Latest 20 confirmed delivered updates; retained receipt window.",
|
||||
|
||||
Reference in new issue
Block a user