mirror of
https://github.com/mvschwarz/openrig.git
synced 2026-11-09 03:39:51 -10:00
Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f6180805c3 | ||
|
|
0d9e843856 | ||
|
|
5b43c103e6 | ||
|
|
16e2f1e88c | ||
|
|
f46cba5295 | ||
|
|
e834607350 | ||
|
|
0cb06f8b1c |
No files matched your search
@@ -511,6 +511,7 @@ default-permit record (tag `runner-permission-permit`) opens no row.
|
||||
| `rig queue inbox-pending <destinationSession>` | — | `--json` |
|
||||
| `rig queue outbox-record` | — | `--sender <session>`<br>`--destination <session>` **required**<br>`--body <text>`<br>`--body-file <path>`<br>`--tags <tags>`<br>`--urgency <urgency>`<br>`--audit <pointer>`<br>`--id <outboxId>`<br>`--json` |
|
||||
| `rig queue outbox-list <senderSession>` | — | `--limit <n>`<br>`--json` |
|
||||
| `rig queue erase` | — | `--members <file>`<br>`--rig <id>`<br>`--ids <file>`<br>`--action <action>` (`delete`, `scrub`)<br>`--routine`<br>`--apply`<br>`--actor <session>`<br>`--json` |
|
||||
|
||||
### slack
|
||||
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
# Queue erasure: closed-work retention and account deletion
|
||||
|
||||
A daemon that serves several accounts sometimes needs to remove queue content for good: closed work after its retention window, or everything that belongs to an account when the account is deleted. `rig queue erase` and three retention settings do that.
|
||||
|
||||
Erasure can't be undone. Run every scope as a dry run first, without `--apply`. The dry run prints exactly what `--apply` would erase and changes nothing.
|
||||
|
||||
## What is erased
|
||||
|
||||
A queue row is erased in one of two ways:
|
||||
|
||||
- **Scrub.** The row's content is removed, and the row itself stays, marked `contentErasedAt`. A retried create with the same id still finds the row, so it makes no new work. If anything writes to the row after its scrub (a note, a summary, a reopen and close), the next run erases that content too.
|
||||
- **Delete.** The row and everything linked to it are removed. Only its id is kept, in an erased-id record with no endpoints and no content. A later create that reuses the id is refused with `qitem_erased` (409), so a retried or forwarded create can't bring the work back.
|
||||
|
||||
| A scrub clears | A scrub keeps |
|
||||
|---|---|
|
||||
| body, summary, resolution, tags, blocker text, chain of record, evidence reference, human detail, questions and answers, delegated decision, the input-box reading | id, source and destination, state, priority, tier, times, closure reason, handoff links |
|
||||
| a closure's target when it's a reason or note, in the row, both transition tables and events. That's every target under a reason other than the two at right, and any target under those two that isn't a session address or a queue id | a closure's target that names a successor: under `handed_off_to` or `superseded`, shaped as a session address (`member@rig`, optionally `@host`) or a queue id (`qitem-…`) |
|
||||
| transition notes and recorded principals, live and archived | each transition's state, actor and time |
|
||||
| events' `summary`, `reason` and `error` (the event keeps its type and ids, marked `contentErased`) | |
|
||||
| mission-control audit text and snapshots | |
|
||||
|
||||
**The row's own copies go with it:**
|
||||
- its transitions;
|
||||
- events whose payload names it;
|
||||
- its mission-control audit rows.
|
||||
|
||||
**Copies held with someone else go only when they're the owner's.** These are outbox and inbox entries that point at the row or wake for it, and stream reader actions and dispositions that name it (including `duplicate_of`). For account deletion, the owner is the account: both of a copy's sessions must be members. Otherwise the owner is the row's own source and destination. Any other copy is kept and listed under `keptCopies`:
|
||||
- `mixed`: one of its sessions is the owner's;
|
||||
- `foreign`: none of them are.
|
||||
|
||||
**A row a workflow still references is never erased.** That covers a step trail, a frontier binding, a failure record, or a live instance's frontier. It's reported under `gaps` with the code `workflow-reference`.
|
||||
|
||||
## Scopes
|
||||
|
||||
Name exactly one.
|
||||
|
||||
| Scope | Erases | Action |
|
||||
|---|---|---|
|
||||
| `--members <file>` | Account deletion. Every row whose source and destination are both in the file, in any state, open work included. Add `--rig <id>` for the account rig's own records | delete |
|
||||
| `--ids <file>` | Exactly the listed rows. A report's `qitems` entries, or a ledger's lines, keep each entry's own action | `--action delete` (default for plain ids) or `--action scrub` |
|
||||
| `--routine` | Closed rows past this daemon's retention windows (below) | scrub, then delete |
|
||||
|
||||
A file is one entry per line (blank lines and `#` comments are skipped; a line starting with `{` is a JSON entry) or a JSON array.
|
||||
|
||||
**Account deletion is exact.**
|
||||
- The members file is the account's recorded membership: the exact session names of its seats, including departed ones. List every address form the account used, such as a host-suffixed form. Names are compared as given, never by pattern, so a row addressed through an unlisted form is neither erased nor reported.
|
||||
- **Mixed rows:** a row with exactly one member endpoint is reported under `mixed` (with which endpoint is the member) and left untouched. Deciding who owns it is a separate decision. A row rerouted out of the account while the run is under way (by a fallback, say) is kept too, and reported as `changed-during-run`.
|
||||
- **After the rows:** account deletion also removes outbox and inbox entries between two members that no row owns, and the members' stream items. An entry with one member endpoint, or a stream item something still references, is counted and left.
|
||||
|
||||
**The account's rig.** A rig's own records outlive the rig. Events are kept past rig and node deletion by design, and snapshots, node and seat records and watchdog jobs have no key that cascades. `--rig <id>`, given with `--members`, erases them:
|
||||
- the rig's events, including those of its nodes;
|
||||
- snapshots, operation correlations and bootstrap runs;
|
||||
- seat identity verdicts, delivery-guard changes, environment bindings and usage records for its nodes and member sessions;
|
||||
- watchdog jobs aimed at a member, with their history;
|
||||
- stream subscriptions and thread maps of member sessions.
|
||||
|
||||
Records that name the account but also belong elsewhere are counted under `rig.left` and kept. These are a member's watchdog job aimed outside the account, a workflow instance or spec bound to the rig, classifier leases and custom views.
|
||||
- **While the rig exists,** its sessions and the members must match both ways, or the run is refused (`erase_rig_mismatch`). Every member's session name must carry the rig's name (departed members included), and every current session of the rig must be a member, so the rig holds no one else's seat.
|
||||
- **On that verified run,** the rig's id and name are recorded with a digest of the membership.
|
||||
- **Once the rig is removed,** the same command, with the same membership, uses that record to erase what's left under the rig's id. This includes the events the removal wrote, and the record still lets it count records bound to the rig's name.
|
||||
- **When nothing of the rig is left,** the record becomes a completion receipt: the rig's name is cleared, and its id, the membership digest and the time stay. An identical rerun, after a lost response say, still verifies and reports the same clean result, with `rig.completedAt`. The receipt is kept for good. It holds no content.
|
||||
- **A removed rig's id that was never verified** with this membership erases nothing keyed by it. Those records are counted under `rig.left`, the name-bound tables are listed under `rig.unknown`, and the run isn't clean.
|
||||
|
||||
**Retention windows** (`rig config set …`, read at the daily sweep and at daemon start):
|
||||
|
||||
| Setting | Default | Effect |
|
||||
|---|---|---|
|
||||
| `retention.closed_content_days` | `0` (off) | Scrub closed rows this many days after they closed |
|
||||
| `retention.closed_metadata_days` | `0` (off) | Delete closed rows, keeping only their ids, this many days after they closed |
|
||||
| `retention.erasure_ledger` | empty (none) | An absolute file path, on local disk, since each batch appends to it while it holds the daemon. Every applied erasure, by the sweep or the command, appends each row's id and action there, one JSON line each, before anything in its batch changes. A path that can't be written stops the run before it changes anything |
|
||||
|
||||
- **Closed** means `done`, `canceled`, `denied` or `handed-off`. `failed` and open work are never touched by the windows.
|
||||
- **Age:** a row's age counts from when it entered its final closed state. A note on a closed row doesn't restart it, and a row reopened and closed again counts from its latest close.
|
||||
- **Running them now:** `rig queue erase --routine --apply` runs the windows immediately. `retention.enabled false` turns off the whole sweep, these windows included.
|
||||
- **When the sweep's erasure stops on an error,** that's logged, and the sweep's other passes still run.
|
||||
|
||||
## Account deletion, in order
|
||||
|
||||
1. Revoke the account's access.
|
||||
2. Keep its exact membership list and rig id.
|
||||
3. Run `rig queue erase --members <file> --rig <id>` as a dry run, then with `--apply`. Rerun until `complete` is true.
|
||||
4. Remove the rig.
|
||||
5. Rerun the same command with `--apply`, to erase the records the removal wrote.
|
||||
6. The account is deleted only when the report's `clean` is true. Otherwise its `mixed`, `gaps`, `keptCopies`, `unlinked` and `rig.left` or `rig.unknown` entries need a decision. Rerunning won't resolve them.
|
||||
|
||||
## Output
|
||||
|
||||
The command prints one JSON report (`--json` makes it a single line):
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `applied` | `false` for a dry run |
|
||||
| `complete` | the run finished. `false` when it stopped at its batch limit or on an error; rerun the same command to continue |
|
||||
| `clean` | `complete`, and nothing was left: no `mixed` rows, mixed `keptCopies`, `gaps`, `unreadableEvents`, left-over entries, or rig records left or unknown. Only a clean report means the scope is fully erased |
|
||||
| `error` | why the run stopped (`stage` and `message`). Everything listed in `qitems` was committed before it; the command exits 2 |
|
||||
| `qitems` | each row erased (or that would be), with its `action` and `state` |
|
||||
| `mixed` | members scope: rows with one member endpoint, left untouched |
|
||||
| `keptCopies` | copies of erased rows held with someone else, kept: `table`, `id`, `qitemId`, `ownership` (`mixed` or `foreign`) |
|
||||
| `gaps` | rows not erased and why: `workflow-reference`; `not-found` for an id to scrub that doesn't exist; `changed-during-run` for a row that no longer qualified when its batch ran (reopened, rerouted out of the account, or written to so it left its window) or an absent id created since selection; `unreadable-event` for a row an unreadable event names |
|
||||
| `alreadyAbsent` | ids to delete with no row; their erased-id record is written (again), and they're listed once it's committed. They're rechecked as the record is written, so an id created during the run is reported as `changed-during-run` instead |
|
||||
| `unreadableEvents` | events naming a selected row whose payload can't be read: `seq` and the row ids found in its text. They're kept, since what they hold can't be inspected, and so are the rows they name, so every rerun reports them again until the event is repaired or removed. The run isn't clean |
|
||||
| `linked` | counts of the copies erased with the rows |
|
||||
| `unlinked` | members scope: entries and stream items erased that no row owned, plus those left (`mixedEntries`, `referencedStreamItems`) |
|
||||
| `rig` | members scope with `--rig`: `found`, `verified`, records erased per table (`erased`, counted as each chunk commits), records left (`left`), tables that can't be counted (`unknown`), and `completedAt` once the rig's erasure is complete |
|
||||
|
||||
**How a run works:**
|
||||
- **Selection:** a run selects everything first and writes nothing.
|
||||
- **Events:** it reads the events table once, in small windows, to find each row's events.
|
||||
- **Batches:** each batch, of up to 100 rows, is one transaction. It rechecks its rows, writes their ledger lines, then changes their events, copies and rows.
|
||||
- **The recheck drops** a row that a workflow has taken up, or that an unreadable event names. For account deletion, it also drops a row that is no longer between two members. For a routine run, it drops one that was reopened or no longer falls in the same window.
|
||||
- **Each dropped row** is reported in `gaps`.
|
||||
- **The current endpoints** decide whose its copies are.
|
||||
- **How long it holds the daemon:** the batch caps rows, not events or copies, so a row with many of either makes its batch longer. On a copy of a store with 2.4 million events, the longest single hold in a first routine run was about 0.25 s.
|
||||
- **Stopping:** a run that stops leaves whole batches done, events included. A rerun picks up the rest, and erasing a row a second time changes nothing.
|
||||
- **Ledger lines are recorded intentions:** if a batch's transaction fails after its lines are written, the ledger lists rows that weren't erased. A replay erases those ids as listed, whatever happened to the rows since.
|
||||
|
||||
**Refusals**, with nothing erased:
|
||||
|
||||
| Code | Cause |
|
||||
|---|---|
|
||||
| `erase_scope_required` | no scope named |
|
||||
| `erase_scope_conflict` | more than one scope |
|
||||
| `erase_members_invalid` | an empty list, or names with surrounding spaces |
|
||||
| `erase_rig_invalid` | a rig without `--members`, or an empty rig id |
|
||||
| `erase_rig_mismatch` | the rig exists, and its sessions and the members don't match both ways |
|
||||
| `erase_ids_invalid` | an empty ids list, or ids with surrounding spaces |
|
||||
| `erase_action_invalid` | an action other than `delete` or `scrub` |
|
||||
| `erase_routine_off` | both windows are `0` |
|
||||
|
||||
The command records who ran it from `OPENRIG_SESSION_NAME`, or from `--actor <session>` when that isn't set.
|
||||
|
||||
## Restores and backups
|
||||
|
||||
A restored database can bring erased content back: rows, copies, rig records and the erased-id records themselves. It can even bring back a row that was still open when the backup was taken and was closed and erased later. Before a restored daemon serves anyone, replay what was erased after the backup was taken:
|
||||
|
||||
1. **Keep a ledger.** Set `retention.erasure_ledger` to a file outside the backed-up data. Every applied erasure appends its ids and actions there before committing. That covers the daily windows, `--routine`, `--ids` and account deletion's rows.
|
||||
2. **Replay the ledger:** `rig queue erase --ids <ledger> --apply`. Each line keeps its own action, so a scrub is replayed as a scrub, never promoted to a deletion. A row that was open in the backup is erased by its id, and other open work isn't touched.
|
||||
3. **Replay account deletions:** rerun `rig queue erase --members <file> --rig <id> --apply` for each account deleted since the backup, from its membership list and rig id. Keep those outside the backed-up data until every backup taken before the deletion has expired. This also covers the copies and rig records the ledger doesn't list.
|
||||
4. **Rerun the windows:** `rig queue erase --routine --apply`.
|
||||
|
||||
Backup expiry is the deployment's own schedule. Content erased from the live database stays in backups until they expire.
|
||||
|
||||
## What this doesn't cover
|
||||
|
||||
These stores need their own handling where a deployment uses them:
|
||||
- terminal transcripts;
|
||||
- daemon and service logs;
|
||||
- gateway and Slack stores and Slack's own copies;
|
||||
- downloaded files and artifacts;
|
||||
- chat messages (removed with their rig);
|
||||
- events tied neither to a row nor to the account's rig.
|
||||
|
||||
The rig itself (its nodes, sessions and bindings) is removed by removing the rig, between the two runs above.
|
||||
@@ -107,6 +107,14 @@ Keys:
|
||||
than N days; 0 keeps them (default). Removed events no longer replay from /api/events, leave
|
||||
the telemetry export, the log and the feed, and can't be brought back. Read at the next sweep
|
||||
(daily and at daemon start)
|
||||
retention.closed_content_days remove the content of closed queue work (done, canceled, denied, handed-off) N
|
||||
days after it closed; the row stays, without content. 0 keeps it (default). Can't be undone
|
||||
retention.closed_metadata_days delete closed queue rows N days after they closed, keeping only their ids so a retry
|
||||
can't recreate them. 0 keeps them (default). Both read at the next sweep; "rig queue erase
|
||||
--routine --apply" runs them now
|
||||
retention.erasure_ledger an absolute file where queue erasure appends each erased id and its action, one JSON
|
||||
line each, before it commits them; keep it outside backups and replay it with "rig queue
|
||||
erase --ids <file> --apply" after a restore. Empty keeps no ledger (default)
|
||||
launch.non_interruptive accept harness first-launch warnings at full bypass for new rigs (default off)
|
||||
launch.codex_login_shell defer to Codex's own login-shell setting in managed Codex seats (default off)
|
||||
Off adds allow_login_shell=false: tool commands inherit the seat's environment, so plain rig
|
||||
|
||||
@@ -1999,6 +1999,100 @@ Examples:
|
||||
});
|
||||
});
|
||||
|
||||
cmd
|
||||
.command("erase")
|
||||
.description("Erase queue rows in one exact scope: an account's rows, a list of ids, or the closed-work retention windows; a dry run unless --apply")
|
||||
.option("--members <file>", "Account deletion: the account's recorded session names (exact, departed seats included), one per line or a JSON array")
|
||||
.option("--rig <id>", "With --members: the account rig's id. Also erases the rig's own records that outlive it (events, snapshots, node and seat records); refused unless every member is a session of it")
|
||||
.option("--ids <file>", "These qitem ids, one per line or a JSON array; a report's qitems entries, or an erasure ledger's JSON lines, keep each one's action")
|
||||
.option("--action <action>", "With --ids: delete (default) removes each row and keeps only its id; scrub removes its content and keeps the row", enumArg(["delete", "scrub"]))
|
||||
.option("--routine", "Run this daemon's closed-work windows now (retention.closed_content_days, retention.closed_metadata_days)")
|
||||
.option("--apply", "Erase. Without it the command reports what would be erased and changes nothing")
|
||||
.option("--actor <session>", "Who is erasing, when OPENRIG_SESSION_NAME isn't set (recorded as a claimed identity)")
|
||||
.option("--json", "JSON output for agents")
|
||||
.addHelpText("after", [
|
||||
"",
|
||||
"Name exactly one scope. What each one erases:",
|
||||
" --members every row whose source and destination are both members, in any state, deleted with its transitions,",
|
||||
" outbox and inbox copies, stream actions, events and audit rows; then member-to-member outbox and inbox",
|
||||
" entries no row owns, and the members' stream items. A row with one member endpoint is reported under",
|
||||
" mixed and left untouched. With --rig, also the rig's own events, snapshots and node and seat records.",
|
||||
" --ids the listed rows, deleted or scrubbed (a report's qitems entries keep their action); a deleted id",
|
||||
" that's already gone keeps (or regains) its id marker.",
|
||||
" --routine closed rows (done, canceled, denied, handed-off) past the windows, aged from when they closed.",
|
||||
"A deleted row keeps only its id, and a create that reuses that id is refused (qitem_erased). A row a workflow still",
|
||||
"references is reported under gaps and left. Output is JSON: clean (true only when nothing was left), complete,",
|
||||
"qitems, mixed, gaps, alreadyAbsent, linked, unlinked and rig. Refusals: erase_scope_required, erase_scope_conflict,",
|
||||
"erase_members_invalid, erase_rig_invalid, erase_rig_mismatch, erase_ids_invalid, erase_action_invalid,",
|
||||
"erase_routine_off. A run that stops on an error returns what it committed, with error.stage, and exits 2.",
|
||||
"Reference: docs/reference/queue-erasure.md",
|
||||
].join("\n"))
|
||||
.action(async (opts: { members?: string; rig?: string; ids?: string; action?: string; routine?: boolean; apply?: boolean; actor?: string; json?: boolean }) => {
|
||||
const list = (flag: string, path: string): unknown[] | undefined => {
|
||||
let text: string;
|
||||
try {
|
||||
text = fs.readFileSync(path, "utf8");
|
||||
} catch (err) {
|
||||
console.error(`--${flag} ${path}: can't be read (${err instanceof Error ? err.message : String(err)}). Nothing was erased.`);
|
||||
process.exitCode = 1;
|
||||
return undefined;
|
||||
}
|
||||
if (text.trimStart().startsWith("[")) {
|
||||
try {
|
||||
const parsed: unknown = JSON.parse(text);
|
||||
if (Array.isArray(parsed)) return parsed;
|
||||
} catch { /* reported below */ }
|
||||
console.error(`--${flag} ${path}: not a JSON array. Nothing was erased.`);
|
||||
process.exitCode = 1;
|
||||
return undefined;
|
||||
}
|
||||
// One entry per line: a plain id or session, or (an erasure ledger's line) a JSON object.
|
||||
const entries: unknown[] = [];
|
||||
for (const line of text.split(/\r?\n/).map((entry) => entry.trim()).filter((entry) => entry.length > 0 && !entry.startsWith("#"))) {
|
||||
if (!line.startsWith("{")) {
|
||||
entries.push(line);
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
entries.push(JSON.parse(line));
|
||||
} catch {
|
||||
console.error(`--${flag} ${path}: a line isn't valid JSON (${line.slice(0, 60)}). Nothing was erased.`);
|
||||
process.exitCode = 1;
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
return entries;
|
||||
};
|
||||
const body: Record<string, unknown> = { apply: opts.apply === true };
|
||||
if (opts.members !== undefined) {
|
||||
const members = list("members", opts.members);
|
||||
if (!members) return;
|
||||
body.members = members;
|
||||
}
|
||||
// Sent as given: the daemon refuses a rig without members (erase_rig_invalid), never drops it.
|
||||
if (opts.rig !== undefined) body.rig = opts.rig;
|
||||
if (opts.ids !== undefined) {
|
||||
const ids = list("ids", opts.ids);
|
||||
if (!ids) return;
|
||||
body.ids = ids;
|
||||
if (opts.action) body.action = opts.action;
|
||||
}
|
||||
if (opts.routine) body.routine = true;
|
||||
if (!readOpenRigEnv("OPENRIG_SESSION_NAME", "RIGGED_SESSION_NAME")) {
|
||||
if (!opts.actor) {
|
||||
console.error("OPENRIG_SESSION_NAME is not set and --actor wasn't given, so there's no one to record as erasing. Nothing was erased.");
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
body.actorSession = opts.actor;
|
||||
}
|
||||
const deps = getDeps();
|
||||
await withClient(deps, async (client) => {
|
||||
const res = await client.post<unknown>("/api/queue/erase", body);
|
||||
printResult(opts.json ?? false, res.data, res.status);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- Inbox subcommands ----
|
||||
|
||||
cmd
|
||||
|
||||
@@ -221,6 +221,9 @@ export interface RiggedConfig {
|
||||
streamDays: number;
|
||||
streamMaxItems: number;
|
||||
eventsDays: number;
|
||||
closedContentDays: number;
|
||||
closedMetadataDays: number;
|
||||
erasureLedger: string;
|
||||
batchSize: number;
|
||||
};
|
||||
// S04 (OPR.0.5.5.4) — pickup-receipt stall threshold (twin of the daemon settings-store).
|
||||
@@ -415,6 +418,9 @@ const DEFAULTS = {
|
||||
streamDays: 30,
|
||||
streamMaxItems: 20000,
|
||||
eventsDays: 0,
|
||||
closedContentDays: 0,
|
||||
closedMetadataDays: 0,
|
||||
erasureLedger: "",
|
||||
batchSize: 500,
|
||||
},
|
||||
// S04 — pickup-receipt stall threshold default (twin of daemon getDefaultValue).
|
||||
@@ -565,6 +571,12 @@ export const VALID_KEYS = [
|
||||
"retention.stream_max_items",
|
||||
// The history-only event types of `events` older than N days, 0 (default) keeps them; lockstep with the daemon twin.
|
||||
"retention.events_days",
|
||||
// Closed queue work: content erased N days after it closed, then the row (only its id kept); 0 (default) keeps them;
|
||||
// lockstep with the daemon twin.
|
||||
"retention.closed_content_days",
|
||||
"retention.closed_metadata_days",
|
||||
// Queue erasure's ledger file (JSON lines of erased ids and actions); empty keeps none; lockstep with the daemon twin.
|
||||
"retention.erasure_ledger",
|
||||
"stream.intake",
|
||||
"retention.batch_size",
|
||||
// S04 — pickup-receipt stall threshold; lockstep with the daemon settings-store twin.
|
||||
@@ -687,6 +699,9 @@ export const ENV_MAP: Record<ValidKey, { primary: string; legacy?: string }> = {
|
||||
"retention.stream_days": { primary: "OPENRIG_RETENTION_STREAM_DAYS" },
|
||||
"retention.stream_max_items": { primary: "OPENRIG_RETENTION_STREAM_MAX_ITEMS" },
|
||||
"retention.events_days": { primary: "OPENRIG_RETENTION_EVENTS_DAYS" },
|
||||
"retention.closed_content_days": { primary: "OPENRIG_RETENTION_CLOSED_CONTENT_DAYS" },
|
||||
"retention.closed_metadata_days": { primary: "OPENRIG_RETENTION_CLOSED_METADATA_DAYS" },
|
||||
"retention.erasure_ledger": { primary: "OPENRIG_RETENTION_ERASURE_LEDGER" },
|
||||
"stream.intake": { primary: "OPENRIG_STREAM_INTAKE" },
|
||||
"retention.batch_size": { primary: "OPENRIG_RETENTION_BATCH_SIZE" },
|
||||
"queue.pickup_stall_threshold_minutes": { primary: "OPENRIG_QUEUE_PICKUP_STALL_THRESHOLD_MINUTES" },
|
||||
@@ -792,6 +807,9 @@ const KEY_TO_PATH: Record<ValidKey, string[]> = {
|
||||
"retention.stream_days": ["retention", "streamDays"],
|
||||
"retention.stream_max_items": ["retention", "streamMaxItems"],
|
||||
"retention.events_days": ["retention", "eventsDays"],
|
||||
"retention.closed_content_days": ["retention", "closedContentDays"],
|
||||
"retention.closed_metadata_days": ["retention", "closedMetadataDays"],
|
||||
"retention.erasure_ledger": ["retention", "erasureLedger"],
|
||||
"stream.intake": ["stream", "intake"],
|
||||
"retention.batch_size": ["retention", "batchSize"],
|
||||
"queue.pickup_stall_threshold_minutes": ["queue", "pickupStallThresholdMinutes"],
|
||||
@@ -1178,6 +1196,24 @@ const KEY_CONSTRAINTS: Partial<Record<ValidKey, (raw: string, coerced: string |
|
||||
throw new Error(`Invalid value for retention.events_days: must be an integer >= 0 (0 keeps the events), got "${raw}"`);
|
||||
}
|
||||
},
|
||||
// 0 (the default) keeps closed work; a positive day count turns its erasure on (queue-erasure).
|
||||
"retention.closed_content_days": (raw, coerced) => {
|
||||
if (!/^-?\d+$/.test((raw ?? "").trim()) || typeof coerced !== "number" || !Number.isInteger(coerced) || coerced < 0) {
|
||||
throw new Error(`Invalid value for retention.closed_content_days: must be an integer >= 0 (0 keeps the content), got "${raw}"`);
|
||||
}
|
||||
},
|
||||
"retention.closed_metadata_days": (raw, coerced) => {
|
||||
if (!/^-?\d+$/.test((raw ?? "").trim()) || typeof coerced !== "number" || !Number.isInteger(coerced) || coerced < 0) {
|
||||
throw new Error(`Invalid value for retention.closed_metadata_days: must be an integer >= 0 (0 keeps the rows), got "${raw}"`);
|
||||
}
|
||||
},
|
||||
// Empty (the default) keeps no erasure ledger; otherwise an absolute file path, kept outside the backed-up data.
|
||||
"retention.erasure_ledger": (raw) => {
|
||||
const value = (raw ?? "").trim();
|
||||
if (value !== "" && !value.startsWith("/")) {
|
||||
throw new Error(`Invalid value for retention.erasure_ledger: must be empty or an absolute file path, got "${raw}"`);
|
||||
}
|
||||
},
|
||||
"retention.batch_size": (raw, coerced) => {
|
||||
if (!/^-?\d+$/.test((raw ?? "").trim()) || typeof coerced !== "number" || !Number.isInteger(coerced) || coerced < 1) {
|
||||
throw new Error(`Invalid value for retention.batch_size: must be an integer >= 1, got "${raw}"`);
|
||||
@@ -1449,6 +1485,9 @@ export class ConfigStore {
|
||||
streamDays: v("retention.stream_days") as number,
|
||||
streamMaxItems: v("retention.stream_max_items") as number,
|
||||
eventsDays: v("retention.events_days") as number,
|
||||
closedContentDays: v("retention.closed_content_days") as number,
|
||||
closedMetadataDays: v("retention.closed_metadata_days") as number,
|
||||
erasureLedger: v("retention.erasure_ledger") as string,
|
||||
batchSize: v("retention.batch_size") as number,
|
||||
},
|
||||
queue: {
|
||||
|
||||
@@ -216,6 +216,9 @@ describe("ConfigStore — extended namespaces (User Settings v0)", () => {
|
||||
"retention.stream_days", // slice 22 outcome 9: stream retention
|
||||
"retention.stream_max_items",
|
||||
"retention.events_days", // the history-only event types of events
|
||||
"retention.closed_content_days", // closed-work erasure: content
|
||||
"retention.closed_metadata_days", // closed-work erasure: the row, keeping its id
|
||||
"retention.erasure_ledger", // queue erasure's replay ledger file
|
||||
"stream.intake",
|
||||
"retention.batch_size",
|
||||
"queue.pickup_stall_threshold_minutes",
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
import type { QueueDeps } from "../src/commands/queue.js";
|
||||
import { createProgram } from "../src/index.js";
|
||||
|
||||
// `rig queue erase`: the file it reads, the request it sends, and what it refuses before sending anything.
|
||||
vi.mock("../src/daemon-lifecycle.js", async () => {
|
||||
const actual = await vi.importActual<Record<string, unknown>>("../src/daemon-lifecycle.js");
|
||||
return {
|
||||
...actual,
|
||||
getDaemonStatus: vi.fn(async () => ({ state: "running", healthy: true, pid: 1234, port: 7433 })),
|
||||
getDaemonUrl: vi.fn(() => "http://localhost:7433"),
|
||||
};
|
||||
});
|
||||
|
||||
function makeDeps(response: { status: number; data: unknown } = { status: 200, data: { ok: true } }) {
|
||||
const posts: Array<{ path: string; body: unknown }> = [];
|
||||
const deps = {
|
||||
lifecycleDeps: {} as QueueDeps["lifecycleDeps"],
|
||||
clientFactory: () => ({
|
||||
post: vi.fn(async (urlPath: string, body: unknown) => {
|
||||
posts.push({ path: urlPath, body });
|
||||
return response;
|
||||
}),
|
||||
}) as unknown as ReturnType<QueueDeps["clientFactory"]>,
|
||||
} as QueueDeps;
|
||||
return { deps, posts };
|
||||
}
|
||||
|
||||
describe("rig queue erase", () => {
|
||||
let dir: string;
|
||||
let errors: string[];
|
||||
beforeEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
vi.stubEnv("OPENRIG_SESSION_NAME", "ops-admin@n-ops");
|
||||
dir = fs.mkdtempSync(path.join(os.tmpdir(), "queue-erase-"));
|
||||
errors = [];
|
||||
vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
vi.spyOn(console, "error").mockImplementation((...args) => errors.push(args.join(" ")));
|
||||
process.exitCode = undefined;
|
||||
});
|
||||
const run = (deps: QueueDeps, ...args: string[]) => createProgram({ queueDeps: deps }).parseAsync(["node", "rig", "queue", "erase", ...args]);
|
||||
|
||||
it("reads a members file (one per line, comments skipped) and sends a dry run unless --apply", async () => {
|
||||
const file = path.join(dir, "members.txt");
|
||||
fs.writeFileSync(file, "# account n-acct\na-one@n-acct\n\n b-two@n-acct \n");
|
||||
const { deps, posts } = makeDeps();
|
||||
await run(deps, "--members", file);
|
||||
await run(deps, "--members", file, "--apply");
|
||||
expect(posts).toEqual([
|
||||
{ path: "/api/queue/erase", body: { apply: false, members: ["a-one@n-acct", "b-two@n-acct"] } },
|
||||
{ path: "/api/queue/erase", body: { apply: true, members: ["a-one@n-acct", "b-two@n-acct"] } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("reads ids as a JSON array and passes the action", async () => {
|
||||
const file = path.join(dir, "ids.json");
|
||||
fs.writeFileSync(file, JSON.stringify(["qitem-1", "qitem-2"]));
|
||||
const { deps, posts } = makeDeps();
|
||||
await run(deps, "--ids", file, "--action", "scrub", "--apply");
|
||||
expect(posts).toEqual([{ path: "/api/queue/erase", body: { apply: true, ids: ["qitem-1", "qitem-2"], action: "scrub" } }]);
|
||||
});
|
||||
|
||||
it("passes the account rig with the members, and a ledger of report entries as given", async () => {
|
||||
const members = path.join(dir, "members.json");
|
||||
fs.writeFileSync(members, JSON.stringify(["a-one@n-acct"]));
|
||||
const ledger = path.join(dir, "ledger.json");
|
||||
const entries = [{ qitemId: "qitem-1", action: "scrub", state: "done" }, "qitem-2"];
|
||||
fs.writeFileSync(ledger, JSON.stringify(entries));
|
||||
const { deps, posts } = makeDeps();
|
||||
await run(deps, "--members", members, "--rig", "rig-acct-id", "--apply");
|
||||
await run(deps, "--ids", ledger, "--apply");
|
||||
expect(posts).toEqual([
|
||||
{ path: "/api/queue/erase", body: { apply: true, members: ["a-one@n-acct"], rig: "rig-acct-id" } },
|
||||
{ path: "/api/queue/erase", body: { apply: true, ids: entries } },
|
||||
]);
|
||||
});
|
||||
|
||||
it("reads an erasure ledger's JSON lines as entries that keep their action", async () => {
|
||||
const ledger = path.join(dir, "ledger.jsonl");
|
||||
fs.writeFileSync(ledger, '{"qitemId":"qitem-1","action":"scrub","reason":"retention-metadata","erasedAt":"t"}\n# a comment\n{"qitemId":"qitem-2","action":"delete"}\n');
|
||||
const { deps, posts } = makeDeps();
|
||||
await run(deps, "--ids", ledger, "--apply");
|
||||
expect(posts).toEqual([{ path: "/api/queue/erase", body: { apply: true, ids: [
|
||||
{ qitemId: "qitem-1", action: "scrub", reason: "retention-metadata", erasedAt: "t" }, { qitemId: "qitem-2", action: "delete" },
|
||||
] } }]);
|
||||
fs.writeFileSync(ledger, '{"qitemId": broken\n');
|
||||
process.exitCode = undefined;
|
||||
await run(deps, "--ids", ledger, "--apply");
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(posts).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("sends nothing when its file can't be read, or when there's no one to record as erasing", async () => {
|
||||
const { deps, posts } = makeDeps();
|
||||
await run(deps, "--members", path.join(dir, "missing.txt"));
|
||||
expect(process.exitCode).toBe(1);
|
||||
vi.stubEnv("OPENRIG_SESSION_NAME", "");
|
||||
process.exitCode = undefined;
|
||||
await run(deps, "--routine");
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(posts).toEqual([]);
|
||||
expect(errors.join("\n")).toMatch(/Nothing was erased/);
|
||||
await run(deps, "--routine", "--actor", "erase-runner@n-ops");
|
||||
expect(posts).toEqual([{ path: "/api/queue/erase", body: { apply: false, routine: true, actorSession: "erase-runner@n-ops" } }]);
|
||||
});
|
||||
|
||||
it("a daemon refusal exits nonzero", async () => {
|
||||
const { deps } = makeDeps({ status: 400, data: { error: "erase_scope_required", message: "Name one scope" } });
|
||||
await run(deps);
|
||||
expect(process.exitCode).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -2,6 +2,8 @@ import { piAccountSwitchesSchema } from "./migrations/151_pi_account_switches.js
|
||||
import { workflowInstanceSpecSchema } from "./migrations/152_workflow_instance_spec.js";
|
||||
import { eventsRetentionMarksSchema } from "./migrations/153_events_retention_marks.js";
|
||||
import { workflowInstanceRootObjectiveSchema } from "./migrations/154_workflow_instance_root_objective.js";
|
||||
import { queueErasureSchema } from "./migrations/155_queue_erasure.js";
|
||||
import { queueErasureLinkIndexesSchema } from "./migrations/156_queue_erasure_link_indexes.js";
|
||||
import { activityGrantsSchema } from "./migrations/143_activity_grants.js";
|
||||
import { environmentBindingsSchema } from "./migrations/144_environment_bindings.js";
|
||||
import { nodePermissionsToSchema } from "./migrations/145_node_permissions_to.js";
|
||||
@@ -311,4 +313,6 @@ export const ALL_MIGRATIONS: Migration[] = [
|
||||
workflowInstanceSpecSchema,
|
||||
eventsRetentionMarksSchema,
|
||||
workflowInstanceRootObjectiveSchema,
|
||||
queueErasureSchema,
|
||||
queueErasureLinkIndexesSchema,
|
||||
];
|
||||
@@ -0,0 +1,29 @@
|
||||
import type { Migration } from "../migrate.js";
|
||||
|
||||
/**
|
||||
* Queue erasure (closed-work retention and account deletion). `content_erased_at` marks a row whose content was
|
||||
* removed: the row stays, without content, so a retried create with its id still finds it and makes no new work.
|
||||
* A deleted row leaves only its id in `queue_erased_ids`, with no endpoints and no content, and the create path
|
||||
* refuses that id (`qitem_erased`). `queue_erased_rigs` keeps an account rig's id and name with a digest of the
|
||||
* membership it was verified against, so the run after the rig is removed still erases and counts its records exactly.
|
||||
* Once nothing of the rig is left, the record becomes a completion receipt (`completed_at` set, the name cleared), so an
|
||||
* identical rerun still verifies and reports the same result.
|
||||
*/
|
||||
export const queueErasureSchema: Migration = {
|
||||
name: "155_queue_erasure.sql",
|
||||
sql: `
|
||||
ALTER TABLE queue_items ADD COLUMN content_erased_at TEXT;
|
||||
CREATE TABLE IF NOT EXISTS queue_erased_ids (
|
||||
qitem_id TEXT PRIMARY KEY,
|
||||
erased_at TEXT NOT NULL,
|
||||
reason TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS queue_erased_rigs (
|
||||
rig_id TEXT PRIMARY KEY,
|
||||
rig_name TEXT NOT NULL,
|
||||
members_digest TEXT NOT NULL,
|
||||
verified_at TEXT NOT NULL,
|
||||
completed_at TEXT
|
||||
);
|
||||
`,
|
||||
};
|
||||
@@ -0,0 +1,15 @@
|
||||
import type { Migration } from "../migrate.js";
|
||||
|
||||
/** Queue erasure finds a row's copies by its id; without these, each lookup scans the whole table. */
|
||||
export const queueErasureLinkIndexesSchema: Migration = {
|
||||
name: "156_queue_erasure_link_indexes.sql",
|
||||
sql: `
|
||||
CREATE INDEX IF NOT EXISTS idx_outbox_entries_audit_pointer ON outbox_entries(audit_pointer);
|
||||
CREATE INDEX IF NOT EXISTS idx_inbox_entries_absorbed_qitem ON inbox_entries(absorbed_qitem_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_inbox_entries_audit_pointer ON inbox_entries(audit_pointer);
|
||||
CREATE INDEX IF NOT EXISTS idx_stream_dispositions_work_qitem ON stream_dispositions(work_qitem_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_stream_dispositions_effect_qitem ON stream_dispositions(effect_qitem_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_stream_reader_actions_work_qitem ON stream_reader_actions(work_qitem_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_stream_reader_actions_duplicate_of ON stream_reader_actions(duplicate_of);
|
||||
`,
|
||||
};
|
||||
@@ -170,16 +170,6 @@ function atxHeading(line: string): { level: number; title: string } | null {
|
||||
export function parseMarkdownSections(text: string): MarkdownSection[] {
|
||||
const lines = text.split("\n");
|
||||
const { hits: headers } = scanHeaders(lines);
|
||||
// Resolve all section ends in one backwards pass. Copying the remaining
|
||||
// header array for every section makes even sibling-only documents quadratic.
|
||||
const fullEnds: number[] = [];
|
||||
const following: HeaderHit[] = [];
|
||||
for (let idx = headers.length - 1; idx >= 0; idx--) {
|
||||
const header = headers[idx]!;
|
||||
while (following.length && following[following.length - 1]!.level > header.level) following.pop();
|
||||
fullEnds[idx] = following[following.length - 1]?.line ?? lines.length;
|
||||
following.push(header);
|
||||
}
|
||||
const sections: MarkdownSection[] = [];
|
||||
let currentH2: string | null = null;
|
||||
for (let idx = 0; idx < headers.length; idx++) {
|
||||
@@ -196,7 +186,7 @@ export function parseMarkdownSections(text: string): MarkdownSection[] {
|
||||
// instead of being silently promoted to top-level addresses.
|
||||
const headerPath = h.level === 2 ? [slug] : currentH2 !== null ? [currentH2, slug] : [slug];
|
||||
// FULL span: to the next header with level <= this one (same-or-higher).
|
||||
const fullEnd = fullEnds[idx]!;
|
||||
const fullEnd = headers.slice(idx + 1).find((n) => n.level <= h.level)?.line ?? lines.length;
|
||||
// OWN text: to the next header of ANY level.
|
||||
const ownEnd = headers[idx + 1]?.line ?? lines.length;
|
||||
sections.push({
|
||||
|
||||
@@ -128,7 +128,7 @@ function ledger(dir: string, io: ScopeFsDeps): Judgment[] {
|
||||
const expected = `${String(receipts.length + 1).padStart(8, "0")}.md`;
|
||||
if (name !== expected) throw new JudgmentError("journal_conflict", `${home}: expected ${expected}, found ${name}; inspect retained receipts`);
|
||||
const raw = io.readFile(path.join(home, name));
|
||||
const fm = raw && /^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n|$)/.exec(raw);
|
||||
const fm = raw && /^---\n([\s\S]*?)\n---(?:\n|$)/.exec(raw);
|
||||
if (!fm) throw new JudgmentError("journal_invalid", `${home}/${name}: incomplete judgment`);
|
||||
const obj = mapping(YAML.parse(fm[1]!), name), { id, ...payload } = obj;
|
||||
if (id !== hash(payload) || obj.version !== 1 || obj.sequence !== receipts.length + 1 || typeof obj.itemId !== "string" || typeof obj.itemRevision !== "string" || typeof obj.policyRevision !== "string" || typeof obj.actor !== "string" || typeof obj.operationId !== "string" || typeof obj.intent !== "string" || !["accept", "reject", "withdraw"].includes(String(obj.verdict)) || !Array.isArray(obj.evidence)) throw new JudgmentError("journal_invalid", `${home}/${name}: malformed or changed judgment; inspect history`);
|
||||
|
||||
@@ -0,0 +1,822 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { appendFileSync } from "node:fs";
|
||||
import type Database from "better-sqlite3";
|
||||
|
||||
/**
|
||||
* Queue erasure: the maintained way to remove queue content, for closed-work retention and account deletion on a
|
||||
* daemon that serves several accounts. Two actions on a queue row:
|
||||
* - scrub: the row's content is removed and the row stays, content-free, with its id, endpoints, state and times
|
||||
* (`content_erased_at` set). The create path's retry identity still finds it, so a retried create makes no new work.
|
||||
* A later note, summary or reclose makes it eligible again.
|
||||
* - delete: the row and everything linked to it go. Only its id is kept, in `queue_erased_ids` (no endpoints, no
|
||||
* content), and the create path refuses that id (`qitem_erased`).
|
||||
* The row's own copies follow it: its transitions, live and archived; events whose payload names it; its
|
||||
* mission-control audit rows. Copies another party holds (outbox and inbox entries, stream reader actions and
|
||||
* dispositions) go only when they're the owner's: both endpoints the account's (members scope) or the row's own
|
||||
* (other scopes). Others are kept and reported.
|
||||
*
|
||||
* Scope is exact. Account deletion takes the account's recorded membership (exact session names): a row whose two
|
||||
* endpoints are both members is deleted, whatever its state; a row with one member endpoint is reported as mixed and
|
||||
* left untouched. Named with the account's rig, it also removes the rig's own records that outlive the rig. Routine
|
||||
* retention touches only closed rows (done, canceled, denied, handed-off; never failed or open work), aged from the
|
||||
* start of their final closed state. A row a workflow still references is reported, never erased.
|
||||
*
|
||||
* A run selects everything first, then reads the events table once in seq windows to find the events naming each
|
||||
* selected row (no index serves a payload match, and one statement over a large table would hold the daemon), then
|
||||
* erases batch by batch, yielding between steps. Each batch is one transaction (better-sqlite3 commits a transaction
|
||||
* on return and rolls it back on a throw): it rechecks its rows, since anything may have changed them since selection
|
||||
* (a reopened row is open work again; a rerouted one may no longer be the account's), writes their ledger lines, then
|
||||
* changes their events, copies and rows. A row an unreadable event names is kept, so reruns keep reporting it. A run
|
||||
* that stops, by its batch limit or an error, returns only what it committed; a rerun picks up the rest.
|
||||
*/
|
||||
|
||||
/** The closed states routine retention may erase. `failed` stays out: account deletion covers it. */
|
||||
export const CLOSED_QUEUE_STATES = ["done", "canceled", "denied", "handed-off"] as const;
|
||||
/** The closure reasons whose closure_target names a successor; for the others it can be a reason or note. */
|
||||
const ID_CLOSURE_REASONS = ["handed_off_to", "superseded"] as const;
|
||||
/** A successor as a closure target names it: a session address, or a queue id (either with a host suffix). */
|
||||
const ADDRESS_SHAPE = /^[A-Za-z0-9][A-Za-z0-9._-]*@[A-Za-z0-9._-]+(?:@[A-Za-z0-9._-]+)?$/;
|
||||
const QITEM_SHAPE = /^qitem-[A-Za-z0-9._-]+(?:@[A-Za-z0-9._-]+)?$/;
|
||||
|
||||
/** A scrub keeps a closure target only when it names a successor: closure writes don't constrain the target, so text
|
||||
* under a successor reason is cleared like any other note. */
|
||||
function keepsClosureTarget(reason: unknown, target: unknown): boolean {
|
||||
return ID_CLOSURE_REASONS.includes(reason as (typeof ID_CLOSURE_REASONS)[number]) && typeof target === "string"
|
||||
&& target.length <= 256 && (ADDRESS_SHAPE.test(target) || QITEM_SHAPE.test(target));
|
||||
}
|
||||
|
||||
export type EraseAction = "scrub" | "delete";
|
||||
export type EraseScope =
|
||||
| { kind: "members"; members: readonly string[]; rig?: string }
|
||||
| { kind: "ids"; entries: ReadonlyArray<{ qitemId: string; action: EraseAction }> }
|
||||
| { kind: "routine"; contentDays: number; metadataDays: number };
|
||||
|
||||
/** A scope that can't be erased exactly; nothing was written. */
|
||||
export class EraseRefusal extends Error {
|
||||
constructor(readonly code: string, message: string) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
export interface EraseOptions {
|
||||
scope: EraseScope;
|
||||
/** false: report what would be erased and write nothing */
|
||||
apply: boolean;
|
||||
nowIso: string;
|
||||
batchSize?: number;
|
||||
/** selection stops after this many pages and the run reports complete: false; a rerun continues */
|
||||
maxBatches?: number;
|
||||
/** events rows (by seq) read per step of the events walk */
|
||||
eventsWindow?: number;
|
||||
/** append each erased row's {qitemId, action, reason, erasedAt} here, one JSON line each, before its batch changes
|
||||
* anything; the file is opened before the run writes anything at all */
|
||||
ledgerPath?: string;
|
||||
}
|
||||
|
||||
type Stage = "ledger" | "select" | "events" | "rows" | "unlinked" | "rig";
|
||||
type CopyTable = "outbox_entries" | "inbox_entries" | "stream_reader_actions" | "stream_dispositions";
|
||||
export type EraseGap = { code: "workflow-reference" | "not-found" | "changed-during-run" | "unreadable-event"; qitemId: string };
|
||||
|
||||
export interface EraseReport {
|
||||
ok: true;
|
||||
applied: boolean;
|
||||
scope: EraseScope["kind"];
|
||||
/** false when the run stopped at maxBatches or on an error; rerun to continue */
|
||||
complete: boolean;
|
||||
/** complete, and nothing was left: no mixed rows or copies, gaps, unreadable events, or left-over entries or rig
|
||||
* records */
|
||||
clean: boolean;
|
||||
/** why the run stopped early, and where; everything listed in qitems was committed before it */
|
||||
error?: { stage: Stage; message: string };
|
||||
/** rows erased (applied, committed) or that would be (dry run) */
|
||||
qitems: Array<{ qitemId: string; action: EraseAction; state: string }>;
|
||||
/** members scope: rows with exactly one member endpoint, left untouched */
|
||||
mixed: Array<{ qitemId: string; memberEndpoint: "source" | "destination" }>;
|
||||
/** rows not erased, and why: a workflow references them; (ids scope) no such row to scrub; the row changed after
|
||||
* selection so it no longer qualifies (reopened, rerouted out of the account, written to out of its window, or an
|
||||
* absent id created since); or an event naming it can't be read */
|
||||
gaps: EraseGap[];
|
||||
/** ids scope, delete: rows already gone whose id marker is (re)written; listed once the marker is committed */
|
||||
alreadyAbsent: string[];
|
||||
/** an erased row's copies held with someone else: mixed (one of their sessions is the owner's) or foreign (none) */
|
||||
keptCopies: Array<{ table: CopyTable; id: string; qitemId: string; ownership: "mixed" | "foreign" }>;
|
||||
/** events naming a selected row whose payload can't be read: kept, since what they hold can't be inspected. The
|
||||
* rows they name are kept too (gap unreadable-event), so every rerun finds them again until they're resolved */
|
||||
unreadableEvents: Array<{ seq: number; qitemIds: string[] }>;
|
||||
/** copies erased with the rows, or that would be */
|
||||
linked: { transitions: number; archivedTransitions: number; outbox: number; inbox: number; events: number; streamActions: number; missionControl: number };
|
||||
/** members scope: entries between members that no queue row owns; mixed ones are counted and left */
|
||||
unlinked?: { outbox: number; inbox: number; streamItems: number; mixedEntries: number; referencedStreamItems: number };
|
||||
/** members scope with rig: whether the rig was found and verified, its records erased and left per table, the
|
||||
* tables whose records can't be counted (an unverified removed rig), and when its erasure completed */
|
||||
rig?: { id: string; found: boolean; verified: boolean; erased: Record<string, number>; left: Record<string, number>; unknown: string[];
|
||||
completedAt?: string };
|
||||
routine?: { contentDays: number; metadataDays: number };
|
||||
}
|
||||
|
||||
const DEFAULT_BATCH = 500;
|
||||
const DEFAULT_MAX_BATCHES = 10_000;
|
||||
const DEFAULT_EVENTS_WINDOW = 1_000;
|
||||
/** Rows erased per transaction. Each one rechecks its rows and rewrites their events, so it's kept smaller than a
|
||||
* selection page: on a 2.4M-event store, 500 rows held the daemon up to 0.5 s, and 100 up to 0.25 s. */
|
||||
const ROWS_PER_TRANSACTION = 100;
|
||||
|
||||
function cutoffIso(nowIso: string, days: number): string {
|
||||
const now = new Date(nowIso);
|
||||
if (Number.isNaN(now.getTime())) throw new Error(`queue-erasure: invalid nowIso ${JSON.stringify(nowIso)}`);
|
||||
return new Date(now.getTime() - days * 86_400_000).toISOString();
|
||||
}
|
||||
|
||||
function yieldToLoop(): Promise<void> {
|
||||
return new Promise((resolve) => setImmediate(resolve));
|
||||
}
|
||||
|
||||
const IDS = "(SELECT value FROM json_each(@ids))";
|
||||
const MEMBERS = "(SELECT value FROM json_each(@members))";
|
||||
/** The payload keys a row-naming event uses: queue events name it in qitemId; inbox, workflow and mission-control
|
||||
* events use these others. */
|
||||
const EVENT_ID_KEYS = ["qitemId", "promotedQitemId", "priorQitemId", "nextQitemId"] as const;
|
||||
/** The content keys of row-naming events; a scrubbed event keeps its type and ids. closureTarget goes too, unless it
|
||||
* names a successor. */
|
||||
const EVENT_CONTENT_KEYS = ["summary", "reason", "error"] as const;
|
||||
/** instr is a cheap prefilter for row-naming events: every id key ends in "temId" (qitemId, promotedQitemId, ...). */
|
||||
const NAMES_A_ROW = "instr(payload, 'temId') > 0";
|
||||
|
||||
class Statements {
|
||||
constructor(private readonly db: Database.Database) {}
|
||||
private cache = new Map<string, Database.Statement>();
|
||||
get(sql: string): Database.Statement {
|
||||
let statement = this.cache.get(sql);
|
||||
if (!statement) {
|
||||
statement = this.db.prepare(sql);
|
||||
this.cache.set(sql, statement);
|
||||
}
|
||||
return statement;
|
||||
}
|
||||
count(sql: string, params: Record<string, unknown>): number {
|
||||
return (this.get(sql).get(params) as { n: number }).n;
|
||||
}
|
||||
}
|
||||
|
||||
/** Rows a workflow still references (a step trail, a frontier binding or a failure record point at them by key, or
|
||||
* a live instance's frontier names them): erasing them would break the workflow or its record. */
|
||||
function workflowReferenced(st: Statements, ids: string[]): Set<string> {
|
||||
if (ids.length === 0) return new Set();
|
||||
const rows = st.get(`SELECT value AS id FROM json_each(@ids) j WHERE
|
||||
EXISTS (SELECT 1 FROM workflow_step_trails WHERE next_qitem_id = j.value OR prior_qitem_id = j.value)
|
||||
OR EXISTS (SELECT 1 FROM workflow_frontier_bindings WHERE packet_id = j.value)
|
||||
OR EXISTS (SELECT 1 FROM workflow_failure_occurrences WHERE failed_packet_id = j.value OR redrive_packet_id = j.value)
|
||||
OR EXISTS (SELECT 1 FROM workflow_instances WHERE status IN ('active', 'waiting')
|
||||
AND current_frontier_json LIKE '%"' || j.value || '"%')`).all({ ids: JSON.stringify(ids) }) as Array<{ id: string }>;
|
||||
return new Set(rows.map((row) => row.id));
|
||||
}
|
||||
|
||||
type Candidate = { qitemId: string; state: string; action: EraseAction; source: string; destination: string };
|
||||
type Copy = { table: CopyTable; id: string; qitemId: string; sessions: Array<string | null> };
|
||||
type Current = { qitemId: string; state: string; tsUpdated: string; erasedAt: string | null; source: string; destination: string };
|
||||
|
||||
/** The copies of these rows that another party holds, each found through its own indexed link. */
|
||||
function linkedCopies(st: Statements, ids: string[]): Copy[] {
|
||||
const params = { ids: JSON.stringify(ids) };
|
||||
const copies = new Map<string, Copy>();
|
||||
const collect = (table: CopyTable, sql: string) => {
|
||||
for (const row of st.get(sql).all(params) as Array<{ id: string; qitemId: string; a: string | null; b: string | null }>) {
|
||||
copies.set(`${table}:${row.id}`, { table, id: String(row.id), qitemId: row.qitemId, sessions: [row.a, row.b] });
|
||||
}
|
||||
};
|
||||
collect("outbox_entries", `SELECT outbox_id AS id, audit_pointer AS qitemId, sender_session AS a, destination_session AS b
|
||||
FROM outbox_entries WHERE audit_pointer IN ${IDS}`);
|
||||
collect("outbox_entries", `SELECT outbox_id AS id, substr(outbox_id, 13) AS qitemId, sender_session AS a, destination_session AS b
|
||||
FROM outbox_entries WHERE outbox_id IN (SELECT 'wake-intent-' || value FROM json_each(@ids))`);
|
||||
collect("inbox_entries", `SELECT inbox_id AS id, absorbed_qitem_id AS qitemId, sender_session AS a, destination_session AS b
|
||||
FROM inbox_entries WHERE absorbed_qitem_id IN ${IDS}`);
|
||||
collect("inbox_entries", `SELECT inbox_id AS id, audit_pointer AS qitemId, sender_session AS a, destination_session AS b
|
||||
FROM inbox_entries WHERE audit_pointer IN ${IDS}`);
|
||||
collect("stream_reader_actions", `SELECT action_id AS id, work_qitem_id AS qitemId, reader_session AS a, NULL AS b
|
||||
FROM stream_reader_actions WHERE work_qitem_id IN ${IDS}`);
|
||||
collect("stream_reader_actions", `SELECT action_id AS id, duplicate_of AS qitemId, reader_session AS a, NULL AS b
|
||||
FROM stream_reader_actions WHERE duplicate_of IN ${IDS}`);
|
||||
collect("stream_dispositions", `SELECT stream_item_id AS id, work_qitem_id AS qitemId, accepted_by AS a, owner_session AS b
|
||||
FROM stream_dispositions WHERE work_qitem_id IN ${IDS}`);
|
||||
collect("stream_dispositions", `SELECT stream_item_id AS id, effect_qitem_id AS qitemId, accepted_by AS a, owner_session AS b
|
||||
FROM stream_dispositions WHERE effect_qitem_id IN ${IDS}`);
|
||||
return [...copies.values()];
|
||||
}
|
||||
|
||||
const COPY_KEY: Record<CopyTable, string> = {
|
||||
outbox_entries: "outbox_id", inbox_entries: "inbox_id", stream_reader_actions: "action_id", stream_dispositions: "stream_item_id",
|
||||
};
|
||||
|
||||
function eraseCopies(st: Statements, copies: Copy[], action: EraseAction): void {
|
||||
for (const table of Object.keys(COPY_KEY) as CopyTable[]) {
|
||||
const keys = copies.filter((copy) => copy.table === table).map((copy) => copy.id);
|
||||
if (keys.length === 0) continue;
|
||||
const params = { keys: JSON.stringify(keys) };
|
||||
const where = `${COPY_KEY[table]} IN (SELECT value FROM json_each(@keys))`;
|
||||
if (action === "delete") {
|
||||
st.get(`DELETE FROM ${table} WHERE ${where}`).run(params);
|
||||
} else if (table === "outbox_entries") {
|
||||
st.get(`UPDATE outbox_entries SET body = '', tags = NULL WHERE ${where}`).run(params);
|
||||
} else if (table === "inbox_entries") {
|
||||
st.get(`UPDATE inbox_entries SET body = '', tags = NULL, denied_reason = NULL WHERE ${where}`).run(params);
|
||||
} else if (table === "stream_reader_actions") {
|
||||
st.get(`UPDATE stream_reader_actions SET reason = NULL WHERE ${where}`).run(params);
|
||||
} else {
|
||||
st.get(`UPDATE stream_dispositions SET template_text = '', request_json = '{}', provenance_json = '{}', reason = NULL,
|
||||
result_json = NULL, applied_json = NULL WHERE ${where}`).run(params);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function scrubRows(st: Statements, ids: string[], nowIso: string): void {
|
||||
const params = { ids: JSON.stringify(ids) };
|
||||
// Closure targets that don't name a successor, found here: SQLite has no pattern match for the id shapes.
|
||||
for (const [table, key] of [["queue_items", "qitem_id"], ["queue_transitions", "transition_id"], ["queue_transitions_archive", "transition_id"]]) {
|
||||
const notes = (st.get(`SELECT ${key} AS k, closure_reason AS reason, closure_target AS target FROM ${table}
|
||||
WHERE qitem_id IN ${IDS} AND closure_target IS NOT NULL`).all(params) as Array<{ k: string | number; reason: string | null; target: string }>)
|
||||
.filter((row) => !keepsClosureTarget(row.reason, row.target)).map((row) => row.k);
|
||||
if (notes.length > 0) {
|
||||
st.get(`UPDATE ${table} SET closure_target = NULL WHERE ${key} IN (SELECT value FROM json_each(@keys))`).run({ keys: JSON.stringify(notes) });
|
||||
}
|
||||
}
|
||||
st.get(`UPDATE queue_items SET body = '', summary = NULL, resolution = NULL, tags = NULL, blocked_on = NULL,
|
||||
chain_of_record = NULL, evidence_ref = NULL, human_detail = NULL, human_questions = NULL, human_answers = NULL,
|
||||
delegated_decision = NULL, last_nudge_input_box = NULL, content_erased_at = @now WHERE qitem_id IN ${IDS}`).run({ ...params, now: nowIso });
|
||||
for (const table of ["queue_transitions", "queue_transitions_archive"]) {
|
||||
st.get(`UPDATE ${table} SET transition_note = NULL, identity_principal = NULL WHERE qitem_id IN ${IDS}`).run(params);
|
||||
}
|
||||
st.get(`UPDATE mission_control_actions SET before_state_json = NULL, after_state_json = NULL, reason = NULL,
|
||||
annotation = NULL, notify_result = NULL, audit_notes_json = NULL WHERE qitem_id IN ${IDS}`).run(params);
|
||||
}
|
||||
|
||||
function deleteRows(st: Statements, ids: string[], nowIso: string, reason: string): void {
|
||||
const params = { ids: JSON.stringify(ids) };
|
||||
// mission_control_actions references queue_items by key, so it goes before the row.
|
||||
st.get(`DELETE FROM mission_control_actions WHERE qitem_id IN ${IDS}`).run(params);
|
||||
st.get(`DELETE FROM queue_transition_wakes WHERE qitem_id IN ${IDS}`).run(params);
|
||||
st.get(`DELETE FROM queue_transitions WHERE qitem_id IN ${IDS}`).run(params);
|
||||
st.get(`DELETE FROM queue_transitions_archive WHERE qitem_id IN ${IDS}`).run(params);
|
||||
st.get(`DELETE FROM queue_items WHERE qitem_id IN ${IDS}`).run(params);
|
||||
st.get(`INSERT OR IGNORE INTO queue_erased_ids (qitem_id, erased_at, reason) SELECT value, @now, @reason FROM json_each(@ids)`)
|
||||
.run({ ...params, now: nowIso, reason });
|
||||
}
|
||||
|
||||
type EventPayload = Record<string, unknown>;
|
||||
/** A payload's own JSON, or null when it doesn't parse to an object. */
|
||||
function parsePayload(text: string): EventPayload | null {
|
||||
try {
|
||||
const value: unknown = JSON.parse(text);
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value) ? value as EventPayload : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
const namedIds = (payload: EventPayload) => EVENT_ID_KEYS.map((key) => payload[key]).filter((id): id is string => typeof id === "string");
|
||||
|
||||
/** Where each target row's events are: one read-only pass over events in seq windows, up to the seq it started at.
|
||||
* Nothing changes here; each batch changes its own rows' events after rechecking the rows. A payload that names a
|
||||
* target but can't be read is listed as unreadable. */
|
||||
async function walkEvents(db: Database.Database, targets: Set<string>, window: number):
|
||||
Promise<{ seqs: Map<string, number[]>; unreadable: Array<{ seq: number; qitemIds: string[] }>; top: number }> {
|
||||
const found = { seqs: new Map<string, number[]>(), unreadable: [] as Array<{ seq: number; qitemIds: string[] }>, top: 0 };
|
||||
if (targets.size === 0) return found;
|
||||
found.top = (db.prepare("SELECT MAX(seq) AS seq FROM events").get() as { seq: number | null }).seq ?? 0;
|
||||
const read = db.prepare(`SELECT seq, payload FROM events WHERE seq > ? AND seq <= ? AND ${NAMES_A_ROW}`);
|
||||
for (let lo = 0; lo < found.top; lo += window) {
|
||||
for (const row of read.all(lo, Math.min(lo + window, found.top)) as Array<{ seq: number; payload: string }>) {
|
||||
const payload = parsePayload(row.payload);
|
||||
if (payload === null) {
|
||||
const named = [...targets].filter((id) => row.payload.includes(id));
|
||||
if (named.length > 0) found.unreadable.push({ seq: row.seq, qitemIds: named });
|
||||
continue;
|
||||
}
|
||||
for (const id of new Set(namedIds(payload))) {
|
||||
if (!targets.has(id)) continue;
|
||||
const seqs = found.seqs.get(id);
|
||||
if (seqs) seqs.push(row.seq);
|
||||
else found.seqs.set(id, [row.seq]);
|
||||
}
|
||||
}
|
||||
await yieldToLoop();
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
/** When a closed row entered its final closed state: the earliest transition after the last one recorded in another
|
||||
* state. A note on a closed row keeps its state, so it doesn't move this; a reopened and closed-again row counts
|
||||
* from its latest close. */
|
||||
function closedSince(st: Statements, qitemId: string, state: string, fallback: string): string {
|
||||
const lastOther = (st.get(`SELECT MAX(id) AS id FROM (
|
||||
SELECT transition_id AS id FROM queue_transitions WHERE qitem_id = @q AND state != @s
|
||||
UNION ALL SELECT transition_id AS id FROM queue_transitions_archive WHERE qitem_id = @q AND state != @s)`)
|
||||
.get({ q: qitemId, s: state }) as { id: number | null }).id ?? -1;
|
||||
const since = (st.get(`SELECT MIN(ts) AS ts FROM (
|
||||
SELECT ts FROM queue_transitions WHERE qitem_id = @q AND transition_id > @after
|
||||
UNION ALL SELECT ts FROM queue_transitions_archive WHERE qitem_id = @q AND transition_id > @after)`)
|
||||
.get({ q: qitemId, after: lastOther }) as { ts: string | null }).ts;
|
||||
return since ?? fallback;
|
||||
}
|
||||
|
||||
function membersDigest(members: readonly string[]): string {
|
||||
return createHash("sha256").update(JSON.stringify([...new Set(members)].sort())).digest("hex");
|
||||
}
|
||||
|
||||
type BatchPlan = {
|
||||
rows: Candidate[];
|
||||
absent: string[];
|
||||
dropped: EraseGap[];
|
||||
owned: Record<EraseAction, Copy[]>;
|
||||
kept: EraseReport["keptCopies"];
|
||||
counts: { transitions: number; archivedTransitions: number; missionControl: number };
|
||||
events: { rewrite: Array<[string, number]>; remove: number[]; seqs: number[] };
|
||||
unreadable: Array<{ seq: number; qitemIds: string[] }>;
|
||||
};
|
||||
|
||||
/** Erase queue rows in one scope. With apply false it reports exactly what an apply would erase and writes nothing. */
|
||||
export async function eraseQueue(db: Database.Database, opts: EraseOptions): Promise<EraseReport> {
|
||||
const st = new Statements(db);
|
||||
const batchSize = opts.batchSize ?? DEFAULT_BATCH;
|
||||
const maxBatches = opts.maxBatches ?? DEFAULT_MAX_BATCHES;
|
||||
const { scope } = opts;
|
||||
|
||||
// A rig named with the members must be theirs, both ways: every member's session is named in the rig (departed
|
||||
// members included), and every session of it is a member. A rig already removed is verified only by the record an
|
||||
// earlier run kept with this membership.
|
||||
let rig: { id: string; name: string | null; found: boolean; verified: boolean; completedAt: string | null } | null = null;
|
||||
if (scope.kind === "members" && scope.rig !== undefined) {
|
||||
const name = (db.prepare("SELECT name FROM rigs WHERE id = ?").get(scope.rig) as { name: string } | undefined)?.name ?? null;
|
||||
if (name !== null) {
|
||||
const outside = scope.members.filter((member) => member.split("@")[1] !== name);
|
||||
const rigSessions = (db.prepare(`SELECT DISTINCT s.session_name AS name FROM sessions s JOIN nodes n ON n.id = s.node_id
|
||||
WHERE n.rig_id = ?`).all(scope.rig) as Array<{ name: string }>).map((row) => row.name);
|
||||
const unlisted = rigSessions.filter((session) => !scope.members.includes(session));
|
||||
if (outside.length > 0 || unlisted.length > 0) {
|
||||
throw new EraseRefusal("erase_rig_mismatch", [
|
||||
outside.length > 0 ? `These members aren't sessions of rig ${name}: ${outside.join(", ")}.` : "",
|
||||
unlisted.length > 0 ? `Rig ${name} has sessions the members don't list: ${unlisted.join(", ")}.` : "",
|
||||
].filter(Boolean).join(" "));
|
||||
}
|
||||
rig = { id: scope.rig, name, found: true, verified: true, completedAt: null };
|
||||
} else {
|
||||
const kept = db.prepare(`SELECT rig_name AS name, members_digest AS digest, completed_at AS completedAt
|
||||
FROM queue_erased_rigs WHERE rig_id = ?`).get(scope.rig) as { name: string; digest: string; completedAt: string | null } | undefined;
|
||||
const verified = kept !== undefined && kept.digest === membersDigest(scope.members);
|
||||
rig = { id: scope.rig, name: verified && kept.name !== "" ? kept.name : null, found: false, verified,
|
||||
completedAt: verified ? kept.completedAt : null };
|
||||
}
|
||||
}
|
||||
|
||||
const report: EraseReport = {
|
||||
ok: true, applied: opts.apply, scope: scope.kind, complete: true, clean: false, qitems: [], mixed: [], gaps: [],
|
||||
alreadyAbsent: [], keptCopies: [], unreadableEvents: [],
|
||||
linked: { transitions: 0, archivedTransitions: 0, outbox: 0, inbox: 0, events: 0, streamActions: 0, missionControl: 0 },
|
||||
};
|
||||
if (scope.kind === "routine") report.routine = { contentDays: scope.contentDays, metadataDays: scope.metadataDays };
|
||||
const reason = scope.kind === "members" ? "account-deletion" : scope.kind === "routine" ? "retention-metadata" : "erase-by-id";
|
||||
const ledgerReason = (action: EraseAction) => scope.kind === "routine" && action === "scrub" ? "retention-content" : reason;
|
||||
const memberSet = new Set(scope.kind === "members" ? scope.members : []);
|
||||
const members = JSON.stringify([...memberSet]);
|
||||
const entries = scope.kind === "ids"
|
||||
? [...new Map(scope.entries.map((entry) => [entry.qitemId, entry.action])).entries()].sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0))
|
||||
: [];
|
||||
const contentCutoff = scope.kind === "routine" && scope.contentDays > 0 ? cutoffIso(opts.nowIso, scope.contentDays) : null;
|
||||
const metadataCutoff = scope.kind === "routine" && scope.metadataDays > 0 ? cutoffIso(opts.nowIso, scope.metadataDays) : null;
|
||||
|
||||
// What routine retention does with a row now: delete it past the row window, scrub it past the content window while
|
||||
// it has content, or nothing. A scrubbed row written to since (a note appends a transition without touching
|
||||
// ts_updated, and may since have been archived; a reclose moves both) has content again.
|
||||
const routineAction = (row: Current): EraseAction | null => {
|
||||
if (!CLOSED_QUEUE_STATES.includes(row.state as (typeof CLOSED_QUEUE_STATES)[number])) return null;
|
||||
const since = closedSince(st, row.qitemId, row.state, row.tsUpdated);
|
||||
if (metadataCutoff && since < metadataCutoff) return "delete";
|
||||
if (!contentCutoff || since >= contentCutoff) return null;
|
||||
const hasContent = !row.erasedAt || row.tsUpdated > row.erasedAt
|
||||
|| st.get(`SELECT 1 FROM queue_transitions WHERE qitem_id = @q AND ts > @since
|
||||
UNION ALL SELECT 1 FROM queue_transitions_archive WHERE qitem_id = @q AND ts > @since LIMIT 1`)
|
||||
.get({ q: row.qitemId, since: row.erasedAt }) !== undefined;
|
||||
return hasContent ? "scrub" : null;
|
||||
};
|
||||
|
||||
// Ids scope, delete: ids with no row at selection; their markers are written at the end.
|
||||
const absentAtSelect: string[] = [];
|
||||
// One page of candidate rows, keyed on qitem_id so selection moves forward without writing anything.
|
||||
const page = (after: string, offset: number): { rows: Candidate[]; last: string | null; seen: number } => {
|
||||
const select = "qitem_id AS qitemId, state, source_session AS source, destination_session AS destination";
|
||||
if (scope.kind === "ids") {
|
||||
const slice = entries.slice(offset, offset + batchSize);
|
||||
const found = new Map((st.get(`SELECT ${select} FROM queue_items WHERE qitem_id IN ${IDS}`)
|
||||
.all({ ids: JSON.stringify(slice.map(([id]) => id)) }) as Array<Omit<Candidate, "action">>).map((row) => [row.qitemId, row]));
|
||||
const rows: Candidate[] = [];
|
||||
for (const [id, action] of slice) {
|
||||
const row = found.get(id);
|
||||
if (row !== undefined) rows.push({ ...row, action });
|
||||
else if (action === "delete") absentAtSelect.push(id);
|
||||
else report.gaps.push({ code: "not-found", qitemId: id });
|
||||
}
|
||||
return { rows, last: null, seen: slice.length };
|
||||
}
|
||||
if (scope.kind === "members") {
|
||||
const found = st.get(`SELECT ${select} FROM queue_items
|
||||
WHERE qitem_id > @after AND (source_session IN ${MEMBERS} OR destination_session IN ${MEMBERS})
|
||||
ORDER BY qitem_id LIMIT @limit`).all({ members, after, limit: batchSize }) as Array<Omit<Candidate, "action">>;
|
||||
const rows: Candidate[] = [];
|
||||
for (const row of found) {
|
||||
const fromMember = memberSet.has(row.source);
|
||||
const toMember = memberSet.has(row.destination);
|
||||
if (fromMember && toMember) rows.push({ ...row, action: "delete" });
|
||||
else report.mixed.push({ qitemId: row.qitemId, memberEndpoint: fromMember ? "source" : "destination" });
|
||||
}
|
||||
return { rows, last: found.at(-1)?.qitemId ?? null, seen: found.length };
|
||||
}
|
||||
// +state keeps the planner on the primary key, so a page reads forward from @after; the state index would sort
|
||||
// every closed row on each page (1.4 s a page on a 44k-row store).
|
||||
const closed = CLOSED_QUEUE_STATES.map((state) => `'${state}'`).join(", ");
|
||||
const found = st.get(`SELECT ${select}, ts_updated AS tsUpdated, content_erased_at AS erasedAt
|
||||
FROM queue_items WHERE qitem_id > @after AND +state IN (${closed}) ORDER BY qitem_id LIMIT @limit`)
|
||||
.all({ after, limit: batchSize }) as Array<Omit<Candidate, "action"> & { tsUpdated: string; erasedAt: string | null }>;
|
||||
const rows: Candidate[] = [];
|
||||
for (const { tsUpdated, erasedAt, ...row } of found) {
|
||||
const action = routineAction({ ...row, tsUpdated, erasedAt });
|
||||
if (action !== null) rows.push({ ...row, action });
|
||||
}
|
||||
return { rows, last: found.at(-1)?.qitemId ?? null, seen: found.length };
|
||||
};
|
||||
|
||||
// Whose copies these are: the account's (members scope), or the row's own participants (other scopes).
|
||||
const ownerOf = (candidate: Candidate) => scope.kind === "members" ? memberSet : new Set([candidate.source, candidate.destination]);
|
||||
const ownership = (copy: Copy, owners: Set<string>): "owned" | "mixed" | "foreign" => {
|
||||
const sessions = copy.sessions.filter((session): session is string => session !== null);
|
||||
const owned = sessions.filter((session) => owners.has(session)).length;
|
||||
return owned === sessions.length ? "owned" : owned > 0 ? "mixed" : "foreign";
|
||||
};
|
||||
// Stream actions and dispositions this run deletes, so the unlinked pass plans on the same references either way.
|
||||
const plannedActions: string[] = [];
|
||||
const plannedDispositions: string[] = [];
|
||||
const eventSeqs = new Set<number>();
|
||||
const unreadable = new Map<number, Set<string>>();
|
||||
const noteUnreadable = (found: Array<{ seq: number; qitemIds: string[] }>) => {
|
||||
for (const { seq, qitemIds } of found) {
|
||||
const ids = unreadable.get(seq) ?? new Set<string>();
|
||||
for (const id of qitemIds) ids.add(id);
|
||||
unreadable.set(seq, ids);
|
||||
}
|
||||
report.unreadableEvents = [...unreadable.entries()].sort(([a], [b]) => a - b).map(([seq, ids]) => ({ seq, qitemIds: [...ids].sort() }));
|
||||
};
|
||||
const ledger = (rows: Array<{ qitemId: string; action: EraseAction }>) => {
|
||||
if (!opts.apply || !opts.ledgerPath || rows.length === 0) return;
|
||||
appendFileSync(opts.ledgerPath, rows.map((row) => JSON.stringify({ qitemId: row.qitemId, action: row.action,
|
||||
reason: ledgerReason(row.action), erasedAt: opts.nowIso })).join("\n") + "\n");
|
||||
};
|
||||
|
||||
let stage: Stage = "ledger";
|
||||
try {
|
||||
// 0. A ledger that can't be written stops the run before anything changes.
|
||||
if (opts.apply && opts.ledgerPath) appendFileSync(opts.ledgerPath, "");
|
||||
|
||||
// 1. Select every candidate, writing nothing.
|
||||
stage = "select";
|
||||
const candidates: Candidate[] = [];
|
||||
let after = "";
|
||||
let offset = 0;
|
||||
for (let batch = 0; ; batch += 1) {
|
||||
if (batch >= maxBatches) {
|
||||
report.complete = false;
|
||||
break;
|
||||
}
|
||||
const { rows, last, seen } = page(after, offset);
|
||||
if (seen === 0) break;
|
||||
const referenced = workflowReferenced(st, rows.map((row) => row.qitemId));
|
||||
for (const id of referenced) report.gaps.push({ code: "workflow-reference", qitemId: id });
|
||||
candidates.push(...rows.filter((row) => !referenced.has(row.qitemId)));
|
||||
if (last !== null) after = last;
|
||||
offset += seen;
|
||||
if (seen < batchSize) break;
|
||||
await yieldToLoop();
|
||||
}
|
||||
|
||||
// 2. Where the candidates' events are: one read-only walk over the table.
|
||||
stage = "events";
|
||||
const events = await walkEvents(db, new Set(candidates.map((row) => row.qitemId)), opts.eventsWindow ?? DEFAULT_EVENTS_WINDOW);
|
||||
noteUnreadable(events.unreadable);
|
||||
const eventsAt = st.get(`SELECT seq, payload FROM events WHERE seq IN (SELECT value FROM json_each(@seqs))
|
||||
UNION ALL SELECT seq, payload FROM events WHERE seq > @top AND ${NAMES_A_ROW}`);
|
||||
const rewriteEvent = db.prepare("UPDATE events SET payload = ? WHERE seq = ?");
|
||||
const removeEvent = db.prepare("DELETE FROM events WHERE seq = ?");
|
||||
|
||||
// One batch, read at the moment it's erased: each row is rechecked, since anything may have changed it after
|
||||
// selection (a reopen makes it open work again; a workflow may have taken it up), then its copies and events.
|
||||
const unreadableAtWalk = new Set(events.unreadable.flatMap((event) => event.qitemIds));
|
||||
const planBatch = (chunk: Candidate[]): BatchPlan => {
|
||||
const chunkIds = chunk.map((row) => row.qitemId);
|
||||
const current = new Map((st.get(`SELECT qitem_id AS qitemId, state, ts_updated AS tsUpdated, content_erased_at AS erasedAt,
|
||||
source_session AS source, destination_session AS destination FROM queue_items WHERE qitem_id IN ${IDS}`)
|
||||
.all({ ids: JSON.stringify(chunkIds) }) as Current[]).map((row) => [row.qitemId, row]));
|
||||
const referenced = workflowReferenced(st, chunkIds);
|
||||
const plan: BatchPlan = { rows: [], absent: [], dropped: [], owned: { scrub: [], delete: [] }, kept: [],
|
||||
counts: { transitions: 0, archivedTransitions: 0, missionControl: 0 }, events: { rewrite: [], remove: [], seqs: [] }, unreadable: [] };
|
||||
for (const candidate of chunk) {
|
||||
const now = current.get(candidate.qitemId);
|
||||
if (now === undefined) {
|
||||
// Gone since selection: a delete still writes its marker; a scrub has nothing to scrub.
|
||||
if (candidate.action === "delete") plan.absent.push(candidate.qitemId);
|
||||
else plan.dropped.push({ code: "changed-during-run", qitemId: candidate.qitemId });
|
||||
} else if (referenced.has(candidate.qitemId)) {
|
||||
plan.dropped.push({ code: "workflow-reference", qitemId: candidate.qitemId });
|
||||
} else if (scope.kind === "members" && !(memberSet.has(now.source) && memberSet.has(now.destination))) {
|
||||
// Rerouted since selection (a fallback, say): no longer the account's own row.
|
||||
plan.dropped.push({ code: "changed-during-run", qitemId: candidate.qitemId });
|
||||
} else if (scope.kind === "routine" && routineAction(now) !== candidate.action) {
|
||||
plan.dropped.push({ code: "changed-during-run", qitemId: candidate.qitemId });
|
||||
} else {
|
||||
// The current endpoints decide whose the row's copies are.
|
||||
plan.rows.push({ ...candidate, state: now.state, source: now.source, destination: now.destination });
|
||||
}
|
||||
}
|
||||
if (plan.rows.length === 0) return plan;
|
||||
// The walk's events for these rows, plus any written since it began. A row an unreadable event names is kept:
|
||||
// what that event holds can't be inspected, and keeping the row means every rerun finds the event and reports
|
||||
// it until it's resolved.
|
||||
const seqs = [...new Set(plan.rows.flatMap((row) => events.seqs.get(row.qitemId) ?? []))];
|
||||
const fetched = new Map<number, string>();
|
||||
for (const event of eventsAt.all({ seqs: JSON.stringify(seqs), top: events.top }) as Array<{ seq: number; payload: string }>) {
|
||||
fetched.set(event.seq, event.payload);
|
||||
}
|
||||
const blocked = new Set(plan.rows.map((row) => row.qitemId).filter((id) => unreadableAtWalk.has(id)));
|
||||
const readable: Array<[number, EventPayload]> = [];
|
||||
for (const [seq, text] of fetched) {
|
||||
const payload = parsePayload(text);
|
||||
if (payload !== null) {
|
||||
readable.push([seq, payload]);
|
||||
continue;
|
||||
}
|
||||
const named = plan.rows.map((row) => row.qitemId).filter((id) => text.includes(id));
|
||||
if (named.length === 0) continue;
|
||||
plan.unreadable.push({ seq, qitemIds: named });
|
||||
for (const id of named) blocked.add(id);
|
||||
}
|
||||
if (blocked.size > 0) {
|
||||
plan.dropped.push(...[...blocked].sort().map((qitemId) => ({ code: "unreadable-event" as const, qitemId })));
|
||||
plan.rows = plan.rows.filter((row) => !blocked.has(row.qitemId));
|
||||
}
|
||||
const ids = plan.rows.map((row) => row.qitemId);
|
||||
if (ids.length === 0) return plan;
|
||||
const byId = new Map(plan.rows.map((row) => [row.qitemId, row]));
|
||||
for (const copy of linkedCopies(st, ids)) {
|
||||
const candidate = byId.get(copy.qitemId)!;
|
||||
const whose = ownership(copy, ownerOf(candidate));
|
||||
if (whose === "owned") plan.owned[candidate.action].push(copy);
|
||||
else plan.kept.push({ table: copy.table, id: copy.id, qitemId: copy.qitemId, ownership: whose });
|
||||
}
|
||||
const params = { ids: JSON.stringify(ids) };
|
||||
plan.counts = {
|
||||
transitions: st.count(`SELECT COUNT(*) AS n FROM queue_transitions WHERE qitem_id IN ${IDS}`, params),
|
||||
archivedTransitions: st.count(`SELECT COUNT(*) AS n FROM queue_transitions_archive WHERE qitem_id IN ${IDS}`, params),
|
||||
missionControl: st.count(`SELECT COUNT(*) AS n FROM mission_control_actions WHERE qitem_id IN ${IDS}`, params),
|
||||
};
|
||||
// An event naming two rows is deleted if either is deleted; one naming rows of a later batch is changed again
|
||||
// there.
|
||||
for (const [seq, payload] of readable) {
|
||||
const actions = namedIds(payload).map((id) => byId.get(id)?.action).filter((action): action is EraseAction => action !== undefined);
|
||||
if (actions.length === 0) continue;
|
||||
plan.events.seqs.push(seq);
|
||||
if (actions.includes("delete")) {
|
||||
plan.events.remove.push(seq);
|
||||
} else {
|
||||
for (const key of EVENT_CONTENT_KEYS) delete payload[key];
|
||||
if (!keepsClosureTarget(payload.closureReason, payload.closureTarget)) delete payload.closureTarget;
|
||||
payload.contentErased = true;
|
||||
plan.events.rewrite.push([JSON.stringify(payload), seq]);
|
||||
}
|
||||
}
|
||||
return plan;
|
||||
};
|
||||
|
||||
// 3. The rows, their events and their other copies, a batch per transaction. A row is listed once its batch has
|
||||
// committed; its ledger lines are written in the transaction, before anything in it changes.
|
||||
stage = "rows";
|
||||
const perTransaction = Math.min(batchSize, ROWS_PER_TRANSACTION);
|
||||
for (let start = 0; start < candidates.length; start += perTransaction) {
|
||||
const chunk = candidates.slice(start, start + perTransaction);
|
||||
const plan = opts.apply
|
||||
? db.transaction(() => {
|
||||
const planned = planBatch(chunk);
|
||||
ledger([...planned.rows, ...planned.absent.map((qitemId) => ({ qitemId, action: "delete" as const }))]);
|
||||
for (const [payload, seq] of planned.events.rewrite) rewriteEvent.run(payload, seq);
|
||||
for (const seq of planned.events.remove) removeEvent.run(seq);
|
||||
eraseCopies(st, planned.owned.scrub, "scrub");
|
||||
eraseCopies(st, planned.owned.delete, "delete");
|
||||
const scrub = planned.rows.filter((row) => row.action === "scrub").map((row) => row.qitemId);
|
||||
const remove = [...planned.rows.filter((row) => row.action === "delete").map((row) => row.qitemId), ...planned.absent];
|
||||
if (scrub.length > 0) scrubRows(st, scrub, opts.nowIso);
|
||||
if (remove.length > 0) deleteRows(st, remove, opts.nowIso, reason);
|
||||
return planned;
|
||||
})()
|
||||
: planBatch(chunk);
|
||||
const all = [...plan.owned.scrub, ...plan.owned.delete];
|
||||
report.linked.transitions += plan.counts.transitions;
|
||||
report.linked.archivedTransitions += plan.counts.archivedTransitions;
|
||||
report.linked.missionControl += plan.counts.missionControl;
|
||||
report.linked.outbox += all.filter((copy) => copy.table === "outbox_entries").length;
|
||||
report.linked.inbox += all.filter((copy) => copy.table === "inbox_entries").length;
|
||||
report.linked.streamActions += all.filter((copy) => copy.table === "stream_reader_actions" || copy.table === "stream_dispositions").length;
|
||||
for (const seq of plan.events.seqs) eventSeqs.add(seq);
|
||||
report.linked.events = eventSeqs.size;
|
||||
noteUnreadable(plan.unreadable);
|
||||
plannedActions.push(...plan.owned.delete.filter((copy) => copy.table === "stream_reader_actions").map((copy) => copy.id));
|
||||
plannedDispositions.push(...plan.owned.delete.filter((copy) => copy.table === "stream_dispositions").map((copy) => copy.id));
|
||||
report.keptCopies.push(...plan.kept);
|
||||
report.gaps.push(...plan.dropped);
|
||||
report.alreadyAbsent.push(...plan.absent);
|
||||
report.qitems.push(...plan.rows.map(({ qitemId, action, state }) => ({ qitemId, action, state })));
|
||||
await yieldToLoop();
|
||||
}
|
||||
if (absentAtSelect.length > 0) {
|
||||
// Rechecked as the markers are written: an id created since selection is live work, never certified absent.
|
||||
const certify = () => {
|
||||
const present = new Set((st.get(`SELECT qitem_id AS id FROM queue_items WHERE qitem_id IN ${IDS}`)
|
||||
.all({ ids: JSON.stringify(absentAtSelect) }) as Array<{ id: string }>).map((row) => row.id));
|
||||
const absent = absentAtSelect.filter((id) => !present.has(id));
|
||||
if (opts.apply && absent.length > 0) {
|
||||
ledger(absent.map((qitemId) => ({ qitemId, action: "delete" as const })));
|
||||
st.get(`INSERT OR IGNORE INTO queue_erased_ids (qitem_id, erased_at, reason) SELECT value, @now, @reason FROM json_each(@ids)`)
|
||||
.run({ ids: JSON.stringify(absent), now: opts.nowIso, reason });
|
||||
}
|
||||
return { absent, created: absentAtSelect.filter((id) => present.has(id)) };
|
||||
};
|
||||
const { absent, created } = opts.apply ? db.transaction(certify)() : certify();
|
||||
report.gaps.push(...created.map((qitemId) => ({ code: "changed-during-run" as const, qitemId })));
|
||||
report.alreadyAbsent.push(...absent);
|
||||
}
|
||||
|
||||
// 4. Account deletion's other passes, once every row is done.
|
||||
if (scope.kind === "members" && report.complete) {
|
||||
stage = "unlinked";
|
||||
report.unlinked = eraseUnlinked(db, st, {
|
||||
members, plannedActions, plannedDispositions, erasedRows: report.qitems.map((row) => row.qitemId),
|
||||
}, opts.apply);
|
||||
if (rig !== null) {
|
||||
stage = "rig";
|
||||
report.rig = { id: rig.id, found: rig.found, verified: rig.verified, erased: {}, left: {}, unknown: [] };
|
||||
await eraseRigRecords(db, st, { ...rig, members, digest: membersDigest(scope.members) }, opts.apply, opts.nowIso, report.rig);
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
// Everything listed so far was committed; the caller keeps those ids, and a rerun continues.
|
||||
report.complete = false;
|
||||
report.error = { stage, message: err instanceof Error ? err.message : String(err) };
|
||||
}
|
||||
|
||||
const unlinkedLeft = report.unlinked ? report.unlinked.mixedEntries + report.unlinked.referencedStreamItems : 0;
|
||||
const rigLeft = report.rig ? Object.values(report.rig.left).reduce((sum, n) => sum + n, 0) + report.rig.unknown.length : 0;
|
||||
report.clean = report.complete && report.mixed.length === 0 && report.gaps.length === 0 && unlinkedLeft === 0 && rigLeft === 0
|
||||
&& report.unreadableEvents.length === 0 && !report.keptCopies.some((copy) => copy.ownership === "mixed");
|
||||
return report;
|
||||
}
|
||||
|
||||
/** Account deletion's second pass: outbox and inbox entries between two members that no queue row owns, and the
|
||||
* members' stream items. An entry with one member endpoint, or a stream item something still references, is counted
|
||||
* and left. Rows and references this run erases are counted as they were before it, so a dry run and an apply report
|
||||
* the same set. */
|
||||
function eraseUnlinked(db: Database.Database, st: Statements,
|
||||
plan: { members: string; plannedActions: string[]; plannedDispositions: string[]; erasedRows: string[] }, apply: boolean): NonNullable<EraseReport["unlinked"]> {
|
||||
// A row this run erased still owns its copies here: they were reported with it (keptCopies), not as unowned.
|
||||
const rows = "(SELECT qitem_id FROM queue_items UNION ALL SELECT value FROM json_each(@rows))";
|
||||
// COALESCE: an entry with no pointer is unowned, not unknown (NULL IN (...) would drop it from both counts).
|
||||
const ownedOutbox = `(COALESCE(audit_pointer, '') IN ${rows}
|
||||
OR (outbox_id LIKE 'wake-intent-%' AND substr(outbox_id, 13) IN ${rows}))`;
|
||||
const ownedInbox = `(COALESCE(absorbed_qitem_id, '') IN ${rows} OR COALESCE(audit_pointer, '') IN ${rows})`;
|
||||
const both = `sender_session IN ${MEMBERS} AND destination_session IN ${MEMBERS}`;
|
||||
const fromMember = `sender_session IN ${MEMBERS} AND destination_session NOT IN ${MEMBERS}`;
|
||||
const toMember = `destination_session IN ${MEMBERS} AND sender_session NOT IN ${MEMBERS}`;
|
||||
// The same references the stream pass keeps an item for (queue-retention pruneStreamItems), less this run's own.
|
||||
const classificationColumns = new Set((db.prepare("PRAGMA table_info(project_classifications)").all() as Array<{ name: string }>)
|
||||
.map((column) => column.name));
|
||||
const notPlannedAction = "action_id NOT IN (SELECT value FROM json_each(@actions))";
|
||||
const references = [
|
||||
"SELECT stream_item_id FROM project_classifications WHERE stream_item_id IS NOT NULL",
|
||||
...(classificationColumns.has("duplicate_of_stream_item_id")
|
||||
? ["SELECT duplicate_of_stream_item_id FROM project_classifications WHERE duplicate_of_stream_item_id IS NOT NULL"] : []),
|
||||
"SELECT stream_item_id FROM classification_attempts WHERE stream_item_id IS NOT NULL",
|
||||
"SELECT stream_item_id FROM stream_dispositions WHERE stream_item_id IS NOT NULL AND stream_item_id NOT IN (SELECT value FROM json_each(@dispositions))",
|
||||
`SELECT stream_item_id FROM stream_reader_actions WHERE stream_item_id IS NOT NULL AND ${notPlannedAction}`,
|
||||
`SELECT duplicate_of FROM stream_reader_actions WHERE duplicate_of IS NOT NULL AND ${notPlannedAction}`,
|
||||
];
|
||||
const referenced = `stream_item_id IN (${references.join(" UNION ")})`;
|
||||
const params = { members: plan.members, actions: JSON.stringify(plan.plannedActions), dispositions: JSON.stringify(plan.plannedDispositions),
|
||||
rows: JSON.stringify(plan.erasedRows) };
|
||||
const result = {
|
||||
outbox: st.count(`SELECT COUNT(*) AS n FROM outbox_entries WHERE ${both} AND NOT ${ownedOutbox}`, params),
|
||||
inbox: st.count(`SELECT COUNT(*) AS n FROM inbox_entries WHERE ${both} AND NOT ${ownedInbox}`, params),
|
||||
streamItems: st.count(`SELECT COUNT(*) AS n FROM stream_items WHERE source_session IN ${MEMBERS} AND NOT ${referenced}`, params),
|
||||
// A row's own copies are reported with that row (mixed, keptCopies); these are the entries no row owns.
|
||||
mixedEntries: [fromMember, toMember].reduce((sum, where) =>
|
||||
sum + st.count(`SELECT COUNT(*) AS n FROM outbox_entries WHERE ${where} AND NOT ${ownedOutbox}`, params)
|
||||
+ st.count(`SELECT COUNT(*) AS n FROM inbox_entries WHERE ${where} AND NOT ${ownedInbox}`, params), 0),
|
||||
referencedStreamItems: st.count(`SELECT COUNT(*) AS n FROM stream_items WHERE source_session IN ${MEMBERS} AND ${referenced}`, params),
|
||||
};
|
||||
if (apply) {
|
||||
db.transaction(() => {
|
||||
st.get(`DELETE FROM outbox_entries WHERE ${both} AND NOT ${ownedOutbox}`).run(params);
|
||||
st.get(`DELETE FROM inbox_entries WHERE ${both} AND NOT ${ownedInbox}`).run(params);
|
||||
st.get(`DELETE FROM stream_items WHERE source_session IN ${MEMBERS} AND NOT ${referenced}`).run(params);
|
||||
})();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// The account rig's own records that outlive the rig (no cascading key to it), in delete order: dependents first.
|
||||
// byRigId: keyed by the rig's id alone, so erased only for a verified id. NODES: the rig's nodes, while it has them.
|
||||
const NODES = "(SELECT id FROM nodes WHERE rig_id = @rig)";
|
||||
const RIG_RECORDS: Array<{ table: string; where: string; byRigId: boolean }> = [
|
||||
{ table: "events", where: "rig_id = @rig", byRigId: true },
|
||||
{ table: "events", where: `node_id IN ${NODES}`, byRigId: false },
|
||||
{ table: "snapshots", where: "rig_id = @rig", byRigId: true },
|
||||
{ table: "operation_correlations", where: "rig_id = @rig", byRigId: true },
|
||||
{ table: "bootstrap_actions", where: "bootstrap_id IN (SELECT id FROM bootstrap_runs WHERE rig_id = @rig)", byRigId: true },
|
||||
{ table: "bootstrap_runs", where: "rig_id = @rig", byRigId: true },
|
||||
{ table: "seat_identity_verdicts", where: `node_id IN ${NODES} OR session_name IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "seat_delivery_guard_changes", where: `node_id IN ${NODES}`, byRigId: false },
|
||||
{ table: "environment_operations",
|
||||
where: `binding_id IN (SELECT binding_id FROM environment_bindings WHERE node_id IN ${NODES} OR seat IN ${MEMBERS})`, byRigId: false },
|
||||
{ table: "environment_bindings", where: `node_id IN ${NODES} OR seat IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "usage_samples", where: `node_id IN ${NODES} OR seat_session IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "pi_usage_entries", where: `node_id IN ${NODES} OR seat_session IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "pi_usage_charges", where: `seat_session IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "watchdog_history",
|
||||
where: `delivery_target_session IN ${MEMBERS} OR job_id IN (SELECT job_id FROM watchdog_jobs WHERE target_session IN ${MEMBERS})`, byRigId: false },
|
||||
{ table: "watchdog_jobs", where: `target_session IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "stream_subscriptions", where: `reader_session IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "thread_seat_map", where: `seat IN ${MEMBERS}`, byRigId: false },
|
||||
{ table: "thread_part_map", where: `seat IN ${MEMBERS}`, byRigId: false },
|
||||
];
|
||||
// Records that name the account but also belong to someone or something else: counted and left. byName: matched by
|
||||
// the rig's name, which an unverified removed rig doesn't have, so those are unknown rather than zero.
|
||||
const RIG_RECORDS_LEFT: Array<{ table: string; where: string; byName: boolean }> = [
|
||||
{ table: "watchdog_jobs", where: `registered_by_session IN ${MEMBERS} AND target_session NOT IN ${MEMBERS}`, byName: false },
|
||||
{ table: "workflow_instances", where: `created_by_session IN ${MEMBERS}`, byName: false },
|
||||
{ table: "workflow_instances", where: "bound_rig = @rigName", byName: true },
|
||||
{ table: "workflow_specs", where: "target_rig = @rigName", byName: true },
|
||||
{ table: "classifier_leases", where: `classifier_session IN ${MEMBERS} OR reclaimed_by_session IN ${MEMBERS}`, byName: false },
|
||||
{ table: "views_custom", where: `registered_by_session IN ${MEMBERS}`, byName: false },
|
||||
];
|
||||
const RIG_RECORDS_CHUNK = 1_000;
|
||||
|
||||
/** The account rig's records, counted into `out` as each chunk commits, so a pass that stops partway still reports
|
||||
* what it erased. */
|
||||
async function eraseRigRecords(db: Database.Database, st: Statements,
|
||||
rig: { id: string; name: string | null; found: boolean; verified: boolean; completedAt: string | null; members: string; digest: string },
|
||||
apply: boolean, nowIso: string, out: NonNullable<EraseReport["rig"]>): Promise<void> {
|
||||
const tables = new Set((db.prepare("SELECT name FROM sqlite_master WHERE type = 'table'").all() as Array<{ name: string }>)
|
||||
.map((row) => row.name));
|
||||
const params = { rig: rig.id, rigName: rig.name ?? "", members: rig.members };
|
||||
const add = (into: Record<string, number>, table: string, n: number) => {
|
||||
into[table] = (into[table] ?? 0) + n;
|
||||
};
|
||||
for (const { table, where, byRigId } of RIG_RECORDS) {
|
||||
if (!tables.has(table)) continue;
|
||||
if (byRigId && !rig.verified) {
|
||||
// An unverified id: its records are counted and left, never erased on a guess.
|
||||
add(out.left, table, st.count(`SELECT COUNT(*) AS n FROM ${table} WHERE ${where}`, params));
|
||||
continue;
|
||||
}
|
||||
if (!apply) {
|
||||
add(out.erased, table, st.count(`SELECT COUNT(*) AS n FROM ${table} WHERE ${where}`, params));
|
||||
continue;
|
||||
}
|
||||
add(out.erased, table, 0);
|
||||
// In chunks by rowid (every one of these tables has one; a declared key can be NULL), so a large table (events)
|
||||
// never holds the daemon in one statement.
|
||||
const chunk = st.get(`DELETE FROM ${table} WHERE rowid IN (SELECT rowid FROM ${table} WHERE ${where} LIMIT ${RIG_RECORDS_CHUNK})`);
|
||||
for (let changes = chunk.run(params).changes; changes > 0; changes = chunk.run(params).changes) {
|
||||
add(out.erased, table, changes);
|
||||
await yieldToLoop();
|
||||
}
|
||||
}
|
||||
const unknown = new Set<string>();
|
||||
for (const { table, where, byName } of RIG_RECORDS_LEFT) {
|
||||
if (!tables.has(table)) continue;
|
||||
if (byName && rig.name === null) {
|
||||
// No name to count by. After a completed erasure that's expected: these were empty when it completed.
|
||||
if (rig.completedAt === null) unknown.add(table);
|
||||
} else {
|
||||
add(out.left, table, st.count(`SELECT COUNT(*) AS n FROM ${table} WHERE ${where}`, params));
|
||||
}
|
||||
}
|
||||
out.unknown = [...unknown];
|
||||
const leftTotal = Object.values(out.left).reduce((sum, n) => sum + n, 0);
|
||||
if (rig.completedAt !== null) out.completedAt = rig.completedAt;
|
||||
if (apply && rig.verified) {
|
||||
if (rig.found) {
|
||||
// While the rig exists, its id and name are kept with this membership, so the run after its removal is verified.
|
||||
db.prepare(`INSERT INTO queue_erased_rigs (rig_id, rig_name, members_digest, verified_at, completed_at) VALUES (?, ?, ?, ?, NULL)
|
||||
ON CONFLICT(rig_id) DO UPDATE SET rig_name = excluded.rig_name, members_digest = excluded.members_digest,
|
||||
verified_at = excluded.verified_at, completed_at = NULL`).run(rig.id, rig.name, rig.digest, nowIso);
|
||||
} else if (leftTotal === 0 && unknown.size === 0 && rig.completedAt === null) {
|
||||
// After the removal, once nothing of the rig is left, the record becomes a completion receipt: the name goes,
|
||||
// and the id, membership digest and time stay, so an identical rerun (say, after a lost response) still
|
||||
// verifies and reports the same clean result.
|
||||
db.prepare("UPDATE queue_erased_rigs SET rig_name = '', completed_at = ? WHERE rig_id = ?").run(nowIso, rig.id);
|
||||
out.completedAt = nowIso;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -66,7 +66,7 @@ import { physicalSessionName } from "./session-registry.js";
|
||||
import type { QueueWakeFacts } from "./draft-aware-delivery.js";
|
||||
import { derivePickup, type PickupReceipt } from "./queue-pickup.js";
|
||||
import { withRecoveryIndex } from "./queue-recovery.js";
|
||||
import { lastMeaningfulTransitionMany, meaningfulFor, readWaitingView, WaitingStatements, type WaitingView, type WaitingActivityReader, type WaitingReads } from "./queue-waiting.js";
|
||||
import { lastMeaningfulTransition, readWaitingView, type WaitingView, type WaitingActivityReader } from "./queue-waiting.js";
|
||||
import { isOpenRigSender, OPENRIG_SENDER, wrapPaneEnvelope } from "../lib/pane-envelope.js";
|
||||
import { isBoundTerminalSeat } from "./terminal-seat.js";
|
||||
import { renderQueueHandoffNudge } from "./queue-nudge-text.js";
|
||||
@@ -308,6 +308,9 @@ export interface QueueItem {
|
||||
* (convention C3). NULL for all non-human-routed items (BR-1); required
|
||||
* at the domain write path only when the §5 predicate is true. */
|
||||
evidenceRef: string | null;
|
||||
/** Set only on a row whose content queue erasure removed (queue-erasure.ts): its body and other content are empty
|
||||
* because they were erased, not because the row was written that way. */
|
||||
contentErasedAt?: string;
|
||||
/** Present only on compact list rows, which omit these fields entirely so omitted content
|
||||
* can't be mistaken for an empty value. Full reads never carry this marker. */
|
||||
fieldsElided?: Array<(typeof COMPACT_ELIDED_FIELDS)[number]>;
|
||||
@@ -363,6 +366,7 @@ interface QueueItemRow {
|
||||
delegated_decision?: string | null;
|
||||
summary: string | null;
|
||||
evidence_ref: string | null;
|
||||
content_erased_at?: string | null;
|
||||
closure_reason: string | null;
|
||||
closure_target: string | null;
|
||||
closure_required_at: string | null;
|
||||
@@ -1075,6 +1079,7 @@ export class QueueRepository {
|
||||
private readonly hasMintingGenColumn: boolean;
|
||||
private readonly hasClaimedGenColumn: boolean;
|
||||
private readonly hasQueueTransitionsTable: boolean;
|
||||
private readonly hasErasedIdsTable: boolean;
|
||||
private readonly hasTransitionProvenanceColumn: boolean;
|
||||
private readonly hasOwnerNotificationColumns: boolean;
|
||||
private readonly loadHumanRegistryFn: () => LoadResult;
|
||||
@@ -1157,6 +1162,7 @@ export class QueueRepository {
|
||||
this.hasDelegatedDecisionColumn = detectQueueColumn(db, "delegated_decision");
|
||||
this.hasEvidenceRefColumn = detectQueueColumn(db, "evidence_ref");
|
||||
this.hasQueueTransitionsTable = detectTable(db, "queue_transitions");
|
||||
this.hasErasedIdsTable = detectTable(db, "queue_erased_ids");
|
||||
const transitionColumns = this.hasQueueTransitionsTable
|
||||
? new Set((db.prepare("PRAGMA table_info(queue_transitions)").all() as Array<{ name: string }>).map((column) => column.name))
|
||||
: new Set<string>();
|
||||
@@ -2274,6 +2280,11 @@ export class QueueRepository {
|
||||
}
|
||||
this.assertHumanAlert(input.humanAlert, input.destinationSession, input.humanIntent);
|
||||
const id = input.qitemId ?? newQitemId();
|
||||
// A row queue erasure deleted keeps only its id: a create that reuses it would bring erased work back.
|
||||
if (input.qitemId && this.hasErasedIdsTable
|
||||
&& this.db.prepare("SELECT 1 FROM queue_erased_ids WHERE qitem_id = ?").get(input.qitemId)) {
|
||||
throw new QueueRepositoryError("qitem_erased", `qitem ${input.qitemId} was erased; its id can't be used again.`);
|
||||
}
|
||||
const ts = new Date().toISOString();
|
||||
const priority = input.priority ?? "routine";
|
||||
const tier = input.tier ?? null;
|
||||
@@ -4432,12 +4443,14 @@ export class QueueRepository {
|
||||
.prepare("SELECT * FROM queue_items WHERE qitem_id = ?")
|
||||
.get(qitemId) as QueueItemRow | undefined;
|
||||
if (!row) return null;
|
||||
const item = this.rowToItem(row);
|
||||
// OPR.0.5.6.14 — the row FACE answers "did it reach them" in one read for
|
||||
// gateway-routed rows; null for pane-bound (absence-governed, no key lies).
|
||||
// The same projection as a listing's, for one row: one read of its owner notifications and ledger.
|
||||
const item = this.projectRows([row], true)[0]!;
|
||||
const ledger = this.deliveryOutcomeFor(item.qitemId);
|
||||
return {
|
||||
...item,
|
||||
deliveryOutcome: ledger?.outcome ?? null,
|
||||
...(ledger && ledger.outcome !== "posted" && ledger.outcome !== "accepted" ? { deliveryFailureDetail: ledger.detail } : {}),
|
||||
...(item.replyTo ? { replyToFallback: this.replyToFallbackFor(item.qitemId) } : {}),
|
||||
};
|
||||
}
|
||||
@@ -4582,25 +4595,18 @@ export class QueueRepository {
|
||||
`SELECT ${columns} FROM queue_items ${where} ORDER BY ${orderBy} LIMIT ?`
|
||||
)
|
||||
.all(...params) as QueueItemRow[];
|
||||
const items = this.projectRows(rows, !opts?.compact);
|
||||
return opts?.compact ? items.map(compactRow) : items;
|
||||
}
|
||||
|
||||
/** The listing projection (list, and show as a listing of one row): each row's item with its delivery outcome, the
|
||||
* owner notifications, ledgers and shared reads read once for all the rows. includeWaiting: full rows (SELECT *). */
|
||||
private projectRows(rows: QueueItemRow[], includeWaiting: boolean): QueueItem[] {
|
||||
const owners = this.transitionLog.ownerNotificationsForQitems(rows.map((row) => row.qitem_id));
|
||||
const ledgers = this.deliveryOutcomesFor(rows, owners, true);
|
||||
const reads = this.listingReads(rows, includeWaiting);
|
||||
return rows.map((r) => {
|
||||
const items = rows.map((r) => {
|
||||
const ledger = ledgers.get(r.qitem_id);
|
||||
const item = this.rowToItem(r, includeWaiting, { owners: owners.get(r.qitem_id) ?? [], ledger: ledger ?? null, reads });
|
||||
const item = this.rowToItem(r, !opts?.compact, { owners: owners.get(r.qitem_id) ?? [], ledger: ledger ?? null });
|
||||
return {
|
||||
...item,
|
||||
deliveryOutcome: ledger?.outcome ?? null,
|
||||
...(ledger && ledger.outcome !== "posted" && ledger.outcome !== "accepted" ? { deliveryFailureDetail: ledger.detail } : {}),
|
||||
};
|
||||
});
|
||||
return opts?.compact ? items.map(compactRow) : items;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -5503,27 +5509,9 @@ export class QueueRepository {
|
||||
try { return this.activityReader?.(session) ?? null; } catch { return null; }
|
||||
}
|
||||
|
||||
/** What a listing reads once for all its rows instead of once per row. includeWaiting: the rows' waiting views read
|
||||
* their qitem blockers' last meaningful transitions too. */
|
||||
private listingReads(rows: QueueItemRow[], includeWaiting: boolean): ListingReads {
|
||||
const ids = rows.map((r) => r.qitem_id);
|
||||
if (includeWaiting) for (const r of rows) if (r.blocked_on?.startsWith("qitem-")) ids.push(r.blocked_on);
|
||||
// Each owner's activity once per listing: a row and its waiting view (and every other row to that owner) read the
|
||||
// same seat state. ownerActivity reads a failure as null, so one session's failure stays that session's.
|
||||
const activities = new Map<string, ReturnType<WaitingActivityReader>>();
|
||||
const activity = (session: string): ReturnType<WaitingActivityReader> => {
|
||||
if (!activities.has(session)) activities.set(session, this.ownerActivity(session));
|
||||
return activities.get(session) ?? null;
|
||||
};
|
||||
return { meaningful: lastMeaningfulTransitionMany(this.db, ids), statements: new WaitingStatements(this.db), activity };
|
||||
}
|
||||
|
||||
/** listed: the row's owner notifications and ledger, when a listing read them for all its rows. */
|
||||
waitingView(qitemId: string, listed?: ListedLedger): WaitingView | null {
|
||||
// The row in hand, when the listing passed it: its later re-reads below are served by it too.
|
||||
const inHand = listed?.row && listed.row.qitem_id === qitemId ? listed.row : undefined;
|
||||
const reads = listed?.reads && inHand ? { ...listed.reads, row: inHand } : listed?.reads;
|
||||
const view = readWaitingView(this.db, qitemId, listed?.reads?.activity ?? this.activityReader, target => this.acceptsDestination(target), reads);
|
||||
const view = readWaitingView(this.db, qitemId, this.activityReader, target => this.acceptsDestination(target));
|
||||
if (view && ["pending", "in-progress"].includes(view.state)) {
|
||||
// #514 — the unclaimed sweep skips a pending ask already posted to its human in the current
|
||||
// episode (same predicate as runStuckSweep), so the view must not promise that sweep.
|
||||
@@ -5531,9 +5519,7 @@ export class QueueRepository {
|
||||
const posted = episodeDelivery !== null;
|
||||
const onSteering = !posted && view.state === "pending" && this.humanAskWithoutAlert(qitemId, listed);
|
||||
const row = view.state === "pending" && !onSteering
|
||||
? inHand
|
||||
? { state: inHand.state as QueueItem["state"], human_intent: inHand.human_intent ?? null, destination_session: inHand.destination_session, last_nudge_result: inHand.last_nudge_result }
|
||||
: this.db.prepare(`SELECT state, ${this.hasHumanIntentColumn ? "human_intent" : "NULL AS human_intent"}, destination_session, last_nudge_result FROM queue_items WHERE qitem_id = ?`).get(qitemId) as { state: QueueItem["state"]; human_intent: QueueItem["humanIntent"]; destination_session: string; last_nudge_result: string | null } | undefined
|
||||
? this.db.prepare(`SELECT state, ${this.hasHumanIntentColumn ? "human_intent" : "NULL AS human_intent"}, destination_session, last_nudge_result FROM queue_items WHERE qitem_id = ?`).get(qitemId) as { state: QueueItem["state"]; human_intent: QueueItem["humanIntent"]; destination_session: string; last_nudge_result: string | null } | undefined
|
||||
: undefined;
|
||||
const humanUpdate = !!row && this.isRegisteredHumanUpdate({ state: row.state, humanIntent: row.human_intent, destinationSession: row.destination_session });
|
||||
const noticeOutcome = !posted && !onSteering ? (listed ? listed.ledger?.outcome : this.deliveryOutcomeFor(qitemId)?.outcome) : undefined;
|
||||
@@ -5560,7 +5546,7 @@ export class QueueRepository {
|
||||
if (recovery && !localOnly) {
|
||||
if (!awaitingHuman && !humanUpdate) view.laterBackstop = { ...view.nextBackstop, note: "Conditional safety net; current delivery/recovery ownership is evaluated first." };
|
||||
view.nextBackstop = recovery;
|
||||
} else if (!awaitingHuman && view.state === "pending" && (inHand ?? this.db.prepare("SELECT last_nudge_result FROM queue_items WHERE qitem_id = ?")
|
||||
} else if (!awaitingHuman && view.state === "pending" && (this.db.prepare("SELECT last_nudge_result FROM queue_items WHERE qitem_id = ?")
|
||||
.get(qitemId) as { last_nudge_result: string | null } | undefined)?.last_nudge_result === "retained:typing_guard") {
|
||||
// Held at the seat on purpose: not stuck, and not picked up. Nothing replays it; reading or delivering it is a
|
||||
// deliberate step.
|
||||
@@ -5607,11 +5593,9 @@ export class QueueRepository {
|
||||
private rowToItem(row: QueueItemRow, includeWaiting = true, listed?: ListedLedger): QueueItem {
|
||||
// S04 — derive the pickup receipt at the ONE shared projection point (list/show/overdue
|
||||
// all flow through here), so the park-vs-strand question is answered by the row face.
|
||||
const meaningful = meaningfulFor(this.db, row.qitem_id, listed?.reads);
|
||||
// A listing's rows are its own SELECT *: the waiting view reads the row in hand instead of reading it again.
|
||||
const waiting = includeWaiting ? this.waitingView(row.qitem_id, listed?.reads ? { ...listed, row } : listed) : null;
|
||||
const listedActivity = listed?.reads?.activity;
|
||||
const activity = listedActivity ? listedActivity(row.destination_session) : this.ownerActivity(row.destination_session);
|
||||
const meaningful = lastMeaningfulTransition(this.db, row.qitem_id);
|
||||
const waiting = includeWaiting ? this.waitingView(row.qitem_id, listed) : null;
|
||||
const activity = this.ownerActivity(row.destination_session);
|
||||
const pickup = derivePickup({
|
||||
state: row.state,
|
||||
lastMeaningfulAt: meaningful?.at,
|
||||
@@ -5668,6 +5652,7 @@ export class QueueRepository {
|
||||
// PL-007: target_repo present only when migration 038 has applied;
|
||||
// older test fixtures supply legacy rows where target_repo is undefined.
|
||||
targetRepo: row.target_repo ?? null,
|
||||
...(row.content_erased_at ? { contentErasedAt: row.content_erased_at } : {}),
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -5677,13 +5662,8 @@ type LedgerItem = Pick<QueueItem, "qitemId" | "state" | "blockedOn" | "destinati
|
||||
|
||||
type DeliveryLedger = { outcome: "posted" | "accepted" | "transport-failed" | "never-posted"; detail: string; episode: { notificationKey: string; startedAt: string } | null };
|
||||
|
||||
/** What a listing reads once for all its rows (see QueueRepository.listingReads): the waiting view's reads, and each
|
||||
* owner's activity, memoized for the listing. */
|
||||
type ListingReads = WaitingReads & { activity?: (session: string) => ReturnType<WaitingActivityReader> };
|
||||
|
||||
/** A listed row's owner notifications (newest first) and its delivery ledger, read once for the whole listing, and the
|
||||
* listing's shared reads. */
|
||||
type ListedLedger = { owners: QueueTransition[]; ledger: DeliveryLedger | null; reads?: ListingReads; row?: QueueItemRow };
|
||||
/** A listed row's owner notifications (newest first) and its delivery ledger, read once for the whole listing. */
|
||||
type ListedLedger = { owners: QueueTransition[]; ledger: DeliveryLedger | null };
|
||||
|
||||
function ledgerItem(row: LedgerRow): LedgerItem {
|
||||
return {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type Database from "better-sqlite3";
|
||||
import { TERMINAL_QUEUE_STATES } from "./queue-repository.js";
|
||||
import { eraseQueue } from "./queue-erasure.js";
|
||||
|
||||
/**
|
||||
* OPR.0.4.6 FS-1 (daemon read-path hardening) — W2 retention/prune runner.
|
||||
@@ -93,6 +94,10 @@ export const RETENTION_DEFAULTS = {
|
||||
eventsWindow: 1_000,
|
||||
/** pause between two events windows, after the event-loop yield */
|
||||
eventsPauseMs: 5,
|
||||
/** remove the content of closed work (done, canceled, denied, handed-off) this many days after it closed; 0 keeps it */
|
||||
closedContentDays: 0,
|
||||
/** delete closed rows, keeping only their ids, this many days after they closed; 0 keeps them */
|
||||
closedMetadataDays: 0,
|
||||
} as const;
|
||||
|
||||
/**
|
||||
@@ -128,6 +133,10 @@ export interface RetentionOptions {
|
||||
eventsRetentionDays?: number;
|
||||
eventsWindow?: number;
|
||||
eventsPauseMs?: number;
|
||||
closedContentDays?: number;
|
||||
closedMetadataDays?: number;
|
||||
/** where closed-work erasure appends the ids and actions it erases (retention.erasure_ledger); unset keeps none */
|
||||
erasureLedgerPath?: string;
|
||||
/** the events pass's yield between windows; injectable so a test can count it */
|
||||
eventsYield?: () => Promise<void>;
|
||||
/** called once per events statement with its seq bounds (lo, hi]; for tests and measurement */
|
||||
@@ -528,6 +537,12 @@ export interface RetentionSweepSummary {
|
||||
eventsBatches: number;
|
||||
/** the events low-water seq persisted after the pass (null when none is stored) */
|
||||
eventsLowWater: number | null;
|
||||
/** closed rows whose content was removed, rows deleted, and rows a workflow kept (queue-erasure, routine) */
|
||||
closedScrubbed: number;
|
||||
closedDeleted: number;
|
||||
closedKept: number;
|
||||
/** why closed-work erasure stopped, when it did; the other passes still ran */
|
||||
closedError: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -554,8 +569,33 @@ export async function runQueueRetentionSweep(
|
||||
eventsDeleted: 0,
|
||||
eventsBatches: 0,
|
||||
eventsLowWater: null,
|
||||
closedScrubbed: 0,
|
||||
closedDeleted: 0,
|
||||
closedKept: 0,
|
||||
closedError: null,
|
||||
};
|
||||
|
||||
// Closed-work retention, opt-in: content first, then the row itself, each from when the row closed.
|
||||
const closedContentDays = opts.closedContentDays ?? RETENTION_DEFAULTS.closedContentDays;
|
||||
const closedMetadataDays = opts.closedMetadataDays ?? RETENTION_DEFAULTS.closedMetadataDays;
|
||||
if (closedContentDays > 0 || closedMetadataDays > 0) {
|
||||
// Reported, never fatal: an erasure that stops leaves the other passes to run (it returns what it committed).
|
||||
try {
|
||||
const closed = await eraseQueue(db, {
|
||||
scope: { kind: "routine", contentDays: closedContentDays, metadataDays: closedMetadataDays },
|
||||
apply: true, nowIso: opts.nowIso, batchSize: opts.batchSize ?? RETENTION_DEFAULTS.batchSize, maxBatches,
|
||||
...(opts.erasureLedgerPath ? { ledgerPath: opts.erasureLedgerPath } : {}),
|
||||
});
|
||||
summary.closedScrubbed = closed.qitems.filter((row) => row.action === "scrub").length;
|
||||
summary.closedDeleted = closed.qitems.filter((row) => row.action === "delete").length;
|
||||
summary.closedKept = closed.gaps.length;
|
||||
summary.closedError = closed.error ? `${closed.error.stage}: ${closed.error.message}` : null;
|
||||
} catch (err) {
|
||||
summary.closedError = err instanceof Error ? err.message : String(err);
|
||||
}
|
||||
if (summary.closedError) console.error(`[queue-retention] closed-work erasure stopped: ${summary.closedError}`);
|
||||
}
|
||||
|
||||
for (let i = 0; i < maxBatches; i++) {
|
||||
const batch = archiveAgedTerminalTransitions(db, opts);
|
||||
if (batch.archivedQitems === 0) break;
|
||||
|
||||
@@ -27,83 +27,15 @@ export function lastMeaningfulTransition(db: Database.Database, id: string): { i
|
||||
return row ? { id: row.transition_id, at: row.ts } : null;
|
||||
}
|
||||
|
||||
/** lastMeaningfulTransition for many rows in one read: the same LAG window, partitioned by row, and the same filters.
|
||||
* Every requested id is a key of the result; an id without a meaningful transition maps to null. */
|
||||
export function lastMeaningfulTransitionMany(db: Database.Database, ids: readonly string[]): Map<string, { id: number; at: string } | null> {
|
||||
const unique = [...new Set(ids)];
|
||||
const out = new Map<string, { id: number; at: string } | null>(unique.map((id) => [id, null]));
|
||||
if (unique.length === 0) return out;
|
||||
if (!db.prepare("SELECT 1 FROM sqlite_master WHERE name = 'queue_transitions'").get()) return out;
|
||||
const hasWakes = db.prepare("SELECT 1 FROM sqlite_master WHERE name = 'queue_transition_wakes'").get();
|
||||
const rows = db.prepare(`SELECT qitem_id, transition_id, ts FROM (
|
||||
SELECT t.qitem_id, t.transition_id, t.ts, ROW_NUMBER() OVER (PARTITION BY t.qitem_id ORDER BY t.transition_id DESC) AS newest FROM (
|
||||
SELECT *, LAG(state) OVER (PARTITION BY qitem_id ORDER BY transition_id) AS previous_state FROM queue_transitions
|
||||
WHERE qitem_id IN (SELECT value FROM json_each(?))
|
||||
) t WHERE (t.actor_session NOT IN ('watchdog@system', 'wake-ladder@system', 'daemon@kernel', 'daemon@system') OR t.state IS NOT t.previous_state)
|
||||
${hasWakes ? "AND NOT EXISTS (SELECT 1 FROM queue_transition_wakes w WHERE w.transition_id = t.transition_id AND w.phase = 'fired')" : ""}
|
||||
) WHERE newest = 1`).all(JSON.stringify(unique)) as Array<{ qitem_id: string; transition_id: number; ts: string }>;
|
||||
for (const row of rows) out.set(row.qitem_id, { id: row.transition_id, at: row.ts });
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The waiting view's statements, each prepared once, and the schema checks they depend on, each made once. A listing
|
||||
* holds one for all its rows; a single read uses a fresh one. A listing reads synchronously, so neither the statements
|
||||
* nor the schema checks can go stale under it. */
|
||||
export class WaitingStatements {
|
||||
private readonly prepared = new Map<string, Database.Statement>();
|
||||
private readonly tables = new Map<string, boolean>();
|
||||
constructor(private readonly db: Database.Database) {}
|
||||
|
||||
statement(sql: string): Database.Statement {
|
||||
let prepared = this.prepared.get(sql);
|
||||
if (!prepared) {
|
||||
prepared = this.db.prepare(sql);
|
||||
this.prepared.set(sql, prepared);
|
||||
}
|
||||
return prepared;
|
||||
}
|
||||
|
||||
hasTable(name: string): boolean {
|
||||
let known = this.tables.get(name);
|
||||
if (known === undefined) {
|
||||
known = this.statement("SELECT 1 FROM sqlite_master WHERE name = ?").get(name) !== undefined;
|
||||
this.tables.set(name, known);
|
||||
}
|
||||
return known;
|
||||
}
|
||||
}
|
||||
|
||||
/** The queue_items columns a waiting view reads from its own row. */
|
||||
export interface WaitingRow {
|
||||
qitem_id: string; source_session: string; destination_session: string; state: string; blocked_on: string | null; closure_required_at: string | null; tags: string | null;
|
||||
}
|
||||
|
||||
/** What a listing has already read for all its rows, lent to each row's waiting view. */
|
||||
export interface WaitingReads {
|
||||
/** The row in hand (a listing's own SELECT *), read in the same synchronous pass; used only when its id is the view's. */
|
||||
row?: WaitingRow;
|
||||
/** The listing's statements and schema checks, shared by all its rows. */
|
||||
statements?: WaitingStatements;
|
||||
/** lastMeaningfulTransitionMany over the listing's rows and their qitem blockers; an id it lacks is read alone. */
|
||||
meaningful?: ReadonlyMap<string, { id: number; at: string } | null>;
|
||||
}
|
||||
|
||||
/** One row's last meaningful transition: from the listing's batch when it read this id, else read alone. */
|
||||
export function meaningfulFor(db: Database.Database, id: string, reads?: WaitingReads): { id: number; at: string } | null {
|
||||
const known = reads?.meaningful;
|
||||
return known && known.has(id) ? known.get(id) ?? null : lastMeaningfulTransition(db, id);
|
||||
}
|
||||
|
||||
/** Grace starts when a row becomes actionable again, not at its birth before a
|
||||
* long healthy park. Same-state notes and deliveries cannot move this boundary. */
|
||||
export function pendingSince(db: Database.Database, id: string, statements = new WaitingStatements(db)): string | null {
|
||||
const created = () => (statements.statement("SELECT ts_created FROM queue_items WHERE qitem_id = ?").get(id) as { ts_created: string } | undefined)?.ts_created ?? null;
|
||||
if (!statements.hasTable("queue_transitions")) return created();
|
||||
const row = statements.statement(`SELECT ts, previous_state FROM (SELECT ts, state, transition_id,
|
||||
export function pendingSince(db: Database.Database, id: string): string | null {
|
||||
if (!db.prepare("SELECT 1 FROM sqlite_master WHERE name = 'queue_transitions'").get()) return (db.prepare("SELECT ts_created FROM queue_items WHERE qitem_id = ?").get(id) as { ts_created: string } | undefined)?.ts_created ?? null;
|
||||
const row = db.prepare(`SELECT ts, previous_state FROM (SELECT ts, state, transition_id,
|
||||
LAG(state) OVER (ORDER BY transition_id) AS previous_state FROM queue_transitions WHERE qitem_id = ?)
|
||||
WHERE state = 'pending' AND previous_state IS NOT 'pending' ORDER BY transition_id DESC LIMIT 1`)
|
||||
.get(id) as { ts: string; previous_state: string | null } | undefined;
|
||||
if (!row || row.previous_state === null) return created();
|
||||
if (!row || row.previous_state === null) return (db.prepare("SELECT ts_created FROM queue_items WHERE qitem_id = ?").get(id) as { ts_created: string } | undefined)?.ts_created ?? null;
|
||||
return row.ts;
|
||||
}
|
||||
function unclaimedSeconds(): number {
|
||||
@@ -135,14 +67,13 @@ export interface WaitingView {
|
||||
attention?: { scope: string; revision: string; source: "last observed by wait timer" };
|
||||
}
|
||||
|
||||
export function readWaitingView(db: Database.Database, id: string, readActivity?: WaitingActivityReader, acceptsDestination?: (destination: string) => boolean,
|
||||
reads?: WaitingReads): WaitingView | null {
|
||||
const statements = reads?.statements ?? new WaitingStatements(db);
|
||||
const held = reads?.row;
|
||||
const row = held && held.qitem_id === id ? held : statements.statement(`SELECT qitem_id, source_session, destination_session, state, blocked_on, closure_required_at, tags
|
||||
FROM queue_items WHERE qitem_id = ?`).get(id) as WaitingRow | undefined;
|
||||
export function readWaitingView(db: Database.Database, id: string, readActivity?: WaitingActivityReader, acceptsDestination?: (destination: string) => boolean): WaitingView | null {
|
||||
const row = db.prepare(`SELECT qitem_id, source_session, destination_session, state, blocked_on, closure_required_at, tags
|
||||
FROM queue_items WHERE qitem_id = ?`).get(id) as {
|
||||
qitem_id: string; source_session: string; destination_session: string; state: string; blocked_on: string | null; closure_required_at: string | null; tags: string | null;
|
||||
} | undefined;
|
||||
if (!row) return null;
|
||||
const blocker = row.blocked_on?.startsWith("qitem-") ? statements.statement(
|
||||
const blocker = row.blocked_on?.startsWith("qitem-") ? db.prepare(
|
||||
"SELECT destination_session, state FROM queue_items WHERE qitem_id = ?",
|
||||
).get(row.blocked_on) as { destination_session: string; state: string } | undefined : undefined;
|
||||
let recoveryOwner: string | null = null;
|
||||
@@ -151,10 +82,10 @@ export function readWaitingView(db: Database.Database, id: string, readActivity?
|
||||
const subject = row.blocked_on ? blocker?.destination_session ?? row.blocked_on : row.destination_session;
|
||||
let observed: ReturnType<WaitingActivityReader> = null;
|
||||
try { observed = readActivity?.(subject) ?? null; } catch { /* unavailable remains unknown */ }
|
||||
const meaningful = meaningfulFor(db, row.blocked_on?.startsWith("qitem-") ? row.blocked_on : id, reads);
|
||||
const meaningful = lastMeaningfulTransition(db, row.blocked_on?.startsWith("qitem-") ? row.blocked_on : id);
|
||||
const sweepInterval = sweepSeconds();
|
||||
const view: WaitingView = {
|
||||
obligation: id, owner: row.destination_session, state: row.state, actionableSince: row.state === "pending" ? pendingSince(db, id, statements) : null,
|
||||
obligation: id, owner: row.destination_session, state: row.state, actionableSince: row.state === "pending" ? pendingSince(db, id) : null,
|
||||
blocker: row.blocked_on ? { ref: row.blocked_on, owner: blocker?.destination_session ?? null, state: blocker?.state ?? null } : null,
|
||||
lastMeaningfulChange: meaningful,
|
||||
liveness: { subject, activity: observed?.activity ?? "unknown", needsInput: observed?.needsInput ?? { count: 0, reason: null }, confidence: observed && observed.activity !== "unknown" ? "oracle" : "unknown" },
|
||||
@@ -169,8 +100,8 @@ export function readWaitingView(db: Database.Database, id: string, readActivity?
|
||||
view.nextBackstop.mechanism = row.state === "pending" ? "queue-stuck-sweep:unclaimed" : "queue-stuck-sweep:pickup (activity checked at detection)";
|
||||
}
|
||||
if (row.state === "blocked") {
|
||||
const hasTimers = statements.hasTable("queue_transition_wakes");
|
||||
const timer = hasTimers ? statements.statement(`SELECT j.job_id, j.policy, j.last_evaluation_at, j.registered_at, j.interval_seconds, j.scan_interval_seconds, j.spec_yaml
|
||||
const hasTimers = db.prepare("SELECT 1 FROM sqlite_master WHERE name = 'queue_transition_wakes'").get();
|
||||
const timer = hasTimers ? db.prepare(`SELECT j.job_id, j.policy, j.last_evaluation_at, j.registered_at, j.interval_seconds, j.scan_interval_seconds, j.spec_yaml
|
||||
FROM queue_transition_wakes w JOIN watchdog_jobs j ON j.job_id = w.wake_ref
|
||||
WHERE w.qitem_id = ? AND w.phase = 'armed' AND j.state = 'active'
|
||||
ORDER BY w.transition_id DESC LIMIT 1`).get(id) as { job_id: string; policy: WatchdogJob["policy"]; last_evaluation_at: string | null; registered_at: string; interval_seconds: number; scan_interval_seconds: number | null; spec_yaml: string } | undefined : undefined;
|
||||
|
||||
@@ -60,7 +60,7 @@ function toExchange(obj: unknown): JsonlExchange | null {
|
||||
* Honest-degraded: missing/unreadable file → []; unparseable/metadata/text-less lines are skipped.
|
||||
*/
|
||||
export function parseJsonlExchanges(path: string, n: number): JsonlExchange[] {
|
||||
if (n <= 0 || !existsSync(path)) return [];
|
||||
if (!existsSync(path)) return [];
|
||||
let raw: string;
|
||||
try {
|
||||
raw = readFileSync(path, "utf8");
|
||||
@@ -68,12 +68,7 @@ export function parseJsonlExchanges(path: string, n: number): JsonlExchange[] {
|
||||
return [];
|
||||
}
|
||||
const exchanges: JsonlExchange[] = [];
|
||||
// Only decode the tail needed for the recap, not every exchange in a long session.
|
||||
let end = raw.length;
|
||||
while (end > 0) {
|
||||
const start = raw.lastIndexOf("\n", end - 1) + 1;
|
||||
const line = raw.slice(start, end);
|
||||
end = start - 1;
|
||||
for (const line of raw.split("\n")) {
|
||||
if (line.trim().length === 0) continue;
|
||||
let obj: unknown;
|
||||
try {
|
||||
@@ -82,7 +77,7 @@ export function parseJsonlExchanges(path: string, n: number): JsonlExchange[] {
|
||||
continue; // corrupt line — skip, never throw
|
||||
}
|
||||
const ex = toExchange(obj);
|
||||
if (ex && exchanges.push(ex) >= n) break;
|
||||
if (ex) exchanges.push(ex);
|
||||
}
|
||||
return exchanges.reverse();
|
||||
return n >= exchanges.length ? exchanges : exchanges.slice(exchanges.length - n);
|
||||
}
|
||||
@@ -4,15 +4,15 @@
|
||||
|
||||
// An ISO date or time, or a leading part of one: 2026, 2026-10, 2026-10-14, 2026-10-14T09,
|
||||
// 2026-10-14T09:30, 2026-10-14T09:30:00.123Z, with an optional zone once there is an hour.
|
||||
const ISO_DATE_OR_TIME = /^(\d{4})(?:-(\d{2})(?:-(\d{2})(?:T(\d{2})(?::(\d{2})(?::(\d{2})(?:\.\d+)?)?)?(?:Z|[+-](\d{2}):(\d{2}))?)?)?)?$/;
|
||||
const ISO_DATE_OR_TIME = /^(\d{4})(?:-(\d{2})(?:-(\d{2})(?:T(\d{2})(?::(\d{2})(?::(\d{2})(?:\.\d+)?)?)?(?:Z|[+-]\d{2}:\d{2})?)?)?)?$/;
|
||||
|
||||
/** Whether a bound reads as an ISO date or time (or a leading part of one) with in-range fields. */
|
||||
export function isRecognisedDateOrTime(value: string): boolean {
|
||||
const match = ISO_DATE_OR_TIME.exec(value);
|
||||
if (!match) return false;
|
||||
const [year, month = 1, day = 1, hour = 0, minute = 0, second = 0, zoneHour = 0, zoneMinute = 0] =
|
||||
match.slice(1).map((part) => (part === undefined ? undefined : Number(part)));
|
||||
if (month < 1 || month > 12 || hour > 23 || minute > 59 || second > 59 || zoneHour > 23 || zoneMinute > 59) return false;
|
||||
const [year, month = 1, day = 1, hour = 0, minute = 0, second = 0] =
|
||||
match.slice(1, 7).map((part) => (part === undefined ? undefined : Number(part)));
|
||||
if (month < 1 || month > 12 || hour > 23 || minute > 59 || second > 59) return false;
|
||||
const lastDay = new Date(0);
|
||||
lastDay.setUTCFullYear(year!, month, 0);
|
||||
return day >= 1 && day <= lastDay.getUTCDate();
|
||||
|
||||
@@ -229,6 +229,12 @@ export const SETTINGS_VALID_KEYS = [
|
||||
"retention.usage_samples_days",
|
||||
// The history-only event types of `events` (queue-retention PRUNABLE_EVENT_TYPES) older than N days; 0 (default) keeps them.
|
||||
"retention.events_days",
|
||||
// Closed queue work (done, canceled, denied, handed-off): its content goes N days after it closed, then the row, keeping
|
||||
// only its id (queue-erasure). 0 (default) keeps them.
|
||||
"retention.closed_content_days",
|
||||
"retention.closed_metadata_days",
|
||||
// Where queue erasure appends each erased id and action (JSON lines), for replaying after a restore; empty keeps none.
|
||||
"retention.erasure_ledger",
|
||||
"retention.watchdog_keep_per_job",
|
||||
// Slice 22 outcome 9 — stream retention: unreferenced items older than N days or beyond the newest N.
|
||||
"retention.stream_days",
|
||||
@@ -355,6 +361,9 @@ const ENV_MAP: Record<SettingsValidKey, { primary: string; legacy?: string }> =
|
||||
"retention.watchdog_days": { primary: "OPENRIG_RETENTION_WATCHDOG_DAYS" },
|
||||
"retention.usage_samples_days": { primary: "OPENRIG_RETENTION_USAGE_SAMPLES_DAYS" },
|
||||
"retention.events_days": { primary: "OPENRIG_RETENTION_EVENTS_DAYS" },
|
||||
"retention.closed_content_days": { primary: "OPENRIG_RETENTION_CLOSED_CONTENT_DAYS" },
|
||||
"retention.closed_metadata_days": { primary: "OPENRIG_RETENTION_CLOSED_METADATA_DAYS" },
|
||||
"retention.erasure_ledger": { primary: "OPENRIG_RETENTION_ERASURE_LEDGER" },
|
||||
"retention.watchdog_keep_per_job": { primary: "OPENRIG_RETENTION_WATCHDOG_KEEP_PER_JOB" },
|
||||
"retention.stream_days": { primary: "OPENRIG_RETENTION_STREAM_DAYS" },
|
||||
"retention.stream_max_items": { primary: "OPENRIG_RETENTION_STREAM_MAX_ITEMS" },
|
||||
@@ -459,6 +468,9 @@ const KEY_TO_PATH: Record<SettingsValidKey, string[]> = {
|
||||
"retention.watchdog_days": ["retention", "watchdogDays"],
|
||||
"retention.usage_samples_days": ["retention", "usageSamplesDays"],
|
||||
"retention.events_days": ["retention", "eventsDays"],
|
||||
"retention.closed_content_days": ["retention", "closedContentDays"],
|
||||
"retention.closed_metadata_days": ["retention", "closedMetadataDays"],
|
||||
"retention.erasure_ledger": ["retention", "erasureLedger"],
|
||||
"retention.watchdog_keep_per_job": ["retention", "watchdogKeepPerJob"],
|
||||
"retention.stream_days": ["retention", "streamDays"],
|
||||
"retention.stream_max_items": ["retention", "streamMaxItems"],
|
||||
@@ -781,6 +793,9 @@ function getDefaultValue(key: SettingsValidKey, workspaceRoot: string): string |
|
||||
case "retention.watchdog_days": return 14;
|
||||
case "retention.usage_samples_days": return 14;
|
||||
case "retention.events_days": return 0; // 0 = keep; pruning is opt-in
|
||||
case "retention.closed_content_days": return 0; // 0 = keep; closed-work erasure is opt-in
|
||||
case "retention.closed_metadata_days": return 0;
|
||||
case "retention.erasure_ledger": return ""; // empty = keep no ledger
|
||||
case "retention.watchdog_keep_per_job": return 50;
|
||||
case "retention.stream_days": return 30;
|
||||
case "retention.stream_max_items": return 20000;
|
||||
@@ -1007,6 +1022,24 @@ const KEY_CONSTRAINTS: Partial<Record<SettingsValidKey, (raw: string, coerced: s
|
||||
throw new Error(`Invalid value for retention.events_days: must be an integer >= 0 (0 keeps the events), got "${raw}"`);
|
||||
}
|
||||
},
|
||||
// 0 (the default) keeps closed work; a positive day count turns its erasure on (queue-erasure).
|
||||
"retention.closed_content_days": (raw, coerced) => {
|
||||
if (!/^-?\d+$/.test((raw ?? "").trim()) || typeof coerced !== "number" || !Number.isInteger(coerced) || coerced < 0) {
|
||||
throw new Error(`Invalid value for retention.closed_content_days: must be an integer >= 0 (0 keeps the content), got "${raw}"`);
|
||||
}
|
||||
},
|
||||
"retention.closed_metadata_days": (raw, coerced) => {
|
||||
if (!/^-?\d+$/.test((raw ?? "").trim()) || typeof coerced !== "number" || !Number.isInteger(coerced) || coerced < 0) {
|
||||
throw new Error(`Invalid value for retention.closed_metadata_days: must be an integer >= 0 (0 keeps the rows), got "${raw}"`);
|
||||
}
|
||||
},
|
||||
// Empty (the default) keeps no erasure ledger; otherwise an absolute file path, kept outside the backed-up data.
|
||||
"retention.erasure_ledger": (raw) => {
|
||||
const value = (raw ?? "").trim();
|
||||
if (value !== "" && !value.startsWith("/")) {
|
||||
throw new Error(`Invalid value for retention.erasure_ledger: must be empty or an absolute file path, got "${raw}"`);
|
||||
}
|
||||
},
|
||||
"retention.watchdog_keep_per_job": (raw, coerced) => {
|
||||
if (!/^-?\d+$/.test((raw ?? "").trim()) || typeof coerced !== "number" || !Number.isInteger(coerced) || coerced < 0) {
|
||||
throw new Error(`Invalid value for retention.watchdog_keep_per_job: must be an integer >= 0, got "${raw}"`);
|
||||
|
||||
@@ -118,7 +118,8 @@ export function startPeriodicSnapshotScheduler(deps: {
|
||||
/** OPR.0.4.6.FS-1 W2 — start the queue-retention maintenance sweep (arch D3): a
|
||||
* boot sweep + a daily tick that archives aged terminal `queue_transitions`
|
||||
* (never-delete) and prunes `watchdog_history`, `usage_samples`, stream items and
|
||||
* the history-only event types of `events`, all in bounded batches with
|
||||
* the history-only event types of `events`, and (opt-in) erases closed work
|
||||
* (`retention.closed_content_days`, `retention.closed_metadata_days`), all in bounded batches with
|
||||
* event-loop yields. Reads the `retention.*` settings (baked defaults via
|
||||
* RETENTION_DEFAULTS). Fire-and-forget with error logging — a sweep failure must
|
||||
* never crash the daemon. Returns the daily interval handle (cleared on
|
||||
@@ -135,6 +136,10 @@ export function startQueueRetentionScheduler(deps: {
|
||||
const v = store ? store.resolveOne(key).value : fallback;
|
||||
return typeof v === "number" ? v : fallback;
|
||||
};
|
||||
const ledger = (): string | undefined => {
|
||||
const v = store ? store.resolveOne("retention.erasure_ledger").value : undefined;
|
||||
return typeof v === "string" && v.length > 0 ? v : undefined;
|
||||
};
|
||||
const runOnce = (): void => {
|
||||
void runQueueRetentionSweep(db, {
|
||||
nowIso: new Date().toISOString(),
|
||||
@@ -146,6 +151,9 @@ export function startQueueRetentionScheduler(deps: {
|
||||
streamMaxItems: num("retention.stream_max_items", RETENTION_DEFAULTS.streamMaxItems),
|
||||
batchSize: num("retention.batch_size", RETENTION_DEFAULTS.batchSize),
|
||||
eventsRetentionDays: num("retention.events_days", RETENTION_DEFAULTS.eventsRetentionDays),
|
||||
closedContentDays: num("retention.closed_content_days", RETENTION_DEFAULTS.closedContentDays),
|
||||
closedMetadataDays: num("retention.closed_metadata_days", RETENTION_DEFAULTS.closedMetadataDays),
|
||||
...(ledger() ? { erasureLedgerPath: ledger() } : {}),
|
||||
}).catch((err: unknown) => {
|
||||
console.error(`[queue-retention] sweep error: ${err instanceof Error ? err.message : String(err)}`);
|
||||
});
|
||||
|
||||
@@ -950,7 +950,7 @@ activityRoutes.get("/parked", (c) => {
|
||||
| undefined;
|
||||
const queueRepo = c.get("queueRepo" as never) as
|
||||
| {
|
||||
listOpenObligations: (destinationSession: string, limit: number) => Array<{ qitemId: string; state: "pending" | "in-progress" | "blocked"; summary: string | null }>;
|
||||
list: (opts: { destinationSession?: string; state?: string[]; limit?: number }) => Array<{ qitemId: string; state: string; summary?: string | null }>;
|
||||
getParkWakeStatus: (qitemId: string) => import("../domain/queue-wake-repository.js").ParkWakeStatus | null;
|
||||
parkWakeServedByOwnReminder?: (qitemId: string) => boolean;
|
||||
}
|
||||
@@ -967,10 +967,10 @@ activityRoutes.get("/parked", (c) => {
|
||||
const { diagnoseSeatParked, diagnoseRigParked, PARKED_OBLIGATION_LIMIT } = parkedQuery;
|
||||
const deps = {
|
||||
getSeatState: (id: string) => nodeExecutorConfirmedStopped(rigRepo.db, id) ? null : oracle.getSeatState(id),
|
||||
// The obligation face only: the same rows, order and limit as a full list() of this destination's open states,
|
||||
// without the per-row projections (pickup, waiting view, delivery ledger) the diagnosis never reads.
|
||||
listOpenObligations: (destination: string, limit: number) => ({
|
||||
rows: queueRepo.listOpenObligations(destination, limit),
|
||||
rows: queueRepo
|
||||
.list({ destinationSession: destination, state: ["pending", "in-progress", "blocked"], limit })
|
||||
.map((r) => ({ qitemId: r.qitemId, state: r.state as "pending" | "in-progress" | "blocked", summary: r.summary ?? null })),
|
||||
limit,
|
||||
}),
|
||||
getParkWake: (qitemId: string) => queueRepo.getParkWakeStatus(qitemId),
|
||||
|
||||
@@ -31,6 +31,7 @@ import { listProjects, projectCatalogPaths } from "../domain/workspace/project-r
|
||||
import { readProjectCatalog } from "../domain/workspace/project-catalog.js";
|
||||
import type { WhoamiService } from "../domain/whoami-service.js";
|
||||
import type Database from "better-sqlite3";
|
||||
import { EraseRefusal, eraseQueue, type EraseAction, type EraseScope } from "../domain/queue-erasure.js";
|
||||
import { HumanModeService } from "../domain/gateway/human-operating-modes.js";
|
||||
import type { HumanQuestion } from "../domain/human-questions.js";
|
||||
import { sharedChannelFor } from "../domain/gateway/participants.js";
|
||||
@@ -227,6 +228,8 @@ export function queueRoutes(): Hono {
|
||||
// caller id-reuse (a bug, not an idempotent retry) — surface as a
|
||||
// conflict, never overwrite.
|
||||
: err.code === "qitem_id_reuse" ? 409
|
||||
// Queue erasure deleted this id's row; it isn't created again.
|
||||
: err.code === "qitem_erased" ? 409
|
||||
// OPR.0.4.6.MH3 FR-4 (C2): a cross-host source-close re-drive naming a
|
||||
// DIFFERENT closure_target than the recorded one — someone else closed
|
||||
// the source meanwhile; surfaced, never overwritten.
|
||||
@@ -654,6 +657,74 @@ export function queueRoutes(): Hono {
|
||||
}
|
||||
});
|
||||
|
||||
// POST /erase — queue erasure in one exact scope: account deletion (members, the account's recorded sessions),
|
||||
// explicit ids (a ledger's reapply after a restore), or the routine closed-work windows from this daemon's settings,
|
||||
// run now. A dry run unless apply is true. domain/queue-erasure.ts; docs/reference/queue-erasure.md.
|
||||
app.post("/erase", async (c) => {
|
||||
const body = await c.req.json<{ members?: unknown; rig?: unknown; ids?: unknown; action?: unknown; routine?: unknown; apply?: unknown; actorSession?: string }>()
|
||||
.catch(() => ({} as never));
|
||||
const identity = requireSenderIdentity(c, { verb: "queue erase", bodyClaim: body.actorSession });
|
||||
if (!identity.ok) return identity.response;
|
||||
const refuse = (error: string, message: string) => c.json({ error, message }, 400);
|
||||
const settings = c.get("settingsStore" as never) as { resolveOne(key: string): { value: unknown } } | undefined;
|
||||
const named = [body.members !== undefined, body.ids !== undefined, body.routine !== undefined].filter(Boolean).length;
|
||||
if (named === 0) return refuse("erase_scope_required", "Name one scope: members (account deletion), ids, or routine.");
|
||||
if (named > 1) return refuse("erase_scope_conflict", "Name exactly one scope: members, ids, or routine.");
|
||||
const exactList = (value: unknown): string[] | null =>
|
||||
Array.isArray(value) && value.length > 0 && value.every((entry) => typeof entry === "string" && entry.length > 0 && entry.trim() === entry)
|
||||
? value as string[] : null;
|
||||
let scope: EraseScope;
|
||||
if (body.rig !== undefined && (body.members === undefined || typeof body.rig !== "string" || body.rig.length === 0 || body.rig.trim() !== body.rig)) {
|
||||
return refuse("erase_rig_invalid", "rig is the account rig's id, given with members.");
|
||||
}
|
||||
if (body.members !== undefined) {
|
||||
const members = exactList(body.members);
|
||||
if (!members) return refuse("erase_members_invalid", "members must be a non-empty list of exact session names.");
|
||||
scope = { kind: "members", members, ...(typeof body.rig === "string" ? { rig: body.rig } : {}) };
|
||||
} else if (body.ids !== undefined) {
|
||||
// Plain ids take the request's action (delete by default); a report's qitems entries keep their own.
|
||||
const fallback = body.action ?? "delete";
|
||||
const isAction = (value: unknown): value is EraseAction => value === "delete" || value === "scrub";
|
||||
if (!isAction(fallback)) return refuse("erase_action_invalid", "action must be delete or scrub.");
|
||||
const raw = Array.isArray(body.ids) ? body.ids : [];
|
||||
const entries = raw.map((entry) => typeof entry === "string" ? { qitemId: entry, action: fallback as EraseAction }
|
||||
: entry && typeof entry === "object" ? { qitemId: (entry as { qitemId?: unknown }).qitemId, action: (entry as { action?: unknown }).action ?? fallback }
|
||||
: null);
|
||||
if (raw.length === 0 || entries.some((entry) => !entry || typeof entry.qitemId !== "string" || entry.qitemId.length === 0
|
||||
|| entry.qitemId.trim() !== entry.qitemId)) {
|
||||
return refuse("erase_ids_invalid", "ids must be a non-empty list of qitem ids, or of {qitemId, action} entries.");
|
||||
}
|
||||
if (entries.some((entry) => !isAction(entry!.action))) return refuse("erase_action_invalid", "action must be delete or scrub.");
|
||||
scope = { kind: "ids", entries: entries as Array<{ qitemId: string; action: EraseAction }> };
|
||||
} else {
|
||||
const days = (key: string): number => {
|
||||
const value = settings?.resolveOne(key).value;
|
||||
return typeof value === "number" ? value : 0;
|
||||
};
|
||||
const contentDays = days("retention.closed_content_days");
|
||||
const metadataDays = days("retention.closed_metadata_days");
|
||||
if (contentDays === 0 && metadataDays === 0) {
|
||||
return refuse("erase_routine_off", "retention.closed_content_days and retention.closed_metadata_days are both 0 (off).");
|
||||
}
|
||||
scope = { kind: "routine", contentDays, metadataDays };
|
||||
}
|
||||
try {
|
||||
const ledger = settings?.resolveOne("retention.erasure_ledger").value;
|
||||
const report = await eraseQueue(c.get("db" as never) as Database.Database, {
|
||||
scope, apply: body.apply === true, nowIso: new Date().toISOString(),
|
||||
...(typeof ledger === "string" && ledger.length > 0 ? { ledgerPath: ledger } : {}),
|
||||
});
|
||||
console.log(`[queue-erase] ${identity.session} ${scope.kind} ${report.applied ? "applied" : "dry run"}: `
|
||||
+ `${report.qitems.length} rows, ${report.mixed.length} mixed, ${report.gaps.length} gaps, complete=${report.complete}, `
|
||||
+ `clean=${report.clean}${report.error ? `, stopped at ${report.error.stage}: ${report.error.message}` : ""}`);
|
||||
// A run that stopped on an error still returns what it committed, so its ids reach the caller's ledger.
|
||||
return c.json(report, report.error ? 500 : 200);
|
||||
} catch (err) {
|
||||
if (err instanceof EraseRefusal) return refuse(err.code, err.message);
|
||||
return c.json({ error: "erase_failed", message: err instanceof Error ? err.message : String(err) }, 500);
|
||||
}
|
||||
});
|
||||
|
||||
// POST /:qitemId/claim
|
||||
app.post("/:qitemId/claim", async (c) => {
|
||||
const qitemId = c.req.param("qitemId");
|
||||
|
||||
@@ -1,202 +0,0 @@
|
||||
// The parked route reads its obligation face with QueueRepository.listOpenObligations (qitem id, state and summary)
|
||||
// instead of a full list() that derives pickup, waiting view and delivery ledger for every row it never shows.
|
||||
// The response must be the same, byte for byte, as the one the full list() produced. Real queue, rig, session and
|
||||
// identity repositories on a full test DB; the only difference between the two apps is the obligation reader.
|
||||
import { describe, it, expect, beforeEach, afterEach } from "vitest";
|
||||
import type Database from "better-sqlite3";
|
||||
import { Hono } from "hono";
|
||||
import { createFullTestDb } from "./helpers/test-app.js";
|
||||
import { activityRoutes } from "../src/routes/activity.js";
|
||||
import { RigRepository } from "../src/domain/rig-repository.js";
|
||||
import { SessionRegistry } from "../src/domain/session-registry.js";
|
||||
import { SeatIdentityStore } from "../src/domain/seat-identity-store.js";
|
||||
import { SeatActivityService } from "../src/domain/seat-activity-service.js";
|
||||
import { EventBus } from "../src/domain/event-bus.js";
|
||||
import { QueueRepository } from "../src/domain/queue-repository.js";
|
||||
import { WatchdogJobsRepository } from "../src/domain/watchdog-jobs-repository.js";
|
||||
import { WatchdogHistoryLog } from "../src/domain/watchdog-history-log.js";
|
||||
import { watchdogHistorySchema } from "../src/db/migrations/032_watchdog_history.js";
|
||||
import { queueItemSummarySchema } from "../src/db/migrations/044_queue_item_summary.js";
|
||||
|
||||
const RIG = "park-rig";
|
||||
const IDLE = "dev-impl@park-rig"; // idle at its prompt: parked whenever it owes work
|
||||
const UNKNOWN = "dev-qa@park-rig"; // no activity evidence: indeterminate
|
||||
const STOPPED = "dev-ops@park-rig"; // idle evidence, but its session is confirmed missing
|
||||
const ALIAS = "dev-old@park-rig"; // an earlier address of IDLE, resolved by resolveSessionAddresses
|
||||
|
||||
describe("GET /api/activity/parked reads the obligation face without the full list()", () => {
|
||||
let db: Database.Database;
|
||||
let queue: QueueRepository;
|
||||
let jobs: WatchdogJobsRepository;
|
||||
let history: WatchdogHistoryLog;
|
||||
let oracle: SeatActivityService;
|
||||
let clock = 0;
|
||||
|
||||
beforeEach(() => {
|
||||
db = createFullTestDb();
|
||||
db.exec(watchdogHistorySchema.sql); // the full test DB doesn't include the engine's audit table
|
||||
db.exec(queueItemSummarySchema.sql); // nor the summary column, which the obligation face carries
|
||||
const rigRepo = new RigRepository(db);
|
||||
const sessions = new SessionRegistry(db);
|
||||
const rig = rigRepo.createRig(RIG);
|
||||
rigRepo.createRig("empty-rig");
|
||||
const now = Date.parse("2026-11-07T09:00:00.000Z");
|
||||
oracle = new SeatActivityService({ tmux: { readPaneLastActivity: async () => null }, defaultWindowSeconds: 3, now: () => new Date(now) });
|
||||
for (const [logicalId, session] of [["dev.impl", IDLE], ["dev.qa", UNKNOWN], ["dev.ops", STOPPED]] as const) {
|
||||
const node = rigRepo.addNode(rig.id, logicalId, { runtime: "claude-code" });
|
||||
const registered = sessions.registerSession(node.id, session);
|
||||
sessions.updateStatus(registered.id, "running");
|
||||
if (session === UNKNOWN) continue;
|
||||
oracle.declareRungInventory({ seatNodeId: node.id, sessionName: session }, {
|
||||
adapterId: "claude-code-adapter", runtime: "claude-code",
|
||||
rungs: [{ rung: "window-sampling", lifecycleCoverage: "full", initialTrust: "authoritative" }],
|
||||
});
|
||||
oracle.reportEvidence({ seatNodeId: node.id, sessionName: session, rung: "window-sampling",
|
||||
sourceId: "tmux:window-activity", seq: 1, observedAt: new Date(now).toISOString(), activity: "idle-at-prompt" });
|
||||
if (session === STOPPED) {
|
||||
sessions.updateBinding(node.id, { tmuxSession: session, tmuxPane: "%7", attachmentType: "tmux" } as never);
|
||||
new SeatIdentityStore(db).upsert({
|
||||
nodeId: node.id, sessionName: session, verdict: "pane_missing", reason: "session_missing", evidenceSource: "tmux_session",
|
||||
evidence: { registeredPane: "%7", observedPid: null, observedCommand: null, matchedLayer: null },
|
||||
observedAt: new Date(Date.now() + 1000).toISOString(),
|
||||
} as never);
|
||||
}
|
||||
}
|
||||
const gen = (session: string) => sessions.currentOccupantGenerationForSession(session);
|
||||
queue = new QueueRepository(db, new EventBus(db), {
|
||||
validateRig: () => true,
|
||||
resolveOccupantGeneration: gen,
|
||||
resolveSessionAddresses: (name: string) => (name === IDLE ? [IDLE, ALIAS] : [name]),
|
||||
} as never);
|
||||
jobs = new WatchdogJobsRepository(db, undefined, gen);
|
||||
queue.attachWatchdogJobsRepository(jobs);
|
||||
history = new WatchdogHistoryLog(db);
|
||||
clock = 0;
|
||||
});
|
||||
afterEach(() => db.close());
|
||||
|
||||
function app(queueRepo: unknown): Hono {
|
||||
const a = new Hono();
|
||||
a.use("*", async (c, next) => {
|
||||
c.set("seatActivityService" as never, oracle as never);
|
||||
c.set("queueRepo" as never, queueRepo as never);
|
||||
c.set("rigRepo" as never, { db } as never);
|
||||
await next();
|
||||
});
|
||||
a.route("/api/activity", activityRoutes);
|
||||
return a;
|
||||
}
|
||||
|
||||
/** The obligation reader the route used before: the full list() of the destination's open states, mapped down. */
|
||||
function fullListFace() {
|
||||
return {
|
||||
listOpenObligations: (destinationSession: string, limit: number) =>
|
||||
queue.list({ destinationSession, state: ["pending", "in-progress", "blocked"], limit })
|
||||
.map((r) => ({ qitemId: r.qitemId, state: r.state, summary: r.summary ?? null })),
|
||||
getParkWakeStatus: (qitemId: string) => queue.getParkWakeStatus(qitemId),
|
||||
parkWakeServedByOwnReminder: (qitemId: string) => queue.parkWakeServedByOwnReminder(qitemId),
|
||||
};
|
||||
}
|
||||
|
||||
/** Both responses, as bytes; they must be equal. Returns the parsed body for the case's own assertions. */
|
||||
async function sameResponse(path: string): Promise<{ status: number; body: any }> {
|
||||
const lean = await app(queue).request(path);
|
||||
const full = await app(fullListFace()).request(path);
|
||||
const leanText = await lean.text();
|
||||
expect(lean.status).toBe(full.status);
|
||||
expect(leanText).toBe(await full.text());
|
||||
return { status: lean.status, body: JSON.parse(leanText) };
|
||||
}
|
||||
|
||||
async function row(destinationSession = IDLE, summary = "owed"): Promise<string> {
|
||||
return (await queue.create({ sourceSession: "orch-lead@park-rig", destinationSession, body: "work", summary, nudge: false } as never)).qitemId;
|
||||
}
|
||||
function reminder(target = IDLE): string {
|
||||
return jobs.register({
|
||||
policy: "periodic-reminder",
|
||||
specYaml: ["policy: periodic-reminder", "target:", ` session: ${JSON.stringify(target)}`, "message: \"check your rows\"", ""].join("\n"),
|
||||
targetSession: target, intervalSeconds: 300, registeredBySession: target,
|
||||
} as never).jobId;
|
||||
}
|
||||
function park(qitemId: string, watchdogId: string) {
|
||||
queue.update({ qitemId, actorSession: IDLE, state: "blocked", blockedOn: "external:ci", transitionNote: "continuation: check CI",
|
||||
wakeWatchdogId: watchdogId } as never);
|
||||
}
|
||||
function fire(jobId: string) {
|
||||
const evaluatedAt = new Date(Date.UTC(2026, 10, 7, 3, 0, clock++)).toISOString();
|
||||
history.record({ jobId, evaluatedAt, outcome: "sent", deliveryStatus: "ok", deliveryTargetSession: IDLE } as never);
|
||||
queue.recordWatchdogWakeAttempt(jobId, "ok");
|
||||
}
|
||||
|
||||
it("0 rows: the rig and the seat read the same, and the board is clean", async () => {
|
||||
const rig = await sameResponse(`/api/activity/parked?rig=${RIG}`);
|
||||
expect(rig.status).toBe(200);
|
||||
expect(rig.body.rig.seats.find((s: { sessionName: string }) => s.sessionName === IDLE).obligations.openCount).toBe(0);
|
||||
const seat = await sameResponse(`/api/activity/parked?seat=${encodeURIComponent(IDLE)}`);
|
||||
expect(seat.body.seat).toMatchObject({ parked: false, confidence: { obligations: "complete" } });
|
||||
});
|
||||
|
||||
it("1 row: the same parked verdict, item and summary", async () => {
|
||||
const q = await row(IDLE, "the one owed row");
|
||||
const { body } = await sameResponse(`/api/activity/parked?seat=${encodeURIComponent(IDLE)}`);
|
||||
expect(body.seat.parked).toBe(true);
|
||||
expect(body.seat.obligations.items).toEqual([{ qitemId: q, state: "pending", summary: "the one owed row" }]);
|
||||
});
|
||||
|
||||
it("more than the limit (500): the same 500 rows, newest first, and truncation-possible; ties on ts_created included", async () => {
|
||||
const insert = db.prepare(`INSERT INTO queue_items (qitem_id, ts_created, ts_updated, source_session, destination_session, state, body, summary)
|
||||
VALUES (?, ?, ?, 'orch-lead@park-rig', ?, ?, 'work', ?)`);
|
||||
const states = ["pending", "in-progress", "blocked"];
|
||||
db.transaction(() => {
|
||||
for (let i = 0; i < 510; i++) {
|
||||
// Two rows per millisecond, so the ORDER BY ts_created DESC window has ties at its edge.
|
||||
const ts = new Date(Date.UTC(2026, 10, 1, 0, 0, 0, Math.floor(i / 2))).toISOString();
|
||||
insert.run(`qitem-bulk-${String(i).padStart(4, "0")}`, ts, ts, i % 7 === 0 ? ALIAS : IDLE, states[i % 3], i % 5 === 0 ? null : `row ${i}`);
|
||||
}
|
||||
})();
|
||||
const { body } = await sameResponse(`/api/activity/parked?seat=${encodeURIComponent(IDLE)}`);
|
||||
expect(body.seat.obligations.openCount + body.seat.obligations.heldCount).toBe(500);
|
||||
expect(body.seat.confidence.obligations).toBe("truncation-possible");
|
||||
expect(body.seat.obligations.complete).toBe(false);
|
||||
});
|
||||
|
||||
it("a held row with a live wake reads healthy on both", async () => {
|
||||
const q = await row();
|
||||
park(q, reminder());
|
||||
const { body } = await sameResponse(`/api/activity/parked?seat=${encodeURIComponent(IDLE)}`);
|
||||
expect(body.seat.obligations.held[0]).toMatchObject({ qitemId: q, healthy: true, wake: { live: true } });
|
||||
});
|
||||
|
||||
it("a held row served by its owner's own reminder reads the same", async () => {
|
||||
const q = await row();
|
||||
const job = reminder();
|
||||
park(q, job);
|
||||
fire(job);
|
||||
const { body } = await sameResponse(`/api/activity/parked?seat=${encodeURIComponent(IDLE)}`);
|
||||
expect(body.seat.obligations.held[0]).toMatchObject({ qitemId: q, healthy: true, servedByOwnReminder: true });
|
||||
});
|
||||
|
||||
it("an indeterminate seat (no activity evidence) reads the same", async () => {
|
||||
await row(UNKNOWN);
|
||||
const { body } = await sameResponse(`/api/activity/parked?rig=${RIG}`);
|
||||
expect(body.rig.seats.find((s: { sessionName: string }) => s.sessionName === UNKNOWN).parked).toBe("indeterminate");
|
||||
});
|
||||
|
||||
it("a seat confirmed stopped reads the same (indeterminate despite its idle evidence)", async () => {
|
||||
await row(STOPPED);
|
||||
const { body } = await sameResponse(`/api/activity/parked?seat=${encodeURIComponent(STOPPED)}`);
|
||||
expect(body.seat.parked).toBe("indeterminate");
|
||||
});
|
||||
|
||||
it("a rig without seats reads the same", async () => {
|
||||
const { status, body } = await sameResponse("/api/activity/parked?rig=empty-rig");
|
||||
expect(status).toBe(200);
|
||||
expect(body.rig).toMatchObject({ parked: false, seats: [] });
|
||||
});
|
||||
|
||||
it("rows addressed to an alias of the seat are found the same way (resolveSessionAddresses)", async () => {
|
||||
const q = await row(ALIAS, "sent to the old address");
|
||||
const { body } = await sameResponse(`/api/activity/parked?seat=${encodeURIComponent(IDLE)}`);
|
||||
expect(body.seat.obligations.items).toEqual([{ qitemId: q, state: "pending", summary: "sent to the old address" }]);
|
||||
});
|
||||
});
|
||||
@@ -51,8 +51,8 @@ describe("Wave-O B2 — parked diagnosis is RIG-SCOPED with the scope named", ()
|
||||
|
||||
// The obligation face: ONLY dev-b@rig-b owes work.
|
||||
const queueRepo = {
|
||||
listOpenObligations: (destinationSession: string) =>
|
||||
destinationSession === "dev-b@rig-b"
|
||||
list: (opts: { destinationSession?: string }) =>
|
||||
opts.destinationSession === "dev-b@rig-b"
|
||||
? [{ qitemId: "qitem-b-1", state: "pending", summary: "owed in rig-b" }]
|
||||
: [],
|
||||
};
|
||||
|
||||
@@ -30,7 +30,7 @@ function makeApp(opts: { withDeps: boolean; rows?: Array<{ qitemId: string; stat
|
||||
app.use("*", async (c, next) => {
|
||||
if (opts.withDeps) {
|
||||
c.set("seatActivityService" as never, svc as never);
|
||||
c.set("queueRepo" as never, { listOpenObligations: () => opts.rows ?? [] } as never);
|
||||
c.set("queueRepo" as never, { list: () => opts.rows ?? [] } as never);
|
||||
c.set("rigRepo" as never, { db: { prepare: (sql: string) => ({
|
||||
get: () => ({ id: "rig-1", name: "v-openrig-build" }),
|
||||
all: () => (sql.includes("SELECT name FROM rigs") ? [] : [{ node_id: SEAT, session_name: SESSION }]),
|
||||
|
||||
@@ -2,7 +2,7 @@ import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
|
||||
import { mkdirSync, writeFileSync, readFileSync, existsSync, rmSync, mkdtempSync, copyFileSync, chmodSync, statSync, readdirSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { execFileSync, spawn, spawnSync } from "node:child_process";
|
||||
import { execSync, spawn, spawnSync } from "node:child_process";
|
||||
import { ClaudeCodeAdapter } from "../src/adapters/claude-code-adapter.js";
|
||||
import { shellQuote } from "../src/adapters/shell-quote.js";
|
||||
import { replaceFile } from "../src/adapters/replace-file.js";
|
||||
@@ -100,7 +100,7 @@ describe("Claude Status Line Collector Script", () => {
|
||||
// T1: Parses valid status line and writes sidecar
|
||||
it("parses valid Claude status line JSON and writes sidecar file", () => {
|
||||
const outputPath = join(tmpDir, "context", "test.json");
|
||||
execFileSync(process.execPath, [collectorPath, outputPath], { input: VALID_STATUS_LINE, encoding: "utf-8" });
|
||||
execSync(`echo '${VALID_STATUS_LINE}' | node ${collectorPath} ${outputPath}`, { encoding: "utf-8" });
|
||||
|
||||
expect(existsSync(outputPath)).toBe(true);
|
||||
const content = JSON.parse(readFileSync(outputPath, "utf-8"));
|
||||
@@ -128,7 +128,7 @@ describe("Claude Status Line Collector Script", () => {
|
||||
// T2: Atomic write (no .tmp left behind on success)
|
||||
it("writes atomically — no .tmp file left behind", () => {
|
||||
const outputPath = join(tmpDir, "context", "atomic.json");
|
||||
execFileSync(process.execPath, [collectorPath, outputPath], { input: VALID_STATUS_LINE, encoding: "utf-8" });
|
||||
execSync(`echo '${VALID_STATUS_LINE}' | node ${collectorPath} ${outputPath}`, { encoding: "utf-8" });
|
||||
|
||||
expect(existsSync(outputPath)).toBe(true);
|
||||
expect(existsSync(outputPath + ".tmp")).toBe(false);
|
||||
@@ -165,7 +165,7 @@ describe("Claude Status Line Collector Script", () => {
|
||||
|
||||
it("writes to a session-specific sidecar when given a context directory", () => {
|
||||
const outputDir = join(tmpDir, "context");
|
||||
execFileSync(process.execPath, [collectorPath, outputDir], { input: VALID_STATUS_LINE, encoding: "utf-8" });
|
||||
execSync(`echo '${VALID_STATUS_LINE}' | node ${collectorPath} ${outputDir}`, { encoding: "utf-8" });
|
||||
|
||||
const sessionSidecar = join(outputDir, "[email protected]");
|
||||
expect(existsSync(sessionSidecar)).toBe(true);
|
||||
@@ -236,7 +236,7 @@ describe("Claude Status Line Collector Script", () => {
|
||||
// T3b: No output path arg — silent exit
|
||||
it("silently exits with no output path argument", () => {
|
||||
// Should not throw
|
||||
execFileSync(process.execPath, [collectorPath], { input: VALID_STATUS_LINE, encoding: "utf-8" });
|
||||
execSync(`echo '${VALID_STATUS_LINE}' | node ${collectorPath}`, { encoding: "utf-8" });
|
||||
});
|
||||
|
||||
it("logs a shape error when a directory target cannot be resolved to a session sidecar", () => {
|
||||
|
||||
@@ -30,7 +30,7 @@ it("keeps a visibly running Codex turn working when the window timestamp is quie
|
||||
const app = new Hono();
|
||||
app.use("*", async (c, next) => {
|
||||
c.set("seatActivityService" as never, oracle as never);
|
||||
c.set("queueRepo" as never, { listOpenObligations: () => [{ qitemId: "work", state: "in-progress", summary: "Work" }] } as never);
|
||||
c.set("queueRepo" as never, { list: () => [{ qitemId: "work", state: "in-progress", summary: "Work" }] } as never);
|
||||
c.set("rigRepo" as never, { db: { prepare: (sql: string) => ({
|
||||
get: () => ({ id: "rig-1", name: "fixture" }),
|
||||
all: () => sql.includes("SELECT name FROM rigs") ? [] : [{ node_id: node, session_name: session }],
|
||||
|
||||
@@ -4,7 +4,6 @@ import { cpSync, mkdirSync, readFileSync, readdirSync, rmSync, writeFileSync } f
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { prepareHermeticEnv, type HermeticScaffold } from "./helpers/hermetic-env.js";
|
||||
import { shellQuote } from "../src/adapters/shell-quote.js";
|
||||
import { runRig, spawnScenarioDaemon, type ScenarioDaemon } from "./helpers/scenario-daemon.js";
|
||||
|
||||
// One community harness end to end, through the real CLI, daemon, SQLite and a private tmux server: a synthetic
|
||||
@@ -12,7 +11,7 @@ import { runRig, spawnScenarioDaemon, type ScenarioDaemon } from "./helpers/scen
|
||||
// preset and a built-in name collision. Nothing here runs a real community harness or touches a real home.
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const rigBin = resolve(here, "../../cli/dist/bin-wrapper.js");
|
||||
const appFixture = resolve(here, "fixtures/community-harness/synth-harness.mjs");
|
||||
const appScript = resolve(here, "fixtures/community-harness/synth-harness.mjs");
|
||||
const agents = resolve(here, "../../test-system/scenarios/agents");
|
||||
const session = "work-worker@community-harness-rig";
|
||||
const sleep = (ms: number) => new Promise(r => setTimeout(r, ms));
|
||||
@@ -21,7 +20,6 @@ describe("a community harness from a data preset", () => {
|
||||
let scaffold: HermeticScaffold;
|
||||
let daemon: ScenarioDaemon;
|
||||
let rigRoot: string;
|
||||
let appScript: string;
|
||||
let rigId: string;
|
||||
|
||||
const rig = (args: string[], timeout?: number) => runRig(args, daemon.readEnv, rigBin, timeout);
|
||||
@@ -53,11 +51,9 @@ describe("a community harness from a data preset", () => {
|
||||
scaffold = prepareHermeticEnv({ baseEnv: { PATH: process.env.PATH, TERM: "xterm-256color" } });
|
||||
rigRoot = join(scaffold.root, "rig");
|
||||
mkdirSync(rigRoot);
|
||||
appScript = join(rigRoot, "synth #harness.mjs");
|
||||
cpSync(appFixture, appScript);
|
||||
cpSync(agents, join(rigRoot, "agents"), { recursive: true });
|
||||
writePreset("synth-harness.yaml", [
|
||||
"format: 1", "name: synth-harness", `command: ${JSON.stringify(`node ${shellQuote(appScript)}`)}`, "maintainer: Example Maintainer",
|
||||
"format: 1", "name: synth-harness", `command: node ${appScript}`, "maintainer: Example Maintainer",
|
||||
"limits:", " - resume unsupported", " - fork unsupported", " - no approval-prompt detection", "",
|
||||
].join("\n"));
|
||||
const previousDir = join(scaffold.env.OPENRIG_HOME!, ["agent", "apps"].join("-"));
|
||||
@@ -88,7 +84,7 @@ exec "$@"
|
||||
}
|
||||
|
||||
it("prefixes a runner-less preset with resolved literal wrapper argv", async () => {
|
||||
writePreset("wrapped-plain.yaml", `format: 1\nname: wrapped-plain\ncommand: ${JSON.stringify(`node ${shellQuote(appScript)}`)}\nmaintainer: Example\n`);
|
||||
writePreset("wrapped-plain.yaml", `format: 1\nname: wrapped-plain\ncommand: node ${appScript}\nmaintainer: Example\n`);
|
||||
const wrapper = recordingWrapper("plain");
|
||||
const up = await rig(["up", specFor("wrapped-plain-rig", "wrapped-plain", wrapper.argv), "--cwd", rigRoot, "--json", "--yes"], 120_000);
|
||||
expect(up.code, JSON.stringify(up)).toBe(0);
|
||||
@@ -99,7 +95,7 @@ exec "$@"
|
||||
it("keeps a capable runner outside the wrapper and wraps its real native child", async () => {
|
||||
const runner = resolve(here, "fixtures/runner-conformance/minimal-acp-runner.mjs");
|
||||
const agent = resolve(here, "../../../docs/reference/runner-conformance/fake-acp-agent.mjs");
|
||||
writePreset("wrapped-runner.yaml", `format: 1\nname: wrapped-runner\ncommand: ${JSON.stringify(`node ${shellQuote(runner)} --agent ${shellQuote(`node ${shellQuote(agent)}`)}`)}\nmaintainer: Example\nrunner: true\nlaunch_result: true\nlaunch_wrapper: true\n`);
|
||||
writePreset("wrapped-runner.yaml", `format: 1\nname: wrapped-runner\ncommand: node ${runner} --agent 'node ${agent}'\nmaintainer: Example\nrunner: true\nlaunch_result: true\nlaunch_wrapper: true\n`);
|
||||
const wrapper = recordingWrapper("runner");
|
||||
const up = await rig(["up", specFor("wrapped-runner-rig", "wrapped-runner", wrapper.argv), "--cwd", rigRoot, "--json", "--yes"], 120_000);
|
||||
expect(up.code, JSON.stringify(up)).toBe(0);
|
||||
@@ -116,7 +112,7 @@ exec "$@"
|
||||
const marker = join(rigRoot, "incapable-started");
|
||||
const launcher = join(rigRoot, "incapable.mjs");
|
||||
writeFileSync(launcher, `import { writeFileSync } from 'node:fs'; writeFileSync(${JSON.stringify(marker)}, 'started');`);
|
||||
writePreset("incapable-runner.yaml", `format: 1\nname: incapable-runner\ncommand: ${JSON.stringify(`node ${shellQuote(launcher)}`)}\nmaintainer: Example\nrunner: true\nlaunch_result: true\n${capability}`);
|
||||
writePreset("incapable-runner.yaml", `format: 1\nname: incapable-runner\ncommand: node ${launcher}\nmaintainer: Example\nrunner: true\nlaunch_result: true\n${capability}`);
|
||||
const wrapper = recordingWrapper("incapable");
|
||||
const up = await rig(["up", specFor("incapable-runner-rig", "incapable-runner", wrapper.argv), "--cwd", rigRoot, "--json", "--yes"], 120_000);
|
||||
expect(up.code, JSON.stringify(up)).not.toBe(0);
|
||||
@@ -152,7 +148,7 @@ exec "$@"
|
||||
});
|
||||
|
||||
it("hands the composed first turn to a real launcher without typing it into its pane", async () => {
|
||||
writePreset("synth-first.yaml", `format: 1\nname: synth-first\ncommand: ${JSON.stringify(`node ${shellQuote(appScript)}`)}\nmaintainer: Example\nfirst_input: true\n`);
|
||||
writePreset("synth-first.yaml", `format: 1\nname: synth-first\ncommand: node ${appScript}\nmaintainer: Example\nfirst_input: true\n`);
|
||||
const up = await rig(["up", specFor("first-input-rig", "synth-first"), "--cwd", rigRoot, "--json", "--yes"], 120_000);
|
||||
expect(up.code, JSON.stringify(up)).toBe(0);
|
||||
const inputDir = join(scaffold.env.OPENRIG_HOME!, "state", "first-input", "work-worker%40first-input-rig");
|
||||
@@ -326,7 +322,7 @@ exec "$@"
|
||||
expect(seat).not.toHaveProperty("communityHarness");
|
||||
};
|
||||
await check();
|
||||
writePreset("free-harness.yaml", ["format: 1", "name: free-harness", `command: ${JSON.stringify(`node ${shellQuote(appScript)}`)}`, "maintainer: Example Maintainer", ""].join("\n"));
|
||||
writePreset("free-harness.yaml", ["format: 1", "name: free-harness", `command: node ${appScript}`, "maintainer: Example Maintainer", ""].join("\n"));
|
||||
await sleep(4_000);
|
||||
await check();
|
||||
// A relaunch doesn't start the same-named app in its place.
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { piAccountSwitchesSchema } from "../../src/db/migrations/151_pi_account_switches.js";
|
||||
import { eventsRetentionMarksSchema } from "../../src/db/migrations/153_events_retention_marks.js";
|
||||
import { queueErasureSchema } from "../../src/db/migrations/155_queue_erasure.js";
|
||||
import { activityGrantsSchema } from "../../src/db/migrations/143_activity_grants.js";
|
||||
import { environmentBindingsSchema } from "../../src/db/migrations/144_environment_bindings.js";
|
||||
import { nodePermissionsToSchema } from "../../src/db/migrations/145_node_permissions_to.js";
|
||||
@@ -162,7 +163,7 @@ import fs from "node:fs";
|
||||
|
||||
/** Seam B R6: the canonical full-fixture migration list, exported so file-backed
|
||||
* DB-reopen tests migrate IDENTICALLY to createFullTestDb. */
|
||||
export const migrationsForFullTestDb = [coreSchema, bindingsSessionsSchema, eventsSchema, snapshotsSchema, checkpointsSchema, resumeMetadataSchema, nodeSpecFieldsSchema, packagesSchema, installJournalSchema, journalSeqSchema, bootstrapSchema, discoverySchema, discoveryFkFix, agentspecRebootSchema, startupContextSchema, chatMessagesSchema, podNamespaceSchema, contextUsageSchema, externalCliAttachmentSchema, rigServicesSchema, seatHandoverObservabilitySchema, nodeCodexConfigProfileSchema, nodePermissionPolicySchema, rigPermissionPolicySchema, nodePolicyProvenanceSchema, rigPolicyProvenanceSchema, streamItemsSchema, queueItemsSchema, queueTransitionsSchema, rigPolicySchema, rigArchiveSchema, resumeProvenanceSchema, resumeVerificationSchema, seatIdentityVerdictsSchema, selfHostIdentitySchema, occupantTenuresSchema, daemonLifecycleSchema, watchdogJobsSchema, occupantGenerationStampsSchema, projectionManifestSchema, watchdogTargetGenerationSchema, appliedLaunchObservationsSchema, appliedLaunchObservationInvalidationsSchema, threadSeatMapSchema, queueTransitionWakesSchema, nodeSessionSourceSchema, scopedOperatingPostureSchema, humanNotificationIntentSchema, reviewReadIndexesSchema, inventoryEventIndexesSchema, rigClaudeManagedBlockFileSchema, nodePermissionSelectionsSchema, humanReplyToSchema, humanQuestionsSchema, nodeEffortSchema, rigNonInterruptiveSchema, rigInstallRootSchema, threadPartMapSchema, queueTransitionsQitemIdOrderSchema, queueItemsRecoveryTagsSchema, resumeRotationSchema, sessionStopIntentSchema, nodeEffortChoiceSchema, modePostureCoexistSchema, seatEnvSchema, appliedLaunchDaemonOptOutSchema, sessionCommunityHarnessSchema, threadSeatMapChannelKeySchema, queueItemWakeInputBoxSchema, providerAccountReadingsSchema, humanQuietWindowsSchema, seatIdentityPanePidSchema, seatIdentityProcessEvidenceSchema, rigStartupLayersSchema, humanOperatingModesSchema, queueDelegatedDecisionSchema, nodeAdvisorSchema, operationCorrelationSchema, terminalPolicySchema, seatRuntimeSwitchesSchema, claudeTokenReadingsSchema, nodeLaunchRecipeSchema, nodeClaudeTokenRefSchema, nodeClaudePermissionsSchema, nodeCodexPermissionsSchema, escalationTargetRefsSchema, sessionAgentRunnerSchema, nodeAccountHomeSchema, nodeClaudeMcpSchema, activityGrantsSchema, environmentBindingsSchema, nodePermissionsToSchema, nodeStartupGuidanceFileSchema, managedBlockWritesSchema, streamIntakeCursorSchema, nodeLaunchWrapperSchema, piAccountSwitchesSchema, eventsRetentionMarksSchema];
|
||||
export const migrationsForFullTestDb = [coreSchema, bindingsSessionsSchema, eventsSchema, snapshotsSchema, checkpointsSchema, resumeMetadataSchema, nodeSpecFieldsSchema, packagesSchema, installJournalSchema, journalSeqSchema, bootstrapSchema, discoverySchema, discoveryFkFix, agentspecRebootSchema, startupContextSchema, chatMessagesSchema, podNamespaceSchema, contextUsageSchema, externalCliAttachmentSchema, rigServicesSchema, seatHandoverObservabilitySchema, nodeCodexConfigProfileSchema, nodePermissionPolicySchema, rigPermissionPolicySchema, nodePolicyProvenanceSchema, rigPolicyProvenanceSchema, streamItemsSchema, queueItemsSchema, queueTransitionsSchema, rigPolicySchema, rigArchiveSchema, resumeProvenanceSchema, resumeVerificationSchema, seatIdentityVerdictsSchema, selfHostIdentitySchema, occupantTenuresSchema, daemonLifecycleSchema, watchdogJobsSchema, occupantGenerationStampsSchema, projectionManifestSchema, watchdogTargetGenerationSchema, appliedLaunchObservationsSchema, appliedLaunchObservationInvalidationsSchema, threadSeatMapSchema, queueTransitionWakesSchema, nodeSessionSourceSchema, scopedOperatingPostureSchema, humanNotificationIntentSchema, reviewReadIndexesSchema, inventoryEventIndexesSchema, rigClaudeManagedBlockFileSchema, nodePermissionSelectionsSchema, humanReplyToSchema, humanQuestionsSchema, nodeEffortSchema, rigNonInterruptiveSchema, rigInstallRootSchema, threadPartMapSchema, queueTransitionsQitemIdOrderSchema, queueItemsRecoveryTagsSchema, resumeRotationSchema, sessionStopIntentSchema, nodeEffortChoiceSchema, modePostureCoexistSchema, seatEnvSchema, appliedLaunchDaemonOptOutSchema, sessionCommunityHarnessSchema, threadSeatMapChannelKeySchema, queueItemWakeInputBoxSchema, providerAccountReadingsSchema, humanQuietWindowsSchema, seatIdentityPanePidSchema, seatIdentityProcessEvidenceSchema, rigStartupLayersSchema, humanOperatingModesSchema, queueDelegatedDecisionSchema, nodeAdvisorSchema, operationCorrelationSchema, terminalPolicySchema, seatRuntimeSwitchesSchema, claudeTokenReadingsSchema, nodeLaunchRecipeSchema, nodeClaudeTokenRefSchema, nodeClaudePermissionsSchema, nodeCodexPermissionsSchema, escalationTargetRefsSchema, sessionAgentRunnerSchema, nodeAccountHomeSchema, nodeClaudeMcpSchema, activityGrantsSchema, environmentBindingsSchema, nodePermissionsToSchema, nodeStartupGuidanceFileSchema, managedBlockWritesSchema, streamIntakeCursorSchema, nodeLaunchWrapperSchema, piAccountSwitchesSchema, eventsRetentionMarksSchema, queueErasureSchema];
|
||||
|
||||
/**
|
||||
* P24 — the DECLARED exclusions for {@link migrationsForFullTestDb}. That list is deliberately a
|
||||
@@ -179,6 +180,7 @@ export const migrationsForFullTestDb = [coreSchema, bindingsSessionsSchema, even
|
||||
* its exclusion); the guard's redundant-exclusion check then keeps the two in sync.
|
||||
*/
|
||||
export const migrationsForFullTestDbExclusions: Record<string, string> = {
|
||||
"156_queue_erasure_link_indexes.sql": "indexes outbox_entries, inbox_entries and the stream tables (all excluded from the core edge) for queue erasure's lookups; erasure tests migrate the canonical full schema.",
|
||||
"082_archive_identity_provenance.sql": "archive extension; 054 is excluded from this core fixture. Archive identity regressions use ALL_MIGRATIONS.",
|
||||
"026_inbox_entries.sql": "inbox subsystem table — not on the shared core edge (inbox suites migrate it inline).",
|
||||
"027_outbox_entries.sql": "outbox subsystem table — not on the shared core edge (outbox suites migrate it inline).",
|
||||
|
||||
@@ -254,6 +254,7 @@ describe("validateMarkdownAddressability — the compose gate", () => {
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe("Markdown block boundaries in context addresses", () => {
|
||||
it("keeps inline backtick spans from swallowing later sections", () => {
|
||||
const text = "## Setup\n```literal ` backticks```\n## Deploy\ndeploy instructions\n";
|
||||
@@ -279,25 +280,3 @@ describe("Markdown block boundaries in context addresses", () => {
|
||||
expect(resolveAddress(text.replace("\n#\n", "\n# Named\n"), ["independent"]).text).toContain("independent instructions");
|
||||
});
|
||||
});
|
||||
|
||||
describe("section boundaries in large documents", () => {
|
||||
it("keeps sibling spans independent", () => {
|
||||
const blocks = Array.from({ length: 10000 }, (_, i) => `## Section ${i}\nbody ${i}\n`);
|
||||
const sections = parseMarkdownSections(blocks.join(""));
|
||||
expect(sections).toHaveLength(blocks.length);
|
||||
sections.forEach((section, i) => {
|
||||
const expected = i === blocks.length - 1 ? blocks[i] : blocks[i]!.trimEnd();
|
||||
expect(section.text).toBe(expected);
|
||||
expect(section.ownText).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
it("ends nested spans at the nearest same-or-higher heading, including H1", () => {
|
||||
const text = "## Parent\nintro\n### Child\nchild\n#### Detail\ndetail\n### Sibling\nsibling\n# Boundary\nend";
|
||||
expect(parseMarkdownSections(text).map(({ title, text, ownText }) => ({ title, text, ownText }))).toEqual([
|
||||
{ title: "Parent", text: "## Parent\nintro\n### Child\nchild\n#### Detail\ndetail\n### Sibling\nsibling", ownText: "## Parent\nintro" },
|
||||
{ title: "Child", text: "### Child\nchild\n#### Detail\ndetail", ownText: "### Child\nchild" },
|
||||
{ title: "Sibling", text: "### Sibling\nsibling", ownText: "### Sibling\nsibling" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -81,18 +81,6 @@ describe("attributed proof judgments and derived readiness", () => {
|
||||
expect(files.map(p => fs.readFileSync(p, "utf8"))).toEqual(before);
|
||||
expect(fs.readFileSync(join(f.alpha, "proof/judgments/00000001.md"), "utf8")).toContain(a.judgment.id);
|
||||
});
|
||||
it("retains judgment identity and correction history after CRLF conversion", () => {
|
||||
const f = fixture(), accepted = f.judge();
|
||||
const receipt = join(f.alpha, "proof/judgments/00000001.md");
|
||||
f.write(receipt, fs.readFileSync(receipt, "utf8").replace(/\n/g, "\r\n"));
|
||||
const current = readSliceReadiness(f.alpha);
|
||||
expect(current).toEqual(accepted.readiness);
|
||||
const correction = f.judge(f.input(undefined, "reject"));
|
||||
expect(correction.judgment.previous).toBe(accepted.judgment.id);
|
||||
expect(correction.readiness.items[0]!.state).toBe("rejected");
|
||||
f.write(receipt, fs.readFileSync(receipt, "utf8").replace("verdict: accept", "verdict: reject"));
|
||||
expect(readSliceReadiness(f.alpha).issues.join()).toContain("changed judgment");
|
||||
});
|
||||
it("rejects unauthorized actors, wrong revisions and missing evidence without a receipt", () => {
|
||||
const f = fixture();
|
||||
expect(() => f.judge(f.input(), "other@trial")).toThrow("not a judge");
|
||||
|
||||
@@ -0,0 +1,670 @@
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { Hono } from "hono";
|
||||
import Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { ALL_MIGRATIONS } from "../src/db/all-migrations.js";
|
||||
import { EventBus } from "../src/domain/event-bus.js";
|
||||
import { OutboxHandler } from "../src/domain/outbox-handler.js";
|
||||
import { InboxHandler } from "../src/domain/inbox-handler.js";
|
||||
import { QueueRepository, QueueRepositoryError } from "../src/domain/queue-repository.js";
|
||||
import { QueueTransitionLog } from "../src/domain/queue-transition-log.js";
|
||||
import { EraseRefusal, eraseQueue } from "../src/domain/queue-erasure.js";
|
||||
import { runQueueRetentionSweep } from "../src/domain/queue-retention.js";
|
||||
import { queueRoutes } from "../src/routes/queue.js";
|
||||
import { recordReaderAction } from "../src/domain/stream-reader-actions.js";
|
||||
import { mkdtempSync, readFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
const NOW = "2026-11-09T12:00:00.000Z";
|
||||
const ago = (days: number) => new Date(Date.parse(NOW) - days * 86_400_000).toISOString();
|
||||
const A = "a-one@n-acct";
|
||||
const B = "b-two@n-acct";
|
||||
const OUT = "x-other@n-else";
|
||||
|
||||
let db: Database.Database;
|
||||
let repo: QueueRepository;
|
||||
let log: QueueTransitionLog;
|
||||
function open(handle: Database.Database) {
|
||||
db = handle;
|
||||
repo = new QueueRepository(db, new EventBus(db));
|
||||
repo.attachOutbox(new OutboxHandler(db));
|
||||
log = new QueueTransitionLog(db);
|
||||
}
|
||||
beforeEach(() => {
|
||||
const handle = createDb();
|
||||
migrate(handle, ALL_MIGRATIONS);
|
||||
open(handle);
|
||||
});
|
||||
afterEach(() => db.close());
|
||||
|
||||
/** A row whose every content field and linked copy carries `secret-<id>`, with its history at chosen ages. */
|
||||
async function row(id: string, source: string, destination: string, history: Array<{ state: string; daysAgo: number }> = []) {
|
||||
const secret = `secret-${id}`;
|
||||
await repo.create({ qitemId: id, sourceSession: source, destinationSession: destination, body: `${secret} body`, summary: `${secret} summary`,
|
||||
tags: [`${secret}-tag`] });
|
||||
db.prepare("UPDATE queue_transitions SET ts = ?, transition_note = ? WHERE qitem_id = ?").run(ago(400), `${secret} created`, id);
|
||||
for (const step of history) {
|
||||
const { transitionId } = log.append({ qitemId: id, state: step.state, actorSession: destination, transitionNote: `${secret} note` });
|
||||
db.prepare("UPDATE queue_transitions SET ts = ? WHERE transition_id = ?").run(ago(step.daysAgo), transitionId);
|
||||
}
|
||||
const state = history.at(-1)?.state ?? "pending";
|
||||
db.prepare("UPDATE queue_items SET state = ?, resolution = ?, ts_updated = ? WHERE qitem_id = ?")
|
||||
.run(state, `${secret} resolution`, ago(history.at(-1)?.daysAgo ?? 400), id);
|
||||
// The create staged the row's wake intent (its body carries the summary); make sure one exists either way.
|
||||
db.prepare(`INSERT OR IGNORE INTO outbox_entries (outbox_id, sender_session, destination_session, body, ts_dispatched, audit_pointer)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`).run(`wake-intent-${id}`, source, destination, `${secret} wake`, NOW, id);
|
||||
db.prepare(`INSERT INTO inbox_entries (inbox_id, destination_session, sender_session, body, ts_dropped, state, absorbed_qitem_id)
|
||||
VALUES (?, ?, ?, ?, ?, 'absorbed', ?)`).run(`inbox-${id}`, destination, source, `${secret} inbox`, NOW, id);
|
||||
db.prepare(`INSERT INTO mission_control_actions (action_id, action_verb, qitem_id, actor_session, acted_at, reason)
|
||||
VALUES (?, 'approve', ?, ?, ?, ?)`).run(`mc-${id}`, id, destination, NOW, `${secret} reason`);
|
||||
}
|
||||
|
||||
/** Every value in every table, so a check can say no copy of a secret is left anywhere. */
|
||||
function everything(): string {
|
||||
const tables = (db.prepare("SELECT name FROM sqlite_master WHERE type = 'table'").all() as Array<{ name: string }>).map((t) => t.name);
|
||||
return tables.map((table) => JSON.stringify(db.prepare(`SELECT * FROM "${table}"`).all())).join("\n");
|
||||
}
|
||||
const exists = (id: string) => db.prepare("SELECT 1 FROM queue_items WHERE qitem_id = ?").get(id) !== undefined;
|
||||
/** Event payloads as stored (everything() escapes their quotes, so match payload text here). */
|
||||
const payloads = () => (db.prepare("SELECT payload FROM events").all() as Array<{ payload: string }>).map((e) => e.payload).join("\n");
|
||||
const marker = (id: string) => db.prepare("SELECT * FROM queue_erased_ids WHERE qitem_id = ?").get(id) as Record<string, unknown> | undefined;
|
||||
|
||||
describe("account deletion (members scope)", () => {
|
||||
beforeEach(async () => {
|
||||
await row("q-done", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
await row("q-open", B, A); // open work: account deletion takes it too
|
||||
await row("q-mixed", A, OUT, [{ state: "done", daysAgo: 2 }]);
|
||||
await row("q-else", OUT, OUT);
|
||||
db.prepare(`INSERT INTO outbox_entries (outbox_id, sender_session, destination_session, body, ts_dispatched)
|
||||
VALUES ('o-loose', ?, ?, 'secret-loose', ?), ('o-cross', ?, ?, 'secret-cross', ?)`).run(A, B, NOW, A, OUT, NOW);
|
||||
});
|
||||
|
||||
it("a dry run reports exactly what an apply erases, and changes nothing", async () => {
|
||||
const before = everything();
|
||||
const dry = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: false, nowIso: NOW });
|
||||
expect(everything()).toBe(before);
|
||||
expect(dry.qitems.map((q) => [q.qitemId, q.action, q.state])).toEqual([["q-done", "delete", "done"], ["q-open", "delete", "pending"]]);
|
||||
expect(dry.mixed).toEqual([{ qitemId: "q-mixed", memberEndpoint: "source" }]);
|
||||
expect(dry.linked.outbox).toBeGreaterThanOrEqual(2);
|
||||
expect(dry.linked.events).toBeGreaterThanOrEqual(2);
|
||||
expect(dry.unlinked).toMatchObject({ outbox: 1, mixedEntries: 1 });
|
||||
const applied = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect({ ...applied, applied: false }).toEqual(dry);
|
||||
});
|
||||
|
||||
it("deletes both members' rows with every copy, keeps only their ids, and leaves mixed and other rows whole", async () => {
|
||||
const report = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect(report.complete).toBe(true);
|
||||
const all = everything();
|
||||
for (const id of ["q-done", "q-open"]) {
|
||||
expect(exists(id)).toBe(false);
|
||||
expect(all).not.toContain(`secret-${id}`);
|
||||
expect(marker(id)).toEqual({ qitem_id: id, erased_at: NOW, reason: "account-deletion" });
|
||||
}
|
||||
expect(all).not.toContain("secret-loose");
|
||||
// The mixed row and its copies stay, and so does the cross-account entry; nothing else was touched.
|
||||
expect(all).toContain("secret-q-mixed body");
|
||||
expect(all).toContain("secret-q-mixed summary");
|
||||
expect(all).toContain("secret-cross");
|
||||
expect(all).toContain("secret-q-else body");
|
||||
expect(marker("q-mixed")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("a create reusing an erased id is refused, so erased work never comes back as new work", async () => {
|
||||
await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
await expect(repo.create({ qitemId: "q-done", sourceSession: A, destinationSession: B, body: "secret-q-done body" }))
|
||||
.rejects.toBeInstanceOf(QueueRepositoryError);
|
||||
await expect(repo.create({ qitemId: "q-done", sourceSession: A, destinationSession: B, body: "secret-q-done body" }))
|
||||
.rejects.toMatchObject({ code: "qitem_erased" });
|
||||
expect(exists("q-done")).toBe(false);
|
||||
});
|
||||
|
||||
it("a run that stops partway says so, and a rerun finishes the rest", async () => {
|
||||
const first = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW, batchSize: 1, maxBatches: 1 });
|
||||
expect(first.complete).toBe(false);
|
||||
expect(first.unlinked).toBeUndefined();
|
||||
expect(first.qitems.map((q) => q.qitemId)).toEqual(["q-done"]);
|
||||
expect(exists("q-open")).toBe(true);
|
||||
const rest = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW, batchSize: 1 });
|
||||
expect(rest.complete).toBe(true);
|
||||
expect(rest.qitems.map((q) => q.qitemId)).toEqual(["q-open"]);
|
||||
expect(everything()).not.toContain("secret-loose");
|
||||
});
|
||||
|
||||
it("a row a workflow still references is reported and left", async () => {
|
||||
db.pragma("foreign_keys = OFF");
|
||||
db.prepare(`INSERT INTO workflow_frontier_bindings (instance_id, packet_id, step_id, created_at) VALUES ('wf-1', 'q-done', 's1', ?)`).run(NOW);
|
||||
db.pragma("foreign_keys = ON");
|
||||
const report = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect(report.gaps).toEqual([{ code: "workflow-reference", qitemId: "q-done" }]);
|
||||
expect(report.qitems.map((q) => q.qitemId)).toEqual(["q-open"]);
|
||||
expect(everything()).toContain("secret-q-done body");
|
||||
});
|
||||
});
|
||||
|
||||
describe("routine closed-work retention", () => {
|
||||
beforeEach(async () => {
|
||||
// Closed 40 days ago; a note yesterday keeps the closed state, so it doesn't restart the clock.
|
||||
await row("r-old-done", A, B, [{ state: "done", daysAgo: 40 }, { state: "done", daysAgo: 1 }]);
|
||||
await row("r-recent", A, B, [{ state: "done", daysAgo: 10 }]);
|
||||
await row("r-ancient-canceled", A, B, [{ state: "canceled", daysAgo: 100 }]);
|
||||
await row("r-failed", A, B, [{ state: "failed", daysAgo: 100 }]);
|
||||
await row("r-open", A, B, [{ state: "in-progress", daysAgo: 200 }]);
|
||||
// Closed long ago, reopened, closed again five days ago: it counts from the latest close.
|
||||
await row("r-reclosed", A, B, [{ state: "done", daysAgo: 50 }, { state: "pending", daysAgo: 20 }, { state: "denied", daysAgo: 5 }]);
|
||||
});
|
||||
|
||||
it("removes content past the content window and the row past the metadata window, from when each row closed", async () => {
|
||||
const report = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 90 }, apply: true, nowIso: NOW });
|
||||
expect(report.qitems.map((q) => [q.qitemId, q.action])).toEqual([["r-ancient-canceled", "delete"], ["r-old-done", "scrub"]]);
|
||||
const all = everything();
|
||||
expect(all).not.toContain("secret-r-old-done");
|
||||
expect(all).not.toContain("secret-r-ancient-canceled");
|
||||
for (const id of ["r-recent", "r-failed", "r-open", "r-reclosed"]) expect(all).toContain(`secret-${id} body`);
|
||||
expect(marker("r-ancient-canceled")).toMatchObject({ reason: "retention-metadata" });
|
||||
// The scrubbed row stays, with its identity, endpoints and state.
|
||||
const kept = db.prepare("SELECT source_session, destination_session, state, body, content_erased_at FROM queue_items WHERE qitem_id = 'r-old-done'").get();
|
||||
expect(kept).toEqual({ source_session: A, destination_session: B, state: "done", body: "", content_erased_at: NOW });
|
||||
const event = db.prepare("SELECT payload FROM events WHERE json_extract(payload, '$.qitemId') = 'r-old-done'").get() as { payload: string };
|
||||
expect(JSON.parse(event.payload)).toMatchObject({ type: "queue.created", qitemId: "r-old-done", contentErased: true });
|
||||
// A second sweep finds nothing more to do.
|
||||
const again = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 90 }, apply: true, nowIso: NOW });
|
||||
expect(again.qitems).toEqual([]);
|
||||
});
|
||||
|
||||
it("a retried create for a scrubbed row returns that row and makes no new work", async () => {
|
||||
await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
const transitions = () => (db.prepare("SELECT COUNT(*) AS n FROM queue_transitions WHERE qitem_id = 'r-old-done'").get() as { n: number }).n;
|
||||
const before = transitions();
|
||||
const retried = await repo.create({ qitemId: "r-old-done", sourceSession: A, destinationSession: B, body: "secret-r-old-done body" });
|
||||
expect(retried).toMatchObject({ qitemId: "r-old-done", state: "done", body: "", contentErasedAt: NOW });
|
||||
expect(transitions()).toBe(before);
|
||||
expect(everything()).not.toContain("secret-r-old-done");
|
||||
});
|
||||
|
||||
it("runs from the retention sweep only when a window is set", async () => {
|
||||
const off = await runQueueRetentionSweep(db, { nowIso: NOW });
|
||||
expect([off.closedScrubbed, off.closedDeleted]).toEqual([0, 0]);
|
||||
expect(everything()).toContain("secret-r-old-done body");
|
||||
const on = await runQueueRetentionSweep(db, { nowIso: NOW, closedContentDays: 30, closedMetadataDays: 90 });
|
||||
expect([on.closedScrubbed, on.closedDeleted, on.closedKept]).toEqual([1, 1, 0]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ids scope: reapplying a ledger after a restore", () => {
|
||||
it("deletes restored rows again and writes back every id marker, including ids already gone", async () => {
|
||||
await row("i-one", A, B, [{ state: "done", daysAgo: 3 }]);
|
||||
await eraseQueue(db, { scope: { kind: "ids", entries: [{ qitemId: "i-one", action: "delete" }] }, apply: true, nowIso: NOW });
|
||||
// A restore from a backup taken before the erase brings the row back and loses the marker.
|
||||
db.prepare("DELETE FROM queue_erased_ids").run();
|
||||
await row("i-one", A, B, [{ state: "done", daysAgo: 3 }]);
|
||||
const reapplied = await eraseQueue(db, { scope: { kind: "ids", entries: [{ qitemId: "i-one", action: "delete" }, { qitemId: "i-gone", action: "delete" }] }, apply: true, nowIso: NOW });
|
||||
expect(reapplied.qitems.map((q) => q.qitemId)).toEqual(["i-one"]);
|
||||
expect(reapplied.alreadyAbsent).toEqual(["i-gone"]);
|
||||
expect(exists("i-one")).toBe(false);
|
||||
expect(everything()).not.toContain("secret-i-one");
|
||||
expect(marker("i-one")).toMatchObject({ reason: "erase-by-id" });
|
||||
expect(marker("i-gone")).toMatchObject({ reason: "erase-by-id" });
|
||||
});
|
||||
|
||||
it("a scrub of an unknown id is reported, not invented", async () => {
|
||||
const report = await eraseQueue(db, { scope: { kind: "ids", entries: [{ qitemId: "nope", action: "scrub" }] }, apply: true, nowIso: NOW });
|
||||
expect(report.gaps).toEqual([{ code: "not-found", qitemId: "nope" }]);
|
||||
expect(marker("nope")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("POST /api/queue/erase", () => {
|
||||
function app(settings: Record<string, unknown> = {}) {
|
||||
const bus = new EventBus(db);
|
||||
const hono = new Hono();
|
||||
hono.use("*", async (c, next) => {
|
||||
c.set("db" as never, db as never);
|
||||
c.set("eventBus" as never, bus as never);
|
||||
c.set("queueRepo" as never, repo as never);
|
||||
c.set("inboxHandler" as never, new InboxHandler(db, bus, repo) as never);
|
||||
c.set("outboxHandler" as never, new OutboxHandler(db) as never);
|
||||
c.set("settingsStore" as never, { resolveOne: (key: string) => ({ value: settings[key] ?? 0 }) } as never);
|
||||
await next();
|
||||
});
|
||||
hono.route("/api/queue", queueRoutes());
|
||||
return hono;
|
||||
}
|
||||
const post = (hono: Hono, body: unknown) => hono.request("/api/queue/erase", {
|
||||
method: "POST", headers: { "content-type": "application/json", "X-OpenRig-Session": "ops-admin@n-ops" }, body: JSON.stringify(body),
|
||||
});
|
||||
|
||||
it("refuses a missing, doubled or malformed scope with a fixed code", async () => {
|
||||
const hono = app();
|
||||
const codes = async (body: unknown) => {
|
||||
const res = await post(hono, body);
|
||||
return [res.status, ((await res.json()) as { error: string }).error];
|
||||
};
|
||||
expect(await codes({})).toEqual([400, "erase_scope_required"]);
|
||||
expect(await codes({ members: [A], ids: ["x"] })).toEqual([400, "erase_scope_conflict"]);
|
||||
expect(await codes({ members: [] })).toEqual([400, "erase_members_invalid"]);
|
||||
expect(await codes({ members: [" a-one@n-acct"] })).toEqual([400, "erase_members_invalid"]);
|
||||
expect(await codes({ ids: ["x"], action: "shred" })).toEqual([400, "erase_action_invalid"]);
|
||||
expect(await codes({ routine: true })).toEqual([400, "erase_routine_off"]);
|
||||
expect(await codes({ ids: ["x"], rig: "rig-id" })).toEqual([400, "erase_rig_invalid"]);
|
||||
expect(await codes({ ids: [{ qitemId: "x", action: "shred" }] })).toEqual([400, "erase_action_invalid"]);
|
||||
});
|
||||
|
||||
it("runs a dry run by default and erases only with apply", async () => {
|
||||
await row("p-one", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
const hono = app();
|
||||
const dry = await post(hono, { members: [A, B] });
|
||||
expect(dry.status).toBe(200);
|
||||
expect(await dry.json()).toMatchObject({ ok: true, applied: false, qitems: [{ qitemId: "p-one", action: "delete" }] });
|
||||
expect(exists("p-one")).toBe(true);
|
||||
const applied = await post(hono, { members: [A, B], apply: true });
|
||||
expect(await applied.json()).toMatchObject({ applied: true, complete: true });
|
||||
expect(exists("p-one")).toBe(false);
|
||||
});
|
||||
|
||||
it("a run that stops on an error answers 500 with what it committed", async () => {
|
||||
await row("p-a", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
await row("p-b", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
db.exec("CREATE TEMP TRIGGER fail_pb BEFORE DELETE ON queue_items WHEN old.qitem_id = 'p-b' BEGIN SELECT RAISE(ABORT, 'disk full (test)'); END;");
|
||||
const res = await post(app(), { members: [A, B], apply: true });
|
||||
expect(res.status).toBe(500);
|
||||
expect(await res.json()).toMatchObject({ complete: false, error: { stage: "rows" } });
|
||||
});
|
||||
|
||||
it("routine reads this daemon's own windows", async () => {
|
||||
await row("p-old", A, B, [{ state: "done", daysAgo: 40 }]);
|
||||
const res = await post(app({ "retention.closed_content_days": 30 }), { routine: true, apply: true });
|
||||
expect(await res.json()).toMatchObject({ routine: { contentDays: 30, metadataDays: 0 }, qitems: [{ qitemId: "p-old", action: "scrub" }] });
|
||||
});
|
||||
});
|
||||
|
||||
describe("account deletion with the account's rig", () => {
|
||||
const RIG = "rig-acct-id";
|
||||
beforeEach(async () => {
|
||||
db.prepare("INSERT INTO rigs (id, name) VALUES (?, 'n-acct'), ('rig-else-id', 'n-else')").run(RIG);
|
||||
db.prepare("INSERT INTO nodes (id, rig_id, logical_id) VALUES ('node-a', ?, 'a.one'), ('node-x', 'rig-else-id', 'x.other')").run(RIG);
|
||||
const bus = new EventBus(db);
|
||||
bus.emit({ type: "rig.imported", rigId: RIG, specName: "secret-rig-spec", specVersion: "1" } as never);
|
||||
bus.emit({ type: "node.launched", rigId: RIG, nodeId: "node-a", logicalId: "a.one", sessionName: A } as never);
|
||||
bus.emit({ type: "node.launched", rigId: "rig-else-id", nodeId: "node-x", logicalId: "x.other", sessionName: OUT } as never);
|
||||
db.prepare("INSERT INTO snapshots (id, rig_id, kind, data) VALUES ('snap-a', ?, 'periodic', '{\"seat\":\"secret-snapshot\"}')").run(RIG);
|
||||
db.prepare("INSERT INTO seat_identity_verdicts (node_id, session_name, verdict, observed_at) VALUES ('node-a', ?, 'verified', ?)").run(A, NOW);
|
||||
db.prepare(`INSERT INTO watchdog_jobs (job_id, policy, spec_yaml, target_session, interval_seconds, registered_by_session, registered_at)
|
||||
VALUES ('wd-a', 'periodic-reminder', 'secret-watchdog', ?, 60, ?, ?), ('wd-cross', 'periodic-reminder', 'x', ?, 60, ?, ?)`)
|
||||
.run(A, B, NOW, OUT, A, NOW);
|
||||
db.prepare("INSERT INTO watchdog_history (job_id, evaluated_at, outcome) VALUES ('wd-a', ?, 'fired')").run(NOW);
|
||||
await row("g-done", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
db.prepare(`INSERT INTO inbox_entries (inbox_id, destination_session, sender_session, body, ts_dropped)
|
||||
VALUES ('i-loose', ?, ?, 'secret-loose-inbox', ?)`).run(B, A, NOW);
|
||||
});
|
||||
|
||||
it("erases the rig's own records, leaves another rig's, and counts a member's job aimed outside", async () => {
|
||||
const report = await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
expect(report.error).toBeUndefined();
|
||||
expect(report.rig).toMatchObject({ id: RIG, found: true, erased: { snapshots: 1, seat_identity_verdicts: 1, watchdog_jobs: 1, watchdog_history: 1 },
|
||||
left: { watchdog_jobs: 1 } });
|
||||
expect(report.rig!.erased.events).toBeGreaterThanOrEqual(2);
|
||||
expect(report.clean).toBe(false); // the member's job aimed outside the account is left, so not clean
|
||||
const all = everything();
|
||||
for (const secret of ["secret-rig-spec", "secret-snapshot", "secret-watchdog", "secret-loose-inbox", "secret-g-done"]) {
|
||||
expect(all).not.toContain(secret);
|
||||
}
|
||||
expect(payloads()).not.toContain(`"sessionName":"${A}"`);
|
||||
expect(payloads()).toContain(`"sessionName":"${OUT}"`);
|
||||
expect(db.prepare("SELECT COUNT(*) AS n FROM events WHERE rig_id = ?").get(RIG)).toEqual({ n: 0 });
|
||||
});
|
||||
|
||||
it("refuses a rig the members don't belong to, and erases nothing", async () => {
|
||||
const before = everything();
|
||||
await expect(eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: "rig-else-id" }, apply: true, nowIso: NOW }))
|
||||
.rejects.toThrowError(EraseRefusal);
|
||||
await expect(eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: "rig-else-id" }, apply: true, nowIso: NOW }))
|
||||
.rejects.toMatchObject({ code: "erase_rig_mismatch" });
|
||||
expect(everything()).toBe(before);
|
||||
});
|
||||
|
||||
it("after the rig is removed, a rerun erases what its removal wrote, by the rig's id", async () => {
|
||||
await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
db.prepare("DELETE FROM watchdog_jobs WHERE job_id = 'wd-cross'").run();
|
||||
db.prepare("DELETE FROM nodes WHERE rig_id = ?").run(RIG);
|
||||
db.prepare("DELETE FROM rigs WHERE id = ?").run(RIG);
|
||||
new EventBus(db).emit({ type: "session.detached", rigId: RIG, nodeId: "node-a", sessionName: A } as never);
|
||||
const rerun = await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
expect(rerun.rig).toMatchObject({ found: false, erased: { events: 1 } });
|
||||
expect(rerun.clean).toBe(true);
|
||||
expect(payloads()).not.toContain(`"sessionName":"${A}"`);
|
||||
expect(payloads()).toContain(`"sessionName":"${OUT}"`);
|
||||
// The record is now a completion receipt without the rig's name, so an identical rerun (after a lost response,
|
||||
// say) still verifies and reports clean.
|
||||
expect(rerun.rig).toMatchObject({ completedAt: NOW });
|
||||
expect(db.prepare("SELECT rig_name, completed_at FROM queue_erased_rigs WHERE rig_id = ?").get(RIG)).toEqual({ rig_name: "", completed_at: NOW });
|
||||
const again = await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
expect(again.rig).toMatchObject({ found: false, verified: true, completedAt: NOW, unknown: [] });
|
||||
expect(again.clean).toBe(true);
|
||||
});
|
||||
|
||||
it("K2: a rig pass that stops partway reports the records it already erased", async () => {
|
||||
db.exec("CREATE TEMP TRIGGER fail_wd BEFORE DELETE ON watchdog_jobs BEGIN SELECT RAISE(ABORT, 'disk I/O error (test)'); END;");
|
||||
const report = await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
expect(report).toMatchObject({ complete: false, error: { stage: "rig" }, rig: { erased: { snapshots: 1, watchdog_history: 1 } } });
|
||||
expect(report.rig!.erased.events).toBeGreaterThanOrEqual(2);
|
||||
expect(db.prepare("SELECT COUNT(*) AS n FROM snapshots WHERE rig_id = ?").get(RIG)).toEqual({ n: 0 });
|
||||
});
|
||||
|
||||
it("restore control: after restoring a backup taken before the erase, the documented replay leaves no account content", async () => {
|
||||
await row("g-scrub", A, OUT, [{ state: "done", daysAgo: 40 }]); // another account's row, scrubbed by routine retention
|
||||
const backup = db.serialize();
|
||||
await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
const routine = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
const ledger = routine.qitems; // what a ledger keeps: ids with their actions
|
||||
// The restore: the database comes back as it was before both runs.
|
||||
db.close();
|
||||
open(new Database(backup));
|
||||
expect(everything()).toContain("secret-g-done body");
|
||||
// The documented replay: the retained membership and rig, then the ledger with its actions, then routine.
|
||||
const replay = await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
await eraseQueue(db, { scope: { kind: "ids", entries: ledger }, apply: true, nowIso: NOW });
|
||||
await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
expect(replay.complete).toBe(true);
|
||||
const all = everything();
|
||||
for (const secret of ["secret-g-done", "secret-loose-inbox", "secret-rig-spec", "secret-snapshot", "secret-watchdog", "secret-g-scrub"]) {
|
||||
expect(all).not.toContain(secret);
|
||||
}
|
||||
expect(marker("g-done")).toMatchObject({ reason: "account-deletion" });
|
||||
// The scrubbed row was scrubbed again, not promoted to a deletion.
|
||||
expect(exists("g-scrub")).toBe(true);
|
||||
expect(marker("g-scrub")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("clean", () => {
|
||||
it("is true only when the run finished and left nothing", async () => {
|
||||
await row("c-one", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
expect((await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: false, nowIso: NOW })).clean).toBe(true);
|
||||
await row("c-mixed", A, OUT, [{ state: "done", daysAgo: 2 }]);
|
||||
expect((await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: false, nowIso: NOW })).clean).toBe(false);
|
||||
const partial = await eraseQueue(db, { scope: { kind: "ids", entries: [{ qitemId: "c-one", action: "delete" }] }, apply: false, nowIso: NOW, batchSize: 1, maxBatches: 0 });
|
||||
expect([partial.complete, partial.clean]).toEqual([false, false]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("round 1 controls", () => {
|
||||
it("J1: the events walk finds the same events whatever its window", async () => {
|
||||
await row("w-one", A, B, [{ state: "done", daysAgo: 40 }]);
|
||||
await row("w-two", A, B, [{ state: "done", daysAgo: 40 }]);
|
||||
const wide = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: false, nowIso: NOW });
|
||||
const narrow = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW, eventsWindow: 1 });
|
||||
expect(narrow.linked.events).toBe(wide.linked.events);
|
||||
expect(narrow.linked.events).toBeGreaterThanOrEqual(2);
|
||||
expect(payloads()).not.toContain("secret-w-");
|
||||
});
|
||||
|
||||
it("J2: a linked copy held with someone outside the account is kept and reported, and the run isn't clean", async () => {
|
||||
await row("o-own", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
db.prepare(`INSERT INTO outbox_entries (outbox_id, sender_session, destination_session, body, ts_dispatched, audit_pointer)
|
||||
VALUES ('o-mixed', ?, ?, 'secret-mixed-copy', ?, 'o-own'), ('o-foreign', ?, ?, 'secret-foreign-copy', ?, 'o-own')`).run(A, OUT, NOW, OUT, OUT, NOW);
|
||||
const report = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect(report.keptCopies).toEqual(expect.arrayContaining([
|
||||
{ table: "outbox_entries", id: "o-mixed", qitemId: "o-own", ownership: "mixed" },
|
||||
{ table: "outbox_entries", id: "o-foreign", qitemId: "o-own", ownership: "foreign" },
|
||||
]));
|
||||
expect(report.clean).toBe(false);
|
||||
const all = everything();
|
||||
expect(all).toContain("secret-mixed-copy");
|
||||
expect(all).toContain("secret-foreign-copy");
|
||||
expect(all).not.toContain("secret-o-own");
|
||||
});
|
||||
|
||||
it("J3: a scrub clears closure notes in every copy and a duplicate reader action's reason, and keeps a successor id", async () => {
|
||||
await row("c-canceled", A, B);
|
||||
repo.update({ qitemId: "c-canceled", actorSession: B, state: "done", closureReason: "canceled", closureTarget: "secret-closure-note" });
|
||||
await row("c-handed", A, B);
|
||||
repo.update({ qitemId: "c-handed", actorSession: B, state: "done", closureReason: "handed_off_to", closureTarget: B });
|
||||
db.prepare("INSERT INTO stream_items (stream_item_id, ts_emitted, stream_sort_key, source_session, body) VALUES ('s-dup', ?, 'k', ?, 'x')").run(NOW, A);
|
||||
recordReaderAction(db, { streamItemId: "s-dup", readerSession: A, action: "duplicate", duplicateOf: "c-canceled", reason: "secret-dup-reason" }, NOW);
|
||||
db.prepare("UPDATE queue_transitions SET ts = ? WHERE qitem_id IN ('c-canceled', 'c-handed')").run(ago(40));
|
||||
await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
const all = everything();
|
||||
expect(all).not.toContain("secret-closure-note");
|
||||
expect(all).not.toContain("secret-dup-reason");
|
||||
expect(db.prepare("SELECT closure_target AS t FROM queue_items WHERE qitem_id = 'c-handed'").get()).toEqual({ t: B });
|
||||
});
|
||||
|
||||
it("J4: content written to a scrubbed row after its scrub is erased by the next run, and a reclose counts from the new close", async () => {
|
||||
await row("n-row", A, B, [{ state: "done", daysAgo: 40 }]);
|
||||
await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
repo.update({ qitemId: "n-row", actorSession: B, transitionNote: "secret-later-note" });
|
||||
const again = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
expect(again.qitems.map((q) => q.qitemId)).toEqual(["n-row"]);
|
||||
expect(everything()).not.toContain("secret-later-note");
|
||||
repo.update({ qitemId: "n-row", actorSession: B, state: "pending", reopen: true, transitionNote: "secret-reopened" });
|
||||
repo.update({ qitemId: "n-row", actorSession: B, state: "done", closureReason: "no-follow-on" });
|
||||
const soon = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
expect(soon.qitems).toEqual([]); // closed again just now: inside the window
|
||||
const later = new Date(Date.now() + 31 * 86_400_000).toISOString();
|
||||
const aged = await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: later });
|
||||
expect(aged.qitems.map((q) => q.qitemId)).toEqual(["n-row"]);
|
||||
expect(everything()).not.toContain("secret-reopened");
|
||||
});
|
||||
|
||||
it("J5: a dry run and an apply agree when deleting a row frees a member's stream item", async () => {
|
||||
await row("s-row", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
db.prepare("INSERT INTO stream_items (stream_item_id, ts_emitted, stream_sort_key, source_session, body) VALUES ('s-act', ?, 'k', ?, 'secret-stream')").run(NOW, A);
|
||||
recordReaderAction(db, { streamItemId: "s-act", readerSession: B, action: "act", workQitemId: "s-row" }, NOW);
|
||||
const dry = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: false, nowIso: NOW });
|
||||
const applied = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect({ ...applied, applied: false }).toEqual(dry);
|
||||
expect(dry.unlinked).toMatchObject({ streamItems: 1, referencedStreamItems: 0 });
|
||||
expect(everything()).not.toContain("secret-stream");
|
||||
});
|
||||
|
||||
it("J8: a failure after a committed batch returns only what committed, and the sweep's other passes still run", async () => {
|
||||
await row("f-1", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
await row("f-2", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
db.exec("CREATE TEMP TRIGGER fail_f2 BEFORE DELETE ON queue_items WHEN old.qitem_id = 'f-2' BEGIN SELECT RAISE(ABORT, 'disk I/O error (test)'); END;");
|
||||
const report = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW, batchSize: 1 });
|
||||
expect(report).toMatchObject({ complete: false, clean: false, error: { stage: "rows", message: "disk I/O error (test)" } });
|
||||
expect(report.qitems.map((q) => q.qitemId)).toEqual(["f-1"]);
|
||||
expect([exists("f-1"), exists("f-2")]).toEqual([false, true]);
|
||||
expect(report.linked.events).toBeGreaterThan(0);
|
||||
expect(payloads()).not.toContain('"qitemId":"f-1"');
|
||||
expect(payloads()).toContain('"qitemId":"f-2"');
|
||||
db.exec("CREATE TEMP TRIGGER fail_scrub BEFORE UPDATE OF content_erased_at ON queue_items BEGIN SELECT RAISE(ABORT, 'disk I/O error (test)'); END;");
|
||||
db.prepare("UPDATE queue_transitions SET ts = ? WHERE qitem_id = 'f-2'").run(ago(40));
|
||||
const swept = await runQueueRetentionSweep(db, { nowIso: NOW, closedContentDays: 30, transitionsRetentionDays: 1 });
|
||||
expect(swept.closedError).toMatch(/disk I\/O error/);
|
||||
expect(swept.archivedQitems).toBeGreaterThan(0); // the transition archive still ran
|
||||
});
|
||||
});
|
||||
|
||||
describe("round 1 controls: the account rig after removal, and a restore across a close", () => {
|
||||
const RIG = "rig-r1-id";
|
||||
beforeEach(() => {
|
||||
db.prepare("INSERT INTO rigs (id, name) VALUES (?, 'n-acct')").run(RIG);
|
||||
db.prepare("INSERT INTO nodes (id, rig_id, logical_id) VALUES ('node-r1', ?, 'a.one')").run(RIG);
|
||||
new EventBus(db).emit({ type: "rig.imported", rigId: RIG, specName: "secret-r1-spec", specVersion: "1" } as never);
|
||||
});
|
||||
|
||||
it("N2: refuses a rig with a session the members don't list", async () => {
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name) VALUES ('sess-x', 'node-r1', ?)").run(OUT);
|
||||
await expect(eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: false, nowIso: NOW }))
|
||||
.rejects.toMatchObject({ code: "erase_rig_mismatch" });
|
||||
});
|
||||
|
||||
it("J6: after removal a name-bound workflow record is counted as left, and an unverified id erases nothing", async () => {
|
||||
db.pragma("foreign_keys = OFF");
|
||||
db.prepare(`INSERT INTO workflow_specs (spec_id, name, version, purpose, target_rig, roles_json, steps_json, coordination_terminal_turn_rule,
|
||||
source_path, source_hash, cached_at) VALUES ('spec-1', 's', '1', 'p', 'n-acct', '[]', '[]', 'hot_potato', '/x', 'h', ?)`).run(NOW);
|
||||
db.pragma("foreign_keys = ON");
|
||||
await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
db.prepare("DELETE FROM nodes WHERE rig_id = ?").run(RIG);
|
||||
db.prepare("DELETE FROM rigs WHERE id = ?").run(RIG);
|
||||
new EventBus(db).emit({ type: "rig.deleted", rigId: RIG } as never);
|
||||
const rerun = await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: RIG }, apply: true, nowIso: NOW });
|
||||
expect(rerun.rig).toMatchObject({ found: false, verified: true, left: { workflow_specs: 1 }, unknown: [] });
|
||||
expect(rerun.clean).toBe(false);
|
||||
// Another removed rig's id, never verified with this membership: nothing keyed by it is erased.
|
||||
new EventBus(db).emit({ type: "rig.deleted", rigId: "rig-someone-else" } as never);
|
||||
const unverified = await eraseQueue(db, { scope: { kind: "members", members: [A, B], rig: "rig-someone-else" }, apply: true, nowIso: NOW });
|
||||
expect(unverified.rig).toMatchObject({ found: false, verified: false, left: { events: 1 }, unknown: ["workflow_instances", "workflow_specs"] });
|
||||
expect(unverified.clean).toBe(false);
|
||||
expect(db.prepare("SELECT COUNT(*) AS n FROM events WHERE rig_id = 'rig-someone-else'").get()).toEqual({ n: 1 });
|
||||
});
|
||||
|
||||
it("J7: a backup taken before a row closed is replayed from the ledger, and restored open work stays", async () => {
|
||||
const ledgerPath = join(mkdtempSync(join(tmpdir(), "erase-ledger-")), "ledger.jsonl");
|
||||
await row("l-row", A, B); // open when the backup is taken
|
||||
await row("l-open", A, B); // still open later
|
||||
const backup = db.serialize();
|
||||
repo.update({ qitemId: "l-row", actorSession: B, state: "done", closureReason: "no-follow-on" });
|
||||
db.prepare("UPDATE queue_transitions SET ts = ? WHERE qitem_id = 'l-row'").run(ago(40));
|
||||
await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW, ledgerPath });
|
||||
const lines = readFileSync(ledgerPath, "utf8").trim().split("\n").map((line) => JSON.parse(line) as { qitemId: string; action: "scrub" });
|
||||
expect(lines).toEqual([{ qitemId: "l-row", action: "scrub", reason: "retention-content", erasedAt: NOW }]);
|
||||
db.close();
|
||||
open(new Database(backup));
|
||||
expect(everything()).toContain("secret-l-row body");
|
||||
await eraseQueue(db, { scope: { kind: "ids", entries: lines }, apply: true, nowIso: NOW });
|
||||
await eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 0 }, apply: true, nowIso: NOW });
|
||||
const all = everything();
|
||||
expect(all).not.toContain("secret-l-row");
|
||||
expect(all).toContain("secret-l-open body");
|
||||
});
|
||||
});
|
||||
|
||||
describe("round 2 controls", () => {
|
||||
const routine = { kind: "routine", contentDays: 30, metadataDays: 0 } as const;
|
||||
|
||||
it("K1: a row reopened or taken up by a workflow after selection is kept and reported, with its events", async () => {
|
||||
await row("k-reopen", A, B, [{ state: "done", daysAgo: 100 }]);
|
||||
await row("k-workflow", A, B, [{ state: "done", daysAgo: 100 }]);
|
||||
await row("k-stays", A, B, [{ state: "done", daysAgo: 100 }]);
|
||||
// The run yields after each events window. With a window of 1, these writes land during the walk, after selection.
|
||||
const run = eraseQueue(db, { scope: { kind: "routine", contentDays: 30, metadataDays: 90 }, apply: true, nowIso: NOW, eventsWindow: 1 });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
repo.update({ qitemId: "k-reopen", actorSession: B, state: "pending", reopen: true, transitionNote: "back to work" });
|
||||
db.pragma("foreign_keys = OFF");
|
||||
db.prepare(`INSERT INTO workflow_frontier_bindings (instance_id, packet_id, step_id, created_at) VALUES ('wf-k', 'k-workflow', 's1', ?)`).run(NOW);
|
||||
db.pragma("foreign_keys = ON");
|
||||
const report = await run;
|
||||
expect(report.qitems.map((q) => q.qitemId)).toEqual(["k-stays"]);
|
||||
expect(report.gaps).toEqual(expect.arrayContaining([{ code: "changed-during-run", qitemId: "k-reopen" },
|
||||
{ code: "workflow-reference", qitemId: "k-workflow" }]));
|
||||
expect(report.clean).toBe(false);
|
||||
expect(db.prepare("SELECT state, body FROM queue_items WHERE qitem_id = 'k-reopen'").get()).toEqual({ state: "pending", body: "secret-k-reopen body" });
|
||||
expect(marker("k-reopen")).toBeUndefined();
|
||||
expect(payloads()).toContain("secret-k-reopen");
|
||||
expect(payloads()).toContain("secret-k-workflow");
|
||||
expect([exists("k-workflow"), exists("k-stays")]).toEqual([true, false]);
|
||||
});
|
||||
|
||||
it("K2: a ledger that can't be written stops the run before anything changes", async () => {
|
||||
await row("l-one", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
const before = everything();
|
||||
const ledgerPath = join(mkdtempSync(join(tmpdir(), "erase-ledger-")), "missing", "ledger.jsonl");
|
||||
const report = await eraseQueue(db, { scope: { kind: "ids", entries: [{ qitemId: "l-one", action: "delete" }, { qitemId: "l-gone", action: "delete" }] },
|
||||
apply: true, nowIso: NOW, ledgerPath });
|
||||
expect(report).toMatchObject({ complete: false, clean: false, error: { stage: "ledger" }, qitems: [], alreadyAbsent: [] });
|
||||
expect(everything()).toBe(before);
|
||||
expect(payloads()).toContain("secret-l-one");
|
||||
});
|
||||
|
||||
it("K3, L3: an event that names a row but can't be read is reported with its row kept, on every rerun until it's resolved", async () => {
|
||||
await row("u-row", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
await row("u-other", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
const bad = db.prepare("INSERT INTO events (type, payload) VALUES ('queue.updated', ?)").run('{"qitemId":"u-row","summary":"secret-truncated');
|
||||
const dry = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: false, nowIso: NOW });
|
||||
const report = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect({ ...report, applied: false }).toEqual(dry);
|
||||
expect(report.unreadableEvents).toEqual([{ seq: Number(bad.lastInsertRowid), qitemIds: ["u-row"] }]);
|
||||
expect(report.gaps).toEqual([{ code: "unreadable-event", qitemId: "u-row" }]);
|
||||
expect([report.complete, report.clean]).toEqual([true, false]);
|
||||
expect([exists("u-row"), exists("u-other")]).toEqual([true, false]);
|
||||
// An identical rerun (after a lost response, say) finds it again.
|
||||
const again = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect(again.unreadableEvents).toEqual(report.unreadableEvents);
|
||||
expect(again.clean).toBe(false);
|
||||
expect(payloads()).toContain("secret-truncated");
|
||||
// Once the event is resolved, the next run erases the row and is clean.
|
||||
db.prepare("DELETE FROM events WHERE seq = ?").run(bad.lastInsertRowid);
|
||||
const resolved = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect([resolved.clean, exists("u-row")]).toEqual([true, false]);
|
||||
});
|
||||
|
||||
it("K4: a successor reason keeps its target only when it names a session or a queue row", async () => {
|
||||
await row("t-prose", A, B);
|
||||
repo.update({ qitemId: "t-prose", actorSession: B, state: "done", closureReason: "handed_off_to", closureTarget: "secret-prose-target, see the thread" });
|
||||
await row("t-super", A, B);
|
||||
repo.update({ qitemId: "t-super", actorSession: B, state: "canceled", closureReason: "superseded", closureTarget: "qitem-successor-row" });
|
||||
db.prepare("UPDATE queue_transitions SET ts = ? WHERE qitem_id IN ('t-prose', 't-super')").run(ago(40));
|
||||
const report = await eraseQueue(db, { scope: routine, apply: true, nowIso: NOW });
|
||||
expect(report.qitems.map((q) => q.qitemId)).toEqual(["t-prose", "t-super"]);
|
||||
expect(everything()).not.toContain("secret-prose-target");
|
||||
expect(payloads()).not.toContain("secret-prose-target");
|
||||
expect(db.prepare("SELECT closure_target AS t FROM queue_items WHERE qitem_id = 't-super'").get()).toEqual({ t: "qitem-successor-row" });
|
||||
});
|
||||
|
||||
it("N1: a note archived while erasure was off still makes a scrubbed row eligible again", async () => {
|
||||
await row("v-row", A, B, [{ state: "done", daysAgo: 40 }]);
|
||||
await eraseQueue(db, { scope: routine, apply: true, nowIso: NOW });
|
||||
repo.update({ qitemId: "v-row", actorSession: B, transitionNote: "secret-archived-note" });
|
||||
const later = new Date(Date.now() + 2 * 86_400_000).toISOString();
|
||||
await runQueueRetentionSweep(db, { nowIso: later, transitionsRetentionDays: 1 });
|
||||
const note = "SELECT COUNT(*) AS n FROM %s WHERE transition_note = 'secret-archived-note'";
|
||||
expect([db.prepare(note.replace("%s", "queue_transitions")).get(), db.prepare(note.replace("%s", "queue_transitions_archive")).get()])
|
||||
.toEqual([{ n: 0 }, { n: 1 }]);
|
||||
const again = await eraseQueue(db, { scope: routine, apply: true, nowIso: later });
|
||||
expect(again.qitems.map((q) => q.qitemId)).toEqual(["v-row"]);
|
||||
expect(everything()).not.toContain("secret-archived-note");
|
||||
});
|
||||
|
||||
it("N2: a dry run and an apply count the same mixed entries when a kept copy points at an erased row", async () => {
|
||||
await row("m-row", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
db.prepare(`INSERT INTO outbox_entries (outbox_id, sender_session, destination_session, body, ts_dispatched, audit_pointer)
|
||||
VALUES ('o-m', ?, ?, 'secret-m-copy', ?, 'm-row')`).run(A, OUT, NOW);
|
||||
const dry = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: false, nowIso: NOW });
|
||||
const applied = await eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW });
|
||||
expect({ ...applied, applied: false }).toEqual(dry);
|
||||
expect(dry.keptCopies).toEqual([{ table: "outbox_entries", id: "o-m", qitemId: "m-row", ownership: "mixed" }]);
|
||||
expect(dry.unlinked).toMatchObject({ mixedEntries: 0 });
|
||||
});
|
||||
});
|
||||
|
||||
describe("round 3 controls", () => {
|
||||
// The run yields after each events window. With a window of 1, a write queued now lands during the walk, after
|
||||
// selection and before the batches.
|
||||
const duringWalk = () => new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
it("L1: a row rerouted out of the account after selection is kept with its events and reported", async () => {
|
||||
await row("l1-moved", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
await row("l1-stays", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
const run = eraseQueue(db, { scope: { kind: "members", members: [A, B] }, apply: true, nowIso: NOW, eventsWindow: 1 });
|
||||
await duringWalk();
|
||||
repo.routeToFallback("l1-moved", OUT, "owner unavailable");
|
||||
const report = await run;
|
||||
expect(report.qitems.map((q) => q.qitemId)).toEqual(["l1-stays"]);
|
||||
expect(report.gaps).toEqual([{ code: "changed-during-run", qitemId: "l1-moved" }]);
|
||||
expect(report.clean).toBe(false);
|
||||
expect(db.prepare("SELECT destination_session AS d, body FROM queue_items WHERE qitem_id = 'l1-moved'").get())
|
||||
.toEqual({ d: OUT, body: "secret-l1-moved body" });
|
||||
expect(marker("l1-moved")).toBeUndefined();
|
||||
expect(payloads()).toContain("secret-l1-moved");
|
||||
});
|
||||
|
||||
it("L2: an id absent at selection but created during the run is reported, never certified absent", async () => {
|
||||
await row("l2-old", A, B, [{ state: "done", daysAgo: 2 }]);
|
||||
const run = eraseQueue(db, { scope: { kind: "ids", entries: [{ qitemId: "l2-new", action: "delete" }, { qitemId: "l2-old", action: "delete" }] },
|
||||
apply: true, nowIso: NOW, eventsWindow: 1 });
|
||||
await duringWalk();
|
||||
await repo.create({ qitemId: "l2-new", sourceSession: A, destinationSession: B, body: "secret-l2-new body" });
|
||||
const report = await run;
|
||||
expect(report.qitems.map((q) => q.qitemId)).toEqual(["l2-old"]);
|
||||
expect(report.alreadyAbsent).toEqual([]);
|
||||
expect(report.gaps).toEqual([{ code: "changed-during-run", qitemId: "l2-new" }]);
|
||||
expect(report.clean).toBe(false);
|
||||
expect(marker("l2-new")).toBeUndefined();
|
||||
expect(everything()).toContain("secret-l2-new body");
|
||||
});
|
||||
});
|
||||
@@ -1,379 +0,0 @@
|
||||
// A listing reads what every row needs once for all its rows instead of once per row: the last meaningful transitions
|
||||
// in one windowed query, and (with it) the waiting view's statements, the owners' activity and the row itself. Each
|
||||
// batched read must answer exactly what the per-row read answers, and a listing must project the same items.
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import Database from "better-sqlite3";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { ALL_MIGRATIONS } from "../src/db/all-migrations.js";
|
||||
import { queueItemsSchema } from "../src/db/migrations/024_queue_items.js";
|
||||
import { queueTransitionsSchema } from "../src/db/migrations/025_queue_transitions.js";
|
||||
import { EventBus } from "../src/domain/event-bus.js";
|
||||
import { QueueRepository } from "../src/domain/queue-repository.js";
|
||||
import { lastMeaningfulTransition, lastMeaningfulTransitionMany } from "../src/domain/queue-waiting.js";
|
||||
|
||||
let db: Database.Database;
|
||||
const at = (s: number) => new Date(Date.UTC(2026, 10, 7, 9, 0, s)).toISOString();
|
||||
|
||||
function item(id: string, o: { state?: string; blockedOn?: string | null } = {}) {
|
||||
db.prepare(`INSERT INTO queue_items(qitem_id, ts_created, ts_updated, source_session, destination_session, state, body, blocked_on)
|
||||
VALUES (?, ?, ?, 'sender@r', 'worker@r', ?, 'body', ?)`).run(id, at(0), at(0), o.state ?? "pending", o.blockedOn ?? null);
|
||||
}
|
||||
function transition(qitemId: string, state: string, actor: string, s: number): number {
|
||||
return Number(db.prepare(`INSERT INTO queue_transitions(qitem_id, ts, state, actor_session) VALUES (?, ?, ?, ?)`)
|
||||
.run(qitemId, at(s), state, actor).lastInsertRowid);
|
||||
}
|
||||
function wake(transitionId: number, qitemId: string, phase: "armed" | "fired") {
|
||||
db.prepare(`INSERT INTO queue_transition_wakes(transition_id, qitem_id, phase, wake_kind, wake_ref) VALUES (?, ?, ?, 'timer', 'job-1')`)
|
||||
.run(transitionId, qitemId, phase);
|
||||
}
|
||||
/** The batched answer and the per-row answer, for the same ids. */
|
||||
function both(ids: string[]) {
|
||||
return {
|
||||
many: Object.fromEntries(lastMeaningfulTransitionMany(db, ids)),
|
||||
perRow: Object.fromEntries(ids.map((id) => [id, lastMeaningfulTransition(db, id)])),
|
||||
};
|
||||
}
|
||||
|
||||
describe("lastMeaningfulTransitionMany answers what lastMeaningfulTransition answers", () => {
|
||||
beforeEach(() => { db = new Database(":memory:"); migrate(db, ALL_MIGRATIONS); });
|
||||
afterEach(() => db.close());
|
||||
|
||||
it("covers a row without transitions, system actors repeating a state, a state change by a system actor, and an author note", () => {
|
||||
item("q-none");
|
||||
item("q-system-only"); // every transition by a system actor, same state: the first one counts
|
||||
transition("q-system-only", "pending", "daemon@system", 1);
|
||||
transition("q-system-only", "pending", "watchdog@system", 2);
|
||||
transition("q-system-only", "pending", "wake-ladder@system", 3);
|
||||
item("q-mixed"); // an author, then system bookkeeping in the same state
|
||||
transition("q-mixed", "pending", "sender@r", 1);
|
||||
transition("q-mixed", "in-progress", "worker@r", 2);
|
||||
transition("q-mixed", "in-progress", "daemon@kernel", 3);
|
||||
transition("q-mixed", "in-progress", "watchdog@system", 4);
|
||||
item("q-system-moves"); // a system actor that changes the state is meaningful
|
||||
transition("q-system-moves", "pending", "sender@r", 1);
|
||||
transition("q-system-moves", "blocked", "worker@r", 2);
|
||||
transition("q-system-moves", "pending", "daemon@system", 3);
|
||||
const { many, perRow } = both(["q-none", "q-system-only", "q-mixed", "q-system-moves", "q-unknown"]);
|
||||
expect(many).toEqual(perRow);
|
||||
expect(many["q-none"]).toBeNull();
|
||||
expect(many["q-unknown"]).toBeNull();
|
||||
expect(many["q-system-only"]).not.toBeNull();
|
||||
});
|
||||
|
||||
it("skips a transition whose wake fired, but not one whose wake is only armed", () => {
|
||||
item("q-fired");
|
||||
transition("q-fired", "pending", "sender@r", 1);
|
||||
const parked = transition("q-fired", "blocked", "worker@r", 2);
|
||||
wake(parked, "q-fired", "armed");
|
||||
const fired = transition("q-fired", "pending", "worker@r", 3);
|
||||
wake(fired, "q-fired", "fired");
|
||||
item("q-armed");
|
||||
const armed = transition("q-armed", "blocked", "worker@r", 1);
|
||||
wake(armed, "q-armed", "armed");
|
||||
const { many, perRow } = both(["q-fired", "q-armed"]);
|
||||
expect(many).toEqual(perRow);
|
||||
expect(many["q-fired"]).toEqual({ id: parked, at: at(2) });
|
||||
});
|
||||
|
||||
it("an empty listing reads nothing and returns an empty map", () => {
|
||||
const spy = vi.spyOn(db, "prepare");
|
||||
expect(lastMeaningfulTransitionMany(db, []).size).toBe(0);
|
||||
expect(spy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("an id asked twice (a listed row that is also another row's blocker) maps to its one transition", () => {
|
||||
item("q-blocker");
|
||||
const t = transition("q-blocker", "in-progress", "worker@r", 1);
|
||||
item("q-waiter", { state: "blocked", blockedOn: "q-blocker" });
|
||||
transition("q-waiter", "blocked", "worker@r", 2);
|
||||
const ids = ["q-blocker", "q-waiter", "q-blocker"];
|
||||
const { many, perRow } = both(ids);
|
||||
expect(many).toEqual(perRow);
|
||||
expect(many["q-blocker"]).toEqual({ id: t, at: at(1) });
|
||||
expect(lastMeaningfulTransitionMany(db, ids).size).toBe(2);
|
||||
});
|
||||
|
||||
it("reads in a constant number of statements, whatever the number of rows", () => {
|
||||
for (let i = 0; i < 40; i++) { item(`q-${i}`); transition(`q-${i}`, "pending", "sender@r", i); }
|
||||
const spy = vi.spyOn(db, "prepare");
|
||||
lastMeaningfulTransitionMany(db, Array.from({ length: 40 }, (_, i) => `q-${i}`));
|
||||
expect(spy.mock.calls.length).toBeLessThanOrEqual(3);
|
||||
});
|
||||
});
|
||||
|
||||
describe("lastMeaningfulTransitionMany on older schemas", () => {
|
||||
afterEach(() => db.close());
|
||||
|
||||
it("a DB without queue_transition_wakes reads every transition as the per-row read does", () => {
|
||||
db = new Database(":memory:");
|
||||
migrate(db, [queueItemsSchema, queueTransitionsSchema]);
|
||||
item("q-a");
|
||||
transition("q-a", "pending", "sender@r", 1);
|
||||
transition("q-a", "pending", "watchdog@system", 2);
|
||||
item("q-b");
|
||||
const { many, perRow } = both(["q-a", "q-b"]);
|
||||
expect(many).toEqual(perRow);
|
||||
expect(many["q-a"]).not.toBeNull();
|
||||
});
|
||||
|
||||
it("a DB without queue_transitions maps every id to null", () => {
|
||||
db = new Database(":memory:");
|
||||
migrate(db, [queueItemsSchema]);
|
||||
item("q-a");
|
||||
expect(Object.fromEntries(lastMeaningfulTransitionMany(db, ["q-a"]))).toEqual({ "q-a": lastMeaningfulTransition(db, "q-a") });
|
||||
expect(lastMeaningfulTransition(db, "q-a")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("a listing projects the same items with its shared reads as with per-row reads", () => {
|
||||
let repo: QueueRepository;
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers({ toFake: ["Date"] });
|
||||
vi.setSystemTime(new Date("2026-11-07T09:30:00.000Z"));
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
repo = new QueueRepository(db, new EventBus(db), { validateRig: () => true, loadHumanRegistry: () => ({ ok: true, entities: [] }) } as never);
|
||||
});
|
||||
afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks(); db.close(); });
|
||||
|
||||
/** The listing as it reads now, and the same listing with its shared reads withheld (every row read alone). */
|
||||
function listedBothWays(opts: Parameters<QueueRepository["list"]>[0]) {
|
||||
const shared = repo.list(opts);
|
||||
const own = repo as unknown as { listingReads: unknown };
|
||||
own.listingReads = () => undefined;
|
||||
try {
|
||||
return { shared, perRow: repo.list(opts) };
|
||||
} finally {
|
||||
delete (own as { listingReads?: unknown }).listingReads;
|
||||
}
|
||||
}
|
||||
|
||||
async function seed() {
|
||||
const make = (destinationSession: string, body: string) =>
|
||||
repo.create({ sourceSession: "orch@r", destinationSession, body, nudge: false } as never).then((q) => q.qitemId);
|
||||
const fresh = await make("worker@r", "fresh");
|
||||
const claimed = await make("worker@r", "claimed");
|
||||
repo.update({ qitemId: claimed, actorSession: "worker@r", state: "in-progress" } as never);
|
||||
const blocker = await make("other@r", "the blocker");
|
||||
const waiter = await make("worker@r", "waits on the blocker");
|
||||
repo.update({ qitemId: waiter, actorSession: "worker@r", state: "blocked", blockedOn: blocker, transitionNote: "continuation: wait" } as never);
|
||||
const timed = await make("worker@r", "parked on a timer");
|
||||
repo.update({ qitemId: timed, actorSession: "worker@r", state: "blocked", blockedOn: "external:ci", transitionNote: "continuation: CI",
|
||||
wakeAfterSeconds: 600 } as never);
|
||||
const done = await make("worker@r", "done");
|
||||
repo.update({ qitemId: done, actorSession: "worker@r", state: "done", closureReason: "no-follow-on" } as never);
|
||||
return { fresh, claimed, blocker, waiter, timed, done };
|
||||
}
|
||||
|
||||
it("full and compact listings, with blockers inside and outside the listing", async () => {
|
||||
await seed();
|
||||
for (const opts of [{}, { compact: true as const }, { destinationSession: "worker@r" }, { destinationSession: "worker@r", compact: true as const },
|
||||
{ state: ["blocked" as const] }, { rig: "r" }]) {
|
||||
const { shared, perRow } = listedBothWays(opts as never);
|
||||
expect(shared).toEqual(perRow);
|
||||
}
|
||||
});
|
||||
|
||||
it("an empty listing", () => {
|
||||
const { shared, perRow } = listedBothWays({});
|
||||
expect(shared).toEqual([]);
|
||||
expect(perRow).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("a listing prepares the waiting view's statements once, not once per row", () => {
|
||||
let repo: QueueRepository;
|
||||
beforeEach(() => {
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
repo = new QueueRepository(db, new EventBus(db), { validateRig: () => true, loadHumanRegistry: () => ({ ok: true, entities: [] }) } as never);
|
||||
});
|
||||
afterEach(() => { vi.restoreAllMocks(); db.close(); });
|
||||
|
||||
/** How many times each kind of statement is prepared while `run` executes. */
|
||||
function preparedWhile(run: () => unknown): Record<string, number> {
|
||||
const prepare = db.prepare.bind(db);
|
||||
const kinds: Record<string, (sql: string) => boolean> = {
|
||||
pendingSince: (sql) => sql.includes("SELECT ts, previous_state FROM"),
|
||||
blocker: (sql) => sql.includes("SELECT destination_session, state FROM queue_items WHERE qitem_id = ?"),
|
||||
timer: (sql) => sql.includes("FROM queue_transition_wakes w JOIN watchdog_jobs j"),
|
||||
waitingRow: (sql) => sql.includes("SELECT qitem_id, source_session, destination_session, state, blocked_on, closure_required_at, tags"),
|
||||
schemaCheck: (sql) => /sqlite_master WHERE name = (\?|'queue_transitions'|'queue_transition_wakes')/.test(sql),
|
||||
};
|
||||
const counts = Object.fromEntries(Object.keys(kinds).map((k) => [k, 0]));
|
||||
const spy = vi.spyOn(db, "prepare").mockImplementation(((sql: string) => {
|
||||
for (const [kind, match] of Object.entries(kinds)) if (match(sql)) counts[kind]!++;
|
||||
return prepare(sql);
|
||||
}) as typeof db.prepare);
|
||||
try { run(); } finally { spy.mockRestore(); }
|
||||
return counts;
|
||||
}
|
||||
|
||||
it("30 pending rows, 30 rows blocked on another row and 30 parked on a timer", async () => {
|
||||
const make = (body: string) => repo.create({ sourceSession: "orch@r", destinationSession: "worker@r", body, nudge: false } as never)
|
||||
.then((q) => q.qitemId);
|
||||
for (let i = 0; i < 30; i++) {
|
||||
await make(`pending ${i}`);
|
||||
const blocker = await make(`blocker ${i}`);
|
||||
repo.update({ qitemId: await make(`waits ${i}`), actorSession: "worker@r", state: "blocked", blockedOn: blocker, transitionNote: "continuation: wait" } as never);
|
||||
repo.update({ qitemId: await make(`timed ${i}`), actorSession: "worker@r", state: "blocked", blockedOn: "external:ci", transitionNote: "continuation: CI",
|
||||
wakeAfterSeconds: 600 } as never);
|
||||
}
|
||||
const counts = preparedWhile(() => repo.list({ limit: 500 }));
|
||||
expect(counts.pendingSince).toBeLessThanOrEqual(1);
|
||||
expect(counts.blocker).toBeLessThanOrEqual(1);
|
||||
expect(counts.timer).toBeLessThanOrEqual(1);
|
||||
expect(counts.waitingRow).toBeLessThanOrEqual(1);
|
||||
// lastMeaningfulTransitionMany's two checks and the waiting statements' one per table, whatever the row count.
|
||||
expect(counts.schemaCheck).toBeLessThanOrEqual(4);
|
||||
});
|
||||
});
|
||||
|
||||
describe("a listing reads each owner's activity once", () => {
|
||||
let repo: QueueRepository;
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers({ toFake: ["Date"] });
|
||||
vi.setSystemTime(new Date("2026-11-07T09:30:00.000Z"));
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
repo = new QueueRepository(db, new EventBus(db), { validateRig: () => true, loadHumanRegistry: () => ({ ok: true, entities: [] }) } as never);
|
||||
});
|
||||
afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks(); db.close(); });
|
||||
|
||||
const idle = { activity: "idle-at-prompt", needsInput: { count: 0, reason: null }, decidedBy: "window-sampling" };
|
||||
const make = (destinationSession: string) =>
|
||||
repo.create({ sourceSession: "orch@r", destinationSession, body: "work", nudge: false } as never).then((q) => q.qitemId);
|
||||
|
||||
it("one read per distinct session, for the pickup receipt and the waiting view alike", async () => {
|
||||
for (let i = 0; i < 10; i++) { await make("a@r"); await make("b@r"); }
|
||||
const reader = vi.fn((session: string) => (session ? idle : null) as never);
|
||||
repo.attachActivityReader(reader);
|
||||
repo.list({});
|
||||
expect(reader.mock.calls.map(([s]) => s).sort()).toEqual(["a@r", "b@r"]);
|
||||
repo.list({ compact: true });
|
||||
expect(reader).toHaveBeenCalledTimes(4); // a new listing reads again
|
||||
});
|
||||
|
||||
it("a reader that throws for one session leaves the other sessions' rows as they were, and that session's rows unknown", async () => {
|
||||
await make("ok@r"); await make("broken@r"); await make("ok@r");
|
||||
repo.attachActivityReader((session) => { if (session === "broken@r") throw new Error("fixture: seat state unreadable"); return idle as never; });
|
||||
const shared = repo.list({});
|
||||
const own = repo as unknown as { listingReads: unknown };
|
||||
own.listingReads = () => undefined;
|
||||
const perRow = repo.list({});
|
||||
delete (own as { listingReads?: unknown }).listingReads;
|
||||
expect(shared).toEqual(perRow);
|
||||
const liveness = (owner: string) => shared.filter((i) => i.destinationSession === owner).map((i) => i.waiting?.liveness.confidence);
|
||||
expect(liveness("ok@r")).toEqual(["oracle", "oracle"]);
|
||||
expect(liveness("broken@r")).toEqual(["unknown"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("a listed row's waiting view reads the row in hand instead of reading it again", () => {
|
||||
let repo: QueueRepository;
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers({ toFake: ["Date"] });
|
||||
vi.setSystemTime(new Date("2026-11-07T09:30:00.000Z"));
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
repo = new QueueRepository(db, new EventBus(db), { validateRig: () => true, loadHumanRegistry: () => ({ ok: true, entities: [] }) } as never);
|
||||
});
|
||||
afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks(); db.close(); });
|
||||
|
||||
const make = (destinationSession: string, body = "work") =>
|
||||
repo.create({ sourceSession: "orch@r", destinationSession, body, nudge: false } as never).then((q) => q.qitemId);
|
||||
const rereads = (sql: string) =>
|
||||
sql.includes("SELECT qitem_id, source_session, destination_session, state, blocked_on, closure_required_at, tags")
|
||||
|| /SELECT state, (human_intent|NULL AS human_intent), destination_session, last_nudge_result FROM queue_items WHERE qitem_id = \?/.test(sql)
|
||||
|| sql.includes("SELECT last_nudge_result FROM queue_items WHERE qitem_id = ?");
|
||||
|
||||
it("prepares none of the waiting view's own-row reads, and projects what the per-row read projects", async () => {
|
||||
const plain = await make("worker@r");
|
||||
const held = await make("worker@r");
|
||||
db.prepare("UPDATE queue_items SET last_nudge_result = 'retained:typing_guard' WHERE qitem_id = ?").run(held);
|
||||
const claimed = await make("worker@r");
|
||||
repo.update({ qitemId: claimed, actorSession: "worker@r", state: "in-progress" } as never);
|
||||
const prepare = db.prepare.bind(db);
|
||||
let n = 0;
|
||||
const spy = vi.spyOn(db, "prepare").mockImplementation(((sql: string) => { if (rereads(sql)) n++; return prepare(sql); }) as typeof db.prepare);
|
||||
const shared = repo.list({});
|
||||
spy.mockRestore();
|
||||
expect(n).toBe(0);
|
||||
const own = repo as unknown as { listingReads: unknown };
|
||||
own.listingReads = () => undefined;
|
||||
const perRow = repo.list({});
|
||||
delete (own as { listingReads?: unknown }).listingReads;
|
||||
expect(shared).toEqual(perRow);
|
||||
expect(shared.find((i) => i.qitemId === held)?.waiting?.nextBackstop).toMatchObject({ mechanism: "held by the seat's typing guard", held: true });
|
||||
expect(shared.find((i) => i.qitemId === plain)?.waiting?.state).toBe("pending");
|
||||
});
|
||||
|
||||
it("a single-row read outside a listing still reads its own row", async () => {
|
||||
const q = await make("worker@r");
|
||||
const prepare = db.prepare.bind(db);
|
||||
let n = 0;
|
||||
const spy = vi.spyOn(db, "prepare").mockImplementation(((sql: string) => { if (rereads(sql)) n++; return prepare(sql); }) as typeof db.prepare);
|
||||
expect(repo.waitingView(q)?.state).toBe("pending");
|
||||
spy.mockRestore();
|
||||
expect(n).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("show (getById) is the listing projection of one row", () => {
|
||||
let repo: QueueRepository;
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers({ toFake: ["Date"] });
|
||||
vi.setSystemTime(new Date("2026-11-07T09:30:00.000Z"));
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
repo = new QueueRepository(db, new EventBus(db), { validateRig: () => true, loadHumanRegistry: () => ({ ok: true, entities: [] }) } as never);
|
||||
});
|
||||
afterEach(() => { vi.useRealTimers(); vi.restoreAllMocks(); db.close(); });
|
||||
|
||||
/** getById as it read before: the per-row projection, then the delivery outcome read on its own. */
|
||||
function getByIdPerRow(qitemId: string) {
|
||||
const own = repo as unknown as { rowToItem: (row: unknown) => Record<string, unknown> };
|
||||
const row = db.prepare("SELECT * FROM queue_items WHERE qitem_id = ?").get(qitemId);
|
||||
const item = own.rowToItem.call(repo, row);
|
||||
const ledger = repo.deliveryOutcomeFor(qitemId);
|
||||
return { ...item, deliveryOutcome: ledger?.outcome ?? null, ...(ledger && ledger.outcome !== "posted" && ledger.outcome !== "accepted" ? { deliveryFailureDetail: ledger.detail } : {}) };
|
||||
}
|
||||
|
||||
it("answers what the per-row projection answered: pending, in-progress, blocked, parked, done and handed-off", async () => {
|
||||
const make = (destinationSession: string, body: string) =>
|
||||
repo.create({ sourceSession: "orch@r", destinationSession, body, nudge: false } as never).then((q) => q.qitemId);
|
||||
const fresh = await make("worker@r", "fresh");
|
||||
const claimed = await make("worker@r", "claimed");
|
||||
repo.update({ qitemId: claimed, actorSession: "worker@r", state: "in-progress" } as never);
|
||||
const blocker = await make("other@r", "the blocker");
|
||||
const waiter = await make("worker@r", "waits");
|
||||
repo.update({ qitemId: waiter, actorSession: "worker@r", state: "blocked", blockedOn: blocker, transitionNote: "continuation: wait" } as never);
|
||||
const timed = await make("worker@r", "timer");
|
||||
repo.update({ qitemId: timed, actorSession: "worker@r", state: "blocked", blockedOn: "external:ci", transitionNote: "continuation: CI",
|
||||
wakeAfterSeconds: 600 } as never);
|
||||
const done = await make("worker@r", "done");
|
||||
repo.update({ qitemId: done, actorSession: "worker@r", state: "done", closureReason: "no-follow-on" } as never);
|
||||
const passed = await make("worker@r", "passed on");
|
||||
const { created: successor } = await repo.handoff({ qitemId: passed, fromSession: "worker@r", toSession: "next@r", nudge: false });
|
||||
const ids = [fresh, claimed, blocker, waiter, timed, done, passed, successor.qitemId];
|
||||
// Each state the face can show is here, so the comparison below covers it.
|
||||
expect(ids.map((id) => repo.getById(id)?.state)).toEqual(
|
||||
["pending", "in-progress", "pending", "blocked", "blocked", "done", "handed-off", "pending"]);
|
||||
expect(repo.getById(timed)?.pickup?.state).toBe("parked");
|
||||
for (const id of ids) expect(repo.getById(id)).toEqual(getByIdPerRow(id));
|
||||
expect(repo.getById("qitem-missing")).toBeNull();
|
||||
});
|
||||
|
||||
it("reads the row's last meaningful transition once, not once for the pickup and once for the waiting view", async () => {
|
||||
const q = (await repo.create({ sourceSession: "orch@r", destinationSession: "worker@r", body: "work", nudge: false } as never)).qitemId;
|
||||
const prepare = db.prepare.bind(db);
|
||||
let n = 0;
|
||||
const spy = vi.spyOn(db, "prepare").mockImplementation(((sql: string) => {
|
||||
if (sql.includes("actor_session NOT IN ('watchdog@system'")) n++;
|
||||
return prepare(sql);
|
||||
}) as typeof db.prepare);
|
||||
repo.getById(q);
|
||||
spy.mockRestore();
|
||||
expect(n).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -2,7 +2,6 @@ import { describe, expect, it } from "vitest";
|
||||
import { execFile, spawnSync } from "node:child_process";
|
||||
import { resolve } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { shellQuote } from "../src/adapters/shell-quote.js";
|
||||
|
||||
// The runner conformance check that ships in docs/reference, run against a test double that keeps the contract, and
|
||||
// against copies of it that each break one promise: the check must pass the first and catch every break.
|
||||
@@ -16,7 +15,7 @@ type Result = { id: string; status: "pass" | "fail" | "skip" | "advisory"; met?:
|
||||
type Report = { results: Result[]; summary: { pass: number; fail: number; skip: number; advisory: number } };
|
||||
|
||||
async function check(args: string[], runnerFlags = "", nodeArgs: string[] = [], timeoutMs = 90_000): Promise<{ code: number; report: Report }> {
|
||||
const runner = `${shellQuote(process.execPath)} ${shellQuote(RUNNER)} --agent {agent}${runnerFlags}`;
|
||||
const runner = `${process.execPath} ${RUNNER} --agent {agent}${runnerFlags}`;
|
||||
try {
|
||||
const { stdout } = await promisify(execFile)(process.execPath, [...nodeArgs, CHECK, "--runner", runner, "--name", "minimal", "--json", ...args], { timeout: timeoutMs });
|
||||
return { code: 0, report: JSON.parse(stdout) };
|
||||
|
||||
@@ -85,29 +85,3 @@ describe("parseJsonlExchanges — claude-projects role/content shape", () => {
|
||||
expect(parseJsonlExchanges(p, 10)).toEqual([{ role: "user", content: "codex hello" }]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("recap tail boundaries", () => {
|
||||
it.each(["", "\n", "\r\n"])("keeps chronological order with trailing %j", (ending) => {
|
||||
const p = fixture([]);
|
||||
const message = (content: string) => JSON.stringify({ message: { role: "user", content } });
|
||||
writeFileSync(p, [message("old"), message("first retained"), "{partial", "", JSON.stringify({ type: "metadata" }), message("last retained")].join("\r\n") + ending);
|
||||
expect(parseJsonlExchanges(p, 2)).toEqual([
|
||||
{ role: "user", content: "first retained" },
|
||||
{ role: "user", content: "last retained" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("returns the first line when it is the only visible exchange", () => {
|
||||
const p = fixture([{ message: { role: "user", content: "first" } }, { type: "metadata" }]);
|
||||
expect(parseJsonlExchanges(p, 6)).toEqual([{ role: "user", content: "first" }]);
|
||||
expect(parseJsonlExchanges(p, 0)).toEqual([]);
|
||||
});
|
||||
|
||||
it("returns only the latest exchanges from a long record", () => {
|
||||
const p = fixture(Array.from({ length: 20_000 }, (_, i) => ({ message: { role: "user", content: String(i) } })));
|
||||
expect(parseJsonlExchanges(p, 2)).toEqual([
|
||||
{ role: "user", content: "19998" },
|
||||
{ role: "user", content: "19999" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -77,7 +77,6 @@ describe("telemetry routes", () => {
|
||||
["since=garbage", [], ['since "garbage"']],
|
||||
["until=garbage", [T10, T10, T12, T12], ['until "garbage"']],
|
||||
["since=2026-13-45", [], ['since "2026-13-45"']],
|
||||
["since=2026-08-07T11%2B25:00", [T12, T12], ['since "2026-08-07T11+25:00"']],
|
||||
["since=garbage&until=garbage", [], ['since "garbage"', 'until "garbage"']],
|
||||
["since=2026-08-07T11", [T12, T12], []],
|
||||
["until=2026-08-07T11", [T10, T10], []],
|
||||
|
||||
@@ -5,7 +5,6 @@ describe("time bound advisory", () => {
|
||||
it.each([
|
||||
"2026", "2026-10", "2026-10-14", "2026-10-14T09", "2026-10-14T09:30", "2026-10-14T09:30:00",
|
||||
"2026-10-14T09:30:00.123Z", "2026-10-14T09Z", "2026-10-14T09:30:00+05:30", "2024-02-29T23:59:59Z",
|
||||
"2026-10-14T09:30:00-00:00", "2026-10-14T09:30:00+23:59",
|
||||
])("recognises %s as a date or time", (value) => {
|
||||
expect(isRecognisedDateOrTime(value)).toBe(true);
|
||||
expect(timeBoundAdvisory("since", value, "installedAt")).toBeUndefined();
|
||||
@@ -14,7 +13,6 @@ describe("time bound advisory", () => {
|
||||
it.each([
|
||||
"garbage", "2026-13-45", "2026-00-10", "2026-02-30", "2025-02-29", "2026-10-14T24", "2026-10-14T09:60",
|
||||
"Oct 14 2026", "2026-10-14 09:00", "1760000000", " 2026-10-14",
|
||||
"2026-10-14T09:30:00+24:00", "2026-10-14T09:30:00-01:60", "2026-10-14T09+99:99",
|
||||
])("does not recognise %s", (value) => {
|
||||
expect(isRecognisedDateOrTime(value)).toBe(false);
|
||||
expect(timeBoundAdvisory("since", value, "installedAt")).toBe(
|
||||
|
||||
@@ -138,10 +138,15 @@ describe("config routes (User Settings v0)", () => {
|
||||
// + 1 acp.to (the ACP agent's receiver; default empty).
|
||||
// + 1 daemon.seat_poll_interval_ms (#2687 the seat pollers' cadence; startup-only, default 1000).
|
||||
// + 1 retention.events_days (the history-only event types of events; default 0 = keep).
|
||||
expect(Object.keys(body.settings).length).toBe(100);
|
||||
// + 2 retention.closed_content_days, retention.closed_metadata_days (closed-work erasure; default 0 = keep).
|
||||
// + 1 retention.erasure_ledger (queue erasure's replay ledger; default empty = none).
|
||||
expect(Object.keys(body.settings).length).toBe(103);
|
||||
expect(body.settings["launch.codex_login_shell"]).toMatchObject({ value: false, source: "default", defaultValue: false });
|
||||
expect(body.settings["transport.draft_delivery"]).toMatchObject({ value: "deliver", source: "default", defaultValue: "deliver" });
|
||||
expect(body.settings["retention.events_days"]).toMatchObject({ value: 0, source: "default", defaultValue: 0 });
|
||||
expect(body.settings["retention.closed_content_days"]).toMatchObject({ value: 0, source: "default", defaultValue: 0 });
|
||||
expect(body.settings["retention.closed_metadata_days"]).toMatchObject({ value: 0, source: "default", defaultValue: 0 });
|
||||
expect(body.settings["retention.erasure_ledger"]).toMatchObject({ value: "", source: "default", defaultValue: "" });
|
||||
expect(body.settings["specs.builtins_visible"]).toMatchObject({ value: true, source: "default", defaultValue: true });
|
||||
expect(body.settings["stream.intake"]).toMatchObject({ value: "queue.worker", source: "default", defaultValue: "queue.worker" });
|
||||
expect(body.settings["retention.stream_days"]).toMatchObject({ value: 30, source: "default", defaultValue: 30 });
|
||||
|
||||
@@ -196,6 +196,9 @@ describe("SettingsStore (User Settings v0)", () => {
|
||||
"retention.watchdog_days",
|
||||
"retention.usage_samples_days", // 51-08 A2 (plan-lock rev-1): the PM-ruled 14d telemetry window
|
||||
"retention.events_days", // the history-only event types of events
|
||||
"retention.closed_content_days", // closed-work erasure: content
|
||||
"retention.closed_metadata_days", // closed-work erasure: the row, keeping its id
|
||||
"retention.erasure_ledger", // queue erasure's replay ledger file
|
||||
"retention.watchdog_keep_per_job",
|
||||
"retention.stream_days", // slice 22 outcome 9: stream retention
|
||||
"retention.stream_max_items",
|
||||
@@ -268,6 +271,24 @@ describe("SettingsStore (User Settings v0)", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
["retention.closed_content_days", "closedContentDays", "OPENRIG_RETENTION_CLOSED_CONTENT_DAYS"],
|
||||
["retention.closed_metadata_days", "closedMetadataDays", "OPENRIG_RETENTION_CLOSED_METADATA_DAYS"],
|
||||
] as const)("%s defaults to 0 (keep), resolves file then env, and refuses negative or non-integer values", (key, fileKey, envKey) => {
|
||||
const store = new SettingsStore(configPath);
|
||||
expect(store.resolveOne(key)).toMatchObject({ value: 0, source: "default" });
|
||||
store.set(key, "30");
|
||||
expect(store.resolveOne(key)).toMatchObject({ value: 30, source: "file" });
|
||||
expect(JSON.parse(readFileSync(configPath, "utf-8")).retention[fileKey]).toBe(30);
|
||||
process.env[envKey] = "7";
|
||||
try {
|
||||
expect(store.resolveOne(key)).toMatchObject({ value: 7, source: "env" });
|
||||
} finally {
|
||||
delete process.env[envKey];
|
||||
}
|
||||
for (const raw of ["-1", "2.5", "30days"]) expect(() => store.set(key, raw)).toThrow(new RegExp(key.replace(".", "\\.")));
|
||||
});
|
||||
|
||||
it("retention.events_days defaults to 0 (keep), resolves file then env, accepts 0 and refuses negative or non-integer values", () => {
|
||||
const store = new SettingsStore(configPath);
|
||||
expect(store.resolveOne("retention.events_days")).toMatchObject({ value: 0, source: "default" });
|
||||
|
||||
@@ -60,7 +60,7 @@ const LABELS: Record<string, string> = {
|
||||
"queue.stuck_sweep_unclaimed_age_minutes": "Unclaimed age", "policies.idle_gate_qitem.auto_register": "Auto-registration",
|
||||
"snapshots.periodic.enabled": "Periodic snapshots", "snapshots.periodic.interval_seconds": "Snapshot interval",
|
||||
"snapshots.periodic.retention_keep": "Snapshots to retain", "ui.terminal.max_live_terminals": "Legacy web terminal limit",
|
||||
"retention.usage_samples_days": "Usage sample retention", "retention.events_days": "Event history retention (days)",
|
||||
"retention.usage_samples_days": "Usage sample retention", "retention.events_days": "Event history retention (days)", "retention.closed_content_days": "Closed work content retention (days)", "retention.closed_metadata_days": "Closed work row retention (days)", "retention.erasure_ledger": "Erasure ledger file",
|
||||
"retention.stream_days": "Stream retention (days)", "retention.stream_max_items": "Stream items kept",
|
||||
"transcripts.poll_interval_seconds": "Transcript refresh", "transcripts.lines": "Transcript lines",
|
||||
"ui.preview.refresh_interval_seconds": "Preview refresh", "ui.preview.max_pins": "Preview pins",
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import { existsSync, realpathSync } from "node:fs";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
/**
|
||||
* Exact tracked docs paths allowed outside the three durable roots.
|
||||
@@ -59,6 +57,6 @@ export function main() {
|
||||
process.exitCode = 1;
|
||||
}
|
||||
|
||||
if (process.argv[1] && existsSync(process.argv[1]) && realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url))) {
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
main();
|
||||
}
|
||||
@@ -1,8 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { existsSync, mkdirSync, readFileSync, readdirSync, realpathSync, writeFileSync } from "node:fs";
|
||||
import { existsSync, mkdirSync, readFileSync, readdirSync, writeFileSync } from "node:fs";
|
||||
import { dirname, isAbsolute, relative, resolve } from "node:path";
|
||||
import {
|
||||
buildInternalLeakMessage,
|
||||
@@ -201,7 +200,7 @@ function splitNul(bytes) {
|
||||
.sort();
|
||||
}
|
||||
|
||||
if (process.argv[1] && existsSync(process.argv[1]) && realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url))) {
|
||||
if (import.meta.url === `file://${resolve(process.argv[1])}`) {
|
||||
try {
|
||||
process.exitCode = main();
|
||||
} catch (error) {
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
chmodSync,
|
||||
@@ -9,7 +8,6 @@ import {
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
readdirSync,
|
||||
realpathSync,
|
||||
rmSync,
|
||||
lstatSync,
|
||||
writeFileSync,
|
||||
@@ -557,7 +555,7 @@ export async function main(argv = process.argv.slice(2), dependencies = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
if (process.argv[1] && existsSync(process.argv[1]) && realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url))) {
|
||||
if (import.meta.url === `file://${resolve(process.argv[1])}`) {
|
||||
try {
|
||||
await main();
|
||||
} catch (err) {
|
||||
|
||||
@@ -1,84 +0,0 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { execFileSync, spawnSync } from "node:child_process";
|
||||
import { copyFileSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
const entrypoints = ["check-docs-guard.mjs", "check-internal-leak-guard.mjs", "mirror-skills.mjs"];
|
||||
|
||||
function fixture(t) {
|
||||
const root = mkdtempSync(join(tmpdir(), "openrig scripts # "));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const scripts = join(root, "scripts");
|
||||
mkdirSync(scripts);
|
||||
for (const file of [...entrypoints, "internal-leak-scanner.mjs", "leak-guard-rules.mjs"]) {
|
||||
copyFileSync(new URL(file, import.meta.url), join(scripts, file));
|
||||
}
|
||||
const env = { ...process.env };
|
||||
delete env.OPENRIG_LEAK_GUARD_RULES;
|
||||
return {
|
||||
root,
|
||||
run: (file, ...args) => spawnSync(process.execPath, [join(scripts, file), ...args], {
|
||||
cwd: root, encoding: "utf8", env,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
test("docs guard rejects tracked plans from a checkout path containing spaces and #", (t) => {
|
||||
const { root, run } = fixture(t);
|
||||
mkdirSync(join(root, "docs/plans"), { recursive: true });
|
||||
writeFileSync(join(root, "docs/plans/example.md"), "local plan\n");
|
||||
execFileSync("git", ["init", "-q"], { cwd: root });
|
||||
execFileSync("git", ["add", "."], { cwd: root });
|
||||
const result = run("check-docs-guard.mjs");
|
||||
assert.equal(result.status, 1, result.stderr);
|
||||
assert.match(result.stderr, /docs\/plans\/example\.md/);
|
||||
});
|
||||
|
||||
test("leak guard reports a missing rules file instead of silently succeeding", (t) => {
|
||||
const { root, run } = fixture(t);
|
||||
const result = run("check-internal-leak-guard.mjs", "--repo", root, "--rules", join(root, "missing.json"), "--mode", "full");
|
||||
assert.equal(result.status, 1, result.stderr);
|
||||
assert.match(result.stderr, /Rules file not found/);
|
||||
});
|
||||
|
||||
test("mirror check reports missing generated inputs instead of silently succeeding", (t) => {
|
||||
const { run } = fixture(t);
|
||||
const result = run("mirror-skills.mjs", "--check");
|
||||
assert.equal(result.status, 1, result.stderr);
|
||||
assert.match(result.stderr, /ENOENT.*skill-edge-layout\.generated\.json/);
|
||||
});
|
||||
|
||||
for (const file of entrypoints) {
|
||||
for (const args of [[], ["missing-entrypoint.mjs"]]) {
|
||||
test(`${file} can be imported with ${args.length ? "a nonexistent" : "no"} entrypoint argument`, (t) => {
|
||||
const { root } = fixture(t);
|
||||
const url = pathToFileURL(join(root, "scripts", file)).href;
|
||||
const result = spawnSync(process.execPath, ["--input-type=module", "-e", `await import(${JSON.stringify(url)})`, ...args], {
|
||||
cwd: root, encoding: "utf8",
|
||||
});
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
assert.equal(result.stdout, "");
|
||||
assert.equal(result.stderr, "");
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
for (const [file, args, error] of [
|
||||
["check-docs-guard.mjs", [], /not a git repository/],
|
||||
["check-internal-leak-guard.mjs", ["--repo", ".", "--rules", "missing.json", "--mode", "full"], /Rules file not found/],
|
||||
["mirror-skills.mjs", ["--check"], /ENOENT.*skill-edge-layout\.generated\.json/],
|
||||
]) {
|
||||
test(`${file} runs through a preserved symlink`, (t) => {
|
||||
const { root } = fixture(t);
|
||||
const link = join(root, "scripts", "linked.mjs");
|
||||
symlinkSync(file, link);
|
||||
const result = spawnSync(process.execPath, ["--preserve-symlinks-main", link, ...args], {
|
||||
cwd: root, encoding: "utf8",
|
||||
});
|
||||
assert.equal(result.status, 1, result.stderr);
|
||||
assert.match(result.stderr, error);
|
||||
});
|
||||
}
|
||||
Reference in new issue
Block a user