mirror of
https://github.com/mvschwarz/openrig.git
synced 2026-11-11 17:32:22 -10:00
Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1d6f5a2d24 | ||
|
|
bf31a9229d | ||
|
|
9e58308c0c | ||
|
|
8df4672263 | ||
|
|
eb6a218010 | ||
|
|
559025c090 | ||
|
|
37e0715c58 | ||
|
|
a8397dd711 | ||
|
|
e35eec247b | ||
|
|
96fc0f3e88 | ||
|
|
f95fa4cd77 |
No files matched your search
+3
-1
@@ -152,7 +152,9 @@ OpenRig. Two repairs aren't in this release: see Known limits.
|
||||
and the stop wasn't exercised. For an adopted session, such as a seat joined
|
||||
with `rig attach --self`, `rig seat stop` still suggests `rig down`, which
|
||||
leaves that terminal running too. Release it with `rig unclaim <session>`
|
||||
(that leaves its process running), then close it in its own terminal. ([#2962](https://github.com/mvschwarz/openrig/pull/2962))
|
||||
(that leaves its process running), then end that process. If it runs in tmux,
|
||||
also end the underlying tmux session, including any remaining shell; closing
|
||||
only its viewing window leaves that session running. ([#2962](https://github.com/mvschwarz/openrig/pull/2962))
|
||||
- **Codex working counter:** a capture that doesn't name its pane keeps the old
|
||||
comparison, a restart that keeps both the pane and its process id isn't seen,
|
||||
and a leftover live frame can count once until it expires. ([#2980](https://github.com/mvschwarz/openrig/pull/2980))
|
||||
|
||||
@@ -140,7 +140,11 @@ that gateway side too. For the pane side, read:
|
||||
|
||||
- `lastNudgeWireResult`: the wake send's result as stored (for example
|
||||
`verified`, `delivered-ack-pending`, `indeterminate:…`, `retained:…`,
|
||||
`failed:…`). A claim never rewrites it; a later wake replaces it.
|
||||
`failed:…`). A claim never rewrites it; a later wake replaces it. Each stored
|
||||
form reads as one delivery fact (`readWakeOutcome`, `connection-contract.ts`).
|
||||
A result no reader recognises reads `unknown`, never delivered: the wake ladder
|
||||
treats it as an unconfirmed wake, which escalates past the window without a
|
||||
re-send.
|
||||
- `lastNudgeResult`: the result as presented. It is the stored result, except
|
||||
that a claimed row whose stored result is `delivered-ack-pending` or
|
||||
`indeterminate:…` reads `claimed` (`presentedNudgeResult`). Unclaiming shows
|
||||
|
||||
@@ -8,7 +8,67 @@ with one grant per participant. The network connector puts one selected local se
|
||||
The connector opens no port: your daemon makes every connection outward, and the hub never connects to your machine. Work the hub
|
||||
sends that participant runs on your seat, under your accounts, and the result goes back to the hub item.
|
||||
|
||||
## Connect
|
||||
## Enroll in an account, then connect by name
|
||||
|
||||
When your hub is an account's network, enroll this daemon once with the one-time code its page shows, then connect
|
||||
seats by name:
|
||||
|
||||
```bash
|
||||
rig network enroll <code> --site <site URL>
|
||||
rig network connect <seat> --as <name>
|
||||
```
|
||||
|
||||
- `enroll` trades the code, with this daemon's host id, for an installation key. The key is stored like a participant's
|
||||
token, under `<OpenRig home>/network-secrets/<installation>/key.json`, and is never returned or shown. A code is
|
||||
single-use and short-lived; a refused one says why, so get a new one from the page. If the account doesn't answer,
|
||||
whether it took the code is unknown: try again with the same code, and get a new one if that's refused.
|
||||
- Enrolling trusts the site with where your connected seats' work goes: for each name, the account answers the front
|
||||
and rig the binding reaches.
|
||||
- The installation key goes only to the site you enrolled at: a redirect from it isn't followed. Each `enroll`,
|
||||
`connect` or `disconnect` spends at most 30 seconds with the account in all, and the command waits 45 seconds for the
|
||||
daemon, so you see the account's own answer or error.
|
||||
- The account keeps one list of the installation's names, and every change sends the whole list. Changes to it run one
|
||||
at a time on this daemon, each reading the names to keep in its turn, so two connects started together both stay
|
||||
connected.
|
||||
- `connect --as <name>` lists the name with the account, which answers that participant's bearer, its MCP front and its
|
||||
rig. The binding then reaches that front as `<name>@<rig>`, the same as a direct connect. A name is lowercase letters,
|
||||
digits and inner dashes, up to 32. One that another connection to the account uses is refused.
|
||||
- If the answer to a connect is lost, the next connect drops the name and lists it again, which gets a fresh bearer and
|
||||
ends the old one.
|
||||
- If the account listed the name but the binding can't be made here, OpenRig asks the account to drop the name again.
|
||||
That's best effort: if it fails too, the error says the account may still list the name.
|
||||
- `disconnect` ends the binding here, then has the account stop listing the name, which revokes it there. Once the
|
||||
account's answer no longer lists the name, `disconnect` says the account stopped listing it. If the account can't be
|
||||
reached or its answer is lost, the binding is still ended, the error says OpenRig couldn't confirm the account dropped
|
||||
the name and names the `rig network disconnect <binding id>` to run again, and `rig network status` keeps showing it
|
||||
as unconfirmed. A later answer from the same account installation that no longer lists the name marks it dropped.
|
||||
- `rig network status` shows the enrollment above the bindings.
|
||||
|
||||
### Enroll again after a lost key or a removed installation
|
||||
|
||||
When the installation key is missing, or the account no longer recognizes the installation, the error says to enroll
|
||||
again with a new code from the account page:
|
||||
|
||||
```bash
|
||||
rig network enroll <code> --site <site URL> --replace
|
||||
```
|
||||
|
||||
Without `--replace`, a second enroll at the same site is refused and the code isn't spent. With it, the replace first
|
||||
waits for any other change to this enrollment to finish, within the same 30 seconds; one that runs out of time while
|
||||
waiting says so and doesn't send the code. The account makes a new installation, with a new key, for the new code. Here,
|
||||
that key takes the old one's place and the bindings made through the old installation end, in one step:
|
||||
|
||||
- If that step fails after the account accepted the code, nothing here changes, and the error says so. The old
|
||||
enrollment keeps working only while the account keeps the old installation, and the account now has a new
|
||||
installation whose key nothing here holds: remove it on the account page, then enroll again with a new code.
|
||||
- Once the step is done, `enroll` names the bindings it ended. The old installation may still list their names, and the
|
||||
new key can't drop them, so remove them on the account page. `rig network status` marks each of them as possibly still
|
||||
listed under the old installation, as it does a binding that ended earlier without the account confirming the drop:
|
||||
only the installation a name was listed under can confirm it.
|
||||
- If cleanup after the step doesn't finish (removing the old key's backup, or an ended binding's credential), `enroll`
|
||||
reports it, and the replace still stands.
|
||||
|
||||
## Connect with a hub URL and a token
|
||||
|
||||
```bash
|
||||
rig network connect <seat> --hub <url> --as <participant> --token-file <path>
|
||||
@@ -69,8 +129,9 @@ rig queue update <local item> --state done --closure-reason no-follow-on --resol
|
||||
rig network status [--json]
|
||||
```
|
||||
|
||||
Each binding shows its seat, participant, hub, state, and its last contact or last error. `--json` adds the
|
||||
connector's view: each binding's next pass, backoff and last pass.
|
||||
Each binding shows its seat, participant, hub, state, and its last contact or last error. An ended binding made through
|
||||
the account also says when the account hasn't confirmed it dropped the name, or when the name may still be listed under
|
||||
a replaced installation. `--json` adds the connector's view: each binding's next pass, backoff and last pass.
|
||||
|
||||
## When things go wrong
|
||||
|
||||
@@ -88,7 +149,8 @@ rig network disconnect <binding-or-seat> [--reason "<why>"]
|
||||
```
|
||||
|
||||
The binding ends here, so no new work comes in through it, and its credential is removed. Local items and results
|
||||
stay. The hub still lists the participant until it's removed there too.
|
||||
stay. A binding made through the account is then dropped there too (see above). For a binding made with a hub URL and
|
||||
a token, the hub still lists the participant until it's removed there too.
|
||||
|
||||
If the credential can't be removed, the binding still ends, and the error names the
|
||||
`rig network disconnect <binding id>` to run again. The daemon also removes credentials whose binding isn't active
|
||||
@@ -104,11 +166,14 @@ each time it starts.
|
||||
An atomic close-if-open needs the hub to offer a conditional close.
|
||||
- After a disconnect or a revocation, work already taken keeps running here, but its pickup and result no longer reach
|
||||
the hub item, which stays at its last reported stage.
|
||||
- If the daemon stops between storing a new installation key and recording the enrollment, that key's folder under
|
||||
`network-secrets/` stays with no enrollment, and nothing removes it. It's private to you and unused, so remove it by
|
||||
hand.
|
||||
- **Plain http needs your choice.** `rig network connect` refuses an `http:` hub URL until you add `--allow-http`, because
|
||||
the participant's token and its work then cross the network unencrypted, where they can be read. Use the hub's
|
||||
`https:` URL where it has one. The choice is recorded on that binding, so the connector doesn't ask again, and
|
||||
`rig network status` (and its `--json`) marks the binding unencrypted. OpenRig doesn't judge the address: a private or
|
||||
tailnet address doesn't prove the connection is protected.
|
||||
tailnet address doesn't prove the connection is protected. `rig network enroll` asks the same for a plain-http site.
|
||||
- **Redirects:** the connector follows a hub's redirect only within its own origin, or to its `https:` form on the same
|
||||
host. Any other redirect stops the connection with its target named, and the token isn't sent there. Connect the
|
||||
binding to the hub's own URL.
|
||||
@@ -15,8 +15,25 @@ interface Binding {
|
||||
lastContactAt: string | null; lastError: string | null; unencrypted?: boolean;
|
||||
/** Hub items someone else closed before this binding's result reached them. */
|
||||
closedElsewhere?: Array<{ hubItem: string; localItem: string }>;
|
||||
participantName?: string | null;
|
||||
/** For an ended binding made through the account: whether the account confirmed it stopped listing the name. */
|
||||
accountRemoval?: "confirmed" | "unconfirmed" | "old-installation" | null;
|
||||
}
|
||||
|
||||
/** The network writes can wait on the account, which the daemon bounds at 30 s in all; the CLI waits past that so the
|
||||
* account's own answer is shown rather than a daemon timeout. */
|
||||
const ACCOUNT_WRITE_TIMEOUT_MS = 45_000;
|
||||
|
||||
const accountNote = (b: Binding): string => {
|
||||
if (b.accountRemoval === "unconfirmed") return `\n couldn't confirm the account dropped ${b.participantName}: run rig network disconnect ${b.id} again`;
|
||||
if (b.accountRemoval === "old-installation") return `\n ${b.participantName} may still be listed under the old installation: remove it on the account page`;
|
||||
return "";
|
||||
};
|
||||
|
||||
interface Installation { id: string; site: string; installationId: string; name: string | null }
|
||||
|
||||
const enrolled = (i: Installation) => `Enrolled at ${i.site} as installation ${i.name ?? i.installationId} (${i.id})`;
|
||||
|
||||
/** The participant's token, read from the file the account gave the person. A file others can read is used, with a note. */
|
||||
function defaultReadTokenFile(path: string): string {
|
||||
const stat = lstatSync(path);
|
||||
@@ -30,7 +47,7 @@ function line(b: Binding): string {
|
||||
const state = b.state === "active" ? contact : `${b.state}${b.stateReason ? ` (${b.stateReason})` : ""}`;
|
||||
const plain = b.unencrypted ? " UNENCRYPTED (http, allowed for this binding)" : "";
|
||||
const conflicts = (b.closedElsewhere ?? []).map((c) => `\n hub item ${c.hubItem} was closed elsewhere before its result arrived; the result stays on ${c.localItem}`).join("");
|
||||
return `${b.id} ${b.localSession} -> ${b.hubSeat} at ${b.hubUrl}${plain} ${state}${b.lastError && b.state === "active" ? ` last error: ${b.lastError}` : ""}${conflicts}`;
|
||||
return `${b.id} ${b.localSession} -> ${b.hubSeat} at ${b.hubUrl}${plain} ${state}${b.lastError && b.state === "active" ? ` last error: ${b.lastError}` : ""}${conflicts}${accountNote(b)}`;
|
||||
}
|
||||
|
||||
/** `rig network`: connect a selected local seat to a participant of a remote hub (an account's network rig, or
|
||||
@@ -56,25 +73,60 @@ export function networkCommand(depsOverride?: NetworkDeps): Command {
|
||||
console.log(json ? JSON.stringify(data, null, 2) : ok(data as Record<string, unknown>));
|
||||
};
|
||||
|
||||
cmd.command("enroll")
|
||||
.description("Enroll this daemon in an account with the one-time code its page shows, so seats connect by name")
|
||||
.argument("<code>", "The one-time code from the account page")
|
||||
.requiredOption("--site <url>", "The account's site")
|
||||
.option("--allow-http", "Accept a plain-http site: the installation key and tokens cross the network unencrypted")
|
||||
.option("--replace", "Replace this daemon's enrollment at the site, once the account accepts the new code (after a lost key or a removed installation)")
|
||||
.option("--json", "Print JSON")
|
||||
.action(async (code: string, opts: { site: string; allowHttp?: boolean; replace?: boolean; json?: boolean }) => {
|
||||
const daemon = await client();
|
||||
if (!daemon) return;
|
||||
const res = await daemon.post<Record<string, unknown>>("/api/network/installations", { code, site: opts.site, allowHttp: opts.allowHttp === true, replace: opts.replace === true },
|
||||
{ headers: operatorAuthHeaders(), timeoutMs: ACCOUNT_WRITE_TIMEOUT_MS });
|
||||
finish(res.status, res.data, opts.json, (i) => {
|
||||
const replaced = (i as { replaced?: { endedBindings?: Array<{ id: string; localSession: string; participantName: string | null }>; problems?: string[] } }).replaced;
|
||||
const ended = replaced?.endedBindings ?? [];
|
||||
const lines = [enrolled(i as unknown as Installation)];
|
||||
if (ended.length) {
|
||||
lines.push(`Ended ${ended.length} binding${ended.length === 1 ? "" : "s"} made through the old installation; their names may still be listed under it, so remove them on the account page:`);
|
||||
for (const b of ended) lines.push(` ${b.id} ${b.localSession} as ${b.participantName}`);
|
||||
}
|
||||
for (const problem of replaced?.problems ?? []) lines.push(`Replaced, with cleanup left over: ${problem}`);
|
||||
lines.push("Connect a seat with rig network connect <seat> --as <name>.");
|
||||
return lines.join("\n");
|
||||
});
|
||||
});
|
||||
|
||||
cmd.command("connect")
|
||||
.description("Connect a local seat to a hub participant, using that participant's token")
|
||||
.description("Connect a local seat to a hub participant: by name through the enrolled account, or with a hub URL and token file")
|
||||
.argument("<seat>", "The local seat to connect (its address, member@rig)")
|
||||
.requiredOption("--hub <url>", "The hub's MCP URL")
|
||||
.requiredOption("--as <participant>", "The participant's address in the hub (member@rig)")
|
||||
.requiredOption("--token-file <path>", "A file holding the participant's token")
|
||||
.requiredOption("--as <participant>", "Through the account: the name to connect as. With --hub: the participant's address in the hub (member@rig)")
|
||||
.option("--hub <url>", "The hub's MCP URL, to connect without the account (with --token-file)")
|
||||
.option("--token-file <path>", "A file holding the participant's token (with --hub)")
|
||||
.option("--allow-http", "Accept a plain-http hub for this binding: its token and work cross the network unencrypted")
|
||||
.option("--json", "Print JSON")
|
||||
.action(async (seat: string, opts: { hub: string; as: string; tokenFile: string; allowHttp?: boolean; json?: boolean }) => {
|
||||
let token: string;
|
||||
try { token = (getDeps().readTokenFile ?? defaultReadTokenFile)(opts.tokenFile); } catch (error) {
|
||||
console.error(`Can't read the token file: ${(error as Error).message}`);
|
||||
.action(async (seat: string, opts: { hub?: string; as: string; tokenFile?: string; allowHttp?: boolean; json?: boolean }) => {
|
||||
if (!opts.hub !== !opts.tokenFile) {
|
||||
console.error("Give both --hub and --token-file, or neither to connect through the enrolled account.");
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
let body: Record<string, unknown> = { session: seat, participantName: opts.as };
|
||||
if (opts.hub && opts.tokenFile) {
|
||||
let token: string;
|
||||
try { token = (getDeps().readTokenFile ?? defaultReadTokenFile)(opts.tokenFile); } catch (error) {
|
||||
console.error(`Can't read the token file: ${(error as Error).message}`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
body = { session: seat, hubUrl: opts.hub, hubSeat: opts.as, token, allowHttp: opts.allowHttp === true };
|
||||
}
|
||||
const daemon = await client();
|
||||
if (!daemon) return;
|
||||
const res = await daemon.post<Record<string, unknown>>("/api/network/bindings", { session: seat, hubUrl: opts.hub, hubSeat: opts.as, token, allowHttp: opts.allowHttp === true }, { headers: operatorAuthHeaders() });
|
||||
finish(res.status, res.data, opts.json, (b) => `Connected ${line(b as unknown as Binding)}\nWork for ${opts.as} reaches ${seat} once the connector runs; rig network status shows its last contact.`);
|
||||
const res = await daemon.post<Record<string, unknown>>("/api/network/bindings", body, { headers: operatorAuthHeaders(), timeoutMs: ACCOUNT_WRITE_TIMEOUT_MS });
|
||||
finish(res.status, res.data, opts.json, (b) => `Connected ${line(b as unknown as Binding)}\nWork for ${(b as unknown as Binding).hubSeat} reaches ${seat} once the connector runs; rig network status shows its last contact.`);
|
||||
});
|
||||
|
||||
cmd.command("status")
|
||||
@@ -83,10 +135,11 @@ export function networkCommand(depsOverride?: NetworkDeps): Command {
|
||||
.action(async (opts: { json?: boolean }) => {
|
||||
const daemon = await client();
|
||||
if (!daemon) return;
|
||||
const res = await daemon.get<{ bindings: Binding[] }>("/api/network/bindings");
|
||||
const res = await daemon.get<{ bindings: Binding[]; installations?: Installation[] }>("/api/network/bindings");
|
||||
finish(res.status, res.data, opts.json, (d) => {
|
||||
const bindings = (d.bindings ?? []) as Binding[];
|
||||
return bindings.length ? bindings.map(line).join("\n") : "No seats are connected to a hub.";
|
||||
const installations = (d.installations ?? []) as Installation[];
|
||||
return [...installations.map(enrolled), ...(bindings.length ? bindings.map(line) : ["No seats are connected to a hub."])].join("\n");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -98,8 +151,14 @@ export function networkCommand(depsOverride?: NetworkDeps): Command {
|
||||
.action(async (ref: string, opts: { reason?: string; json?: boolean }) => {
|
||||
const daemon = await client();
|
||||
if (!daemon) return;
|
||||
const res = await daemon.post<Record<string, unknown>>(`/api/network/bindings/${encodeURIComponent(ref)}/disconnect`, { reason: opts.reason }, { headers: operatorAuthHeaders() });
|
||||
finish(res.status, res.data, opts.json, (b) => `Disconnected ${line(b as unknown as Binding)}\nRemove the participant on the hub too, so the account stops listing it.`);
|
||||
const res = await daemon.post<Record<string, unknown>>(`/api/network/bindings/${encodeURIComponent(ref)}/disconnect`, { reason: opts.reason },
|
||||
{ headers: operatorAuthHeaders(), timeoutMs: ACCOUNT_WRITE_TIMEOUT_MS });
|
||||
finish(res.status, res.data, opts.json, (raw) => {
|
||||
const b = raw as unknown as Binding;
|
||||
if (b.accountRemoval === "confirmed") return `Disconnected ${line(b)}\nThe account stopped listing ${b.participantName}, which revokes it there.`;
|
||||
if (b.accountRemoval) return `Disconnected ${line(b)}`;
|
||||
return `Disconnected ${line(b)}\nRemove the participant on the hub too, so the account stops listing it.`;
|
||||
});
|
||||
});
|
||||
|
||||
return cmd;
|
||||
|
||||
@@ -17,10 +17,22 @@ let db: Database.Database;
|
||||
let home: string;
|
||||
let app: Hono;
|
||||
let rigs: RigRepository;
|
||||
/** Each write the CLI sent, with how long it would wait for the daemon. */
|
||||
const posts: Array<{ url: string; timeoutMs?: number }> = [];
|
||||
/** An account that answers the connector routes: a key for the one-time code, a bearer for each newly listed name. */
|
||||
const listed = new Set<string>();
|
||||
const accountFetch = async (url: string, init: { method: string; body: string }) => {
|
||||
const path = new URL(url).pathname;
|
||||
const body = JSON.parse(init.body) as { code?: string; participants?: string[] };
|
||||
const reply = (status: number, json: unknown) => ({ status, json: async () => json });
|
||||
if (path === "/api/connector/enroll") return body.code === "ec1.page" ? reply(200, { installation: { id: "inst-1", name: "lab" }, key: "oik1_c_inst-1_key" }) : reply(400, { error: "code_not_current" });
|
||||
const participants = (body.participants ?? []).map((name) => (listed.has(name) ? { name, state: "connected" } : (listed.add(name), { name, state: "connected", bearer: `bear-${name}` })));
|
||||
return reply(200, { participants, mcp: "https://account.example/mcp", rig: "acct" });
|
||||
};
|
||||
const routesWith = (bearerToken: string | null) => {
|
||||
const next = new Hono();
|
||||
next.use("*", async (c, n) => { c.set("rigRepo" as never, rigs as never); await n(); });
|
||||
next.route("/api/network", networkRoutes({ home, bearerToken }));
|
||||
next.route("/api/network", networkRoutes({ home, bearerToken, accountFetch, selfHostId: () => "host-a" }));
|
||||
return next;
|
||||
};
|
||||
beforeEach(() => {
|
||||
@@ -31,6 +43,7 @@ beforeEach(() => {
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES ('s1', ?, 'dev-impl@lab', 'running')").run(node.id);
|
||||
home = mkdtempSync(join(tmpdir(), "openrig-network-cli-"));
|
||||
app = routesWith(null);
|
||||
posts.length = 0;
|
||||
});
|
||||
afterEach(() => { db.close(); rmSync(home, { recursive: true, force: true }); vi.restoreAllMocks(); vi.unstubAllEnvs(); process.exitCode = undefined; });
|
||||
|
||||
@@ -46,7 +59,10 @@ function cli(tokenFile: Record<string, string> = {}) {
|
||||
};
|
||||
const deps: NetworkDeps = {
|
||||
lifecycleDeps,
|
||||
clientFactory: () => ({ get: (url: string) => call("GET", url), post: (url: string, body: unknown, options?: { headers?: Record<string, string> }) => call("POST", url, body, options?.headers) }) as never,
|
||||
clientFactory: () => ({ get: (url: string) => call("GET", url), post: (url: string, body: unknown, options?: { headers?: Record<string, string>; timeoutMs?: number }) => {
|
||||
posts.push({ url, timeoutMs: options?.timeoutMs });
|
||||
return call("POST", url, body, options?.headers);
|
||||
} }) as never,
|
||||
readTokenFile: (path) => { if (!(path in tokenFile)) throw new Error("no such file"); return tokenFile[path]!; },
|
||||
};
|
||||
return new Command().exitOverride().addCommand(networkCommand(deps));
|
||||
@@ -63,7 +79,7 @@ it("connects a seat with its token file, shows it in status without the token, a
|
||||
expect(status).not.toContain("tok-abc");
|
||||
|
||||
await cli().parseAsync(["node", "rig", "network", "disconnect", "dev-impl@lab", "--reason", "done"]);
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Disconnected netb-[0-9a-f]{20} dev-impl@lab -> alex@team .*disconnected \(done\)/);
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Disconnected netb-[0-9a-f]{20} dev-impl@lab -> alex@team .*disconnected \(done\)\nRemove the participant on the hub too/);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
});
|
||||
|
||||
@@ -111,6 +127,39 @@ it("status shows a hub item closed elsewhere under its binding, in text and JSON
|
||||
expect(JSON.parse(String(out.mock.calls.at(-1)?.[0])).bindings[0].closedElsewhere).toEqual([{ hubItem: "hub-9", localItem: "qitem-net-0001", note: "closed elsewhere (done, canceled)" }]);
|
||||
});
|
||||
|
||||
it("enrolls with the account page's code, connects a seat by name through it, and shows both in status", async () => {
|
||||
listed.clear();
|
||||
const out = vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
const err = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
await cli().parseAsync(["node", "rig", "network", "enroll", "ec1.page", "--site", "https://account.example"]);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Enrolled at https:\/\/account\.example\/ as installation lab \(neti-[0-9a-f]{20}\)\nConnect a seat with rig network connect <seat> --as <name>\.$/);
|
||||
|
||||
await cli().parseAsync(["node", "rig", "network", "connect", "dev-impl@lab", "--hub", "https://hub.example/mcp", "--as", "alex@team"]);
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(String(err.mock.calls.at(-1)?.[0])).toBe("Give both --hub and --token-file, or neither to connect through the enrolled account.");
|
||||
process.exitCode = undefined;
|
||||
|
||||
await cli().parseAsync(["node", "rig", "network", "connect", "dev-impl@lab", "--as", "alpha"]);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Connected netb-[0-9a-f]{20} dev-impl@lab -> alpha@acct at https:\/\/account\.example\/mcp no contact yet\nWork for alpha@acct reaches dev-impl@lab/);
|
||||
await cli().parseAsync(["node", "rig", "network", "status"]);
|
||||
const status = String(out.mock.calls.at(-1)?.[0]).split("\n");
|
||||
expect(status[0]).toMatch(/^Enrolled at https:\/\/account\.example\/ as installation lab \(neti-/);
|
||||
expect(status[1]).toMatch(/^netb-[0-9a-f]{20} dev-impl@lab -> alpha@acct at https:\/\/account\.example\/mcp/);
|
||||
await cli().parseAsync(["node", "rig", "network", "status", "--json"]);
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).not.toMatch(/oik1_|bear-alpha/);
|
||||
|
||||
await cli().parseAsync(["node", "rig", "network", "disconnect", "dev-impl@lab"]);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
const disconnected = String(out.mock.calls.at(-1)?.[0]);
|
||||
expect(disconnected).toMatch(/\nThe account stopped listing alpha, which revokes it there\.$/);
|
||||
expect(disconnected).not.toContain("Remove the participant on the hub");
|
||||
// Every network write waits past the daemon's 30 s account deadline, so the account's own answer is what's shown.
|
||||
expect(posts.map((p) => p.url)).toEqual(["/api/network/installations", "/api/network/bindings", expect.stringMatching(/^\/api\/network\/bindings\/.+\/disconnect$/)]);
|
||||
for (const p of posts) expect(p.timeoutMs).toBeGreaterThanOrEqual(45_000);
|
||||
});
|
||||
|
||||
it("sends the operator bearer on its writes, so a daemon with one configured takes them", async () => {
|
||||
app = routesWith("op-secret");
|
||||
const out = vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
|
||||
@@ -157,3 +157,46 @@ it("takes the participant's item over the wire, carries pickup and the result ba
|
||||
expect(await pass()).toMatchObject({ error: "revoked" });
|
||||
expect(bindings.get(binding.id)!.state).toBe("revoked");
|
||||
});
|
||||
|
||||
it("a lead's aggregate over its own local child work comes back once on the original hub request, and an open parent publishes nothing early", async () => {
|
||||
const token = await addGrant(tokenFile, { name: "alex", seat: "alex@team", rigs: ["team"], roots: [], workspace: false, owner: false });
|
||||
const task = await hubRepo.create({ sourceSession: "lead@team", destinationSession: "alex@team", summary: "Ship the fix", body: "Fix the bug and have it reviewed.", nudge: false } as never);
|
||||
const localRepo = queue(localDb);
|
||||
const bindings = new NetworkBindingStore(localDb, localHome);
|
||||
const binding = bindings.create({ session: "dev-impl@lab", hubUrl, hubSeat: "alex@team", token, allowHttp: true });
|
||||
const pass = () => runNetworkBindingPass({ db: localDb, queueRepo: localRepo, bindings, home: localHome }, bindings.get(binding.id)!);
|
||||
const hubItem = () => hubRepo.getById(task.qitemId)!;
|
||||
|
||||
expect(await pass()).toMatchObject({ taken: 1 });
|
||||
const [parent] = localRepo.list({ limit: 50 });
|
||||
localRepo.claim({ qitemId: parent!.qitemId, destinationSession: "dev-impl@lab" });
|
||||
|
||||
// The lead delegates through ordinary local rows and waits on each in turn.
|
||||
const build = await localRepo.create({ sourceSession: "dev-impl@lab", destinationSession: "builder@lab", summary: "Build the fix", body: "Fix the bug.", nudge: false } as never);
|
||||
const review = await localRepo.create({ sourceSession: "dev-impl@lab", destinationSession: "reviewer@lab", summary: "Review the fix", body: "Review the build.", nudge: false } as never);
|
||||
await localRepo.update({ qitemId: parent!.qitemId, actorSession: "dev-impl@lab", state: "blocked", blockedOn: build.qitemId, transitionNote: "Waiting on the build." });
|
||||
expect(await pass()).toMatchObject({ pickupsSent: 1, resultsSent: 0 });
|
||||
expect(hubItem()).toMatchObject({ state: "in-progress", resolution: null });
|
||||
|
||||
localRepo.claim({ qitemId: build.qitemId, destinationSession: "builder@lab" });
|
||||
await localRepo.update({ qitemId: build.qitemId, actorSession: "builder@lab", state: "done", closureReason: "no-follow-on", resolution: "Built: the fix is in commit abc123." });
|
||||
await localRepo.update({ qitemId: parent!.qitemId, actorSession: "dev-impl@lab", state: "blocked", blockedOn: review.qitemId, transitionNote: "Waiting on the review." });
|
||||
// A finished child isn't a result: the parent is still open, so nothing final goes to the hub.
|
||||
expect(await pass()).toMatchObject({ resultsSent: 0 });
|
||||
expect(hubItem()).toMatchObject({ state: "in-progress", resolution: null });
|
||||
|
||||
localRepo.claim({ qitemId: review.qitemId, destinationSession: "reviewer@lab" });
|
||||
await localRepo.update({ qitemId: review.qitemId, actorSession: "reviewer@lab", state: "done", closureReason: "no-follow-on", resolution: "Reviewed: approve." });
|
||||
const children = [localRepo.getById(build.qitemId)!, localRepo.getById(review.qitemId)!];
|
||||
const aggregate = children.map((child) => child.resolution).join(" ");
|
||||
await localRepo.update({ qitemId: parent!.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: aggregate });
|
||||
expect(await pass()).toMatchObject({ resultsSent: 1 });
|
||||
expect(hubItem()).toMatchObject({ state: "done", closureReason: "no-follow-on", resolution: "Built: the fix is in commit abc123. Reviewed: approve." });
|
||||
|
||||
// Once: a further pass writes nothing, and the children's own words never reached the hub as notes.
|
||||
const transitions = hubRepo.listTransitions(task.qitemId);
|
||||
await pass();
|
||||
expect(hubRepo.listTransitions(task.qitemId)).toHaveLength(transitions.length);
|
||||
expect(transitions.filter((t) => t.state === "done")).toHaveLength(1);
|
||||
expect(transitions.map((t) => t.transitionNote ?? "").join("\n")).not.toMatch(/Waiting on|Built:.*\n.*Reviewed:/);
|
||||
});
|
||||
@@ -9,6 +9,8 @@ import { queueInboxCheckinsSchema } from "./migrations/158_queue_inbox_checkins.
|
||||
import { queueItemsHandedOffFromIndexSchema } from "./migrations/159_queue_items_handed_off_from_index.js";
|
||||
import { networkBindingsSchema } from "./migrations/160_network_bindings.js";
|
||||
import { networkReceiptsSchema } from "./migrations/161_network_receipts.js";
|
||||
import { sessionAddressReadIndexesSchema } from "./migrations/162_session_address_read_indexes.js";
|
||||
import { networkInstallationsSchema } from "./migrations/163_network_installations.js";
|
||||
import { activityGrantsSchema } from "./migrations/143_activity_grants.js";
|
||||
import { environmentBindingsSchema } from "./migrations/144_environment_bindings.js";
|
||||
import { nodePermissionsToSchema } from "./migrations/145_node_permissions_to.js";
|
||||
@@ -325,4 +327,6 @@ export const ALL_MIGRATIONS: Migration[] = [
|
||||
queueItemsHandedOffFromIndexSchema,
|
||||
networkBindingsSchema,
|
||||
networkReceiptsSchema,
|
||||
sessionAddressReadIndexesSchema,
|
||||
networkInstallationsSchema,
|
||||
];
|
||||
@@ -0,0 +1,13 @@
|
||||
import type { Migration } from "../migrate.js";
|
||||
|
||||
/** Address reads seek a current binding; inventory recency is by id, not wall clock.
|
||||
* Keep the older created_at index for readers that explicitly use that order. */
|
||||
export const sessionAddressReadIndexesSchema: Migration = {
|
||||
name: "162_session_address_read_indexes.sql",
|
||||
sql: `
|
||||
CREATE INDEX IF NOT EXISTS idx_bindings_tmux_session ON bindings(tmux_session)
|
||||
WHERE tmux_session IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_node_latest_id ON sessions(node_id, id DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_sessions_name_latest_id ON sessions(session_name, id DESC);
|
||||
`,
|
||||
};
|
||||
@@ -0,0 +1,24 @@
|
||||
import type { Migration } from "../migrate.js";
|
||||
|
||||
/** This daemon enrolled in an account: the account's site, its installation id, and whether the person chose plain http
|
||||
* for it. The installation key lives in a private file under the OpenRig home, never in this table. A binding made
|
||||
* through the account names its installation, its participant and the account's installation id the name was listed under,
|
||||
* and records when the account stopped listing it. */
|
||||
export const networkInstallationsSchema: Migration = {
|
||||
name: "163_network_installations.sql",
|
||||
sql: `
|
||||
CREATE TABLE IF NOT EXISTS network_installations (
|
||||
id TEXT PRIMARY KEY,
|
||||
site TEXT NOT NULL,
|
||||
installation_id TEXT NOT NULL,
|
||||
name TEXT,
|
||||
allow_http INTEGER NOT NULL DEFAULT 0 CHECK (allow_http IN (0, 1)),
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_network_installations_site ON network_installations(site);
|
||||
ALTER TABLE network_bindings ADD COLUMN installation_id TEXT;
|
||||
ALTER TABLE network_bindings ADD COLUMN participant_name TEXT;
|
||||
ALTER TABLE network_bindings ADD COLUMN account_removed_at TEXT;
|
||||
ALTER TABLE network_bindings ADD COLUMN account_installation_id TEXT;
|
||||
`,
|
||||
};
|
||||
@@ -0,0 +1,92 @@
|
||||
// The shared words for what happened to a piece of work and what a member can do. Today each part of OpenRig names
|
||||
// these for itself (the transport, the outbox, the queue's wake record, the hosted ring, the watchdog), and the names
|
||||
// drift: a result one reader doesn't recognise reads as fine to it and as a failure to another. This module is where
|
||||
// the words live, so a consumer asks it instead of matching raw strings.
|
||||
//
|
||||
// It translates what is already stored; it never renames it. Activity stays in activity-taxonomy.ts, which this module
|
||||
// doesn't redefine. An unrecognised word always reads `unknown`, never success and never failure.
|
||||
|
||||
/** The two contracts' identifiers, in the daemon's existing `<name>.v<N>` form (`operation-correlation.v1`). N changes
|
||||
* only when a word's meaning changes incompatibly; new jobs and capabilities are advertised, not versioned. */
|
||||
export const TEAM_OPERATIONS_CONTRACT = "team-operations.v1";
|
||||
export const MEMBER_CONNECTION_CONTRACT = "member-connection.v1";
|
||||
|
||||
/** Whether a member or a daemon supports something. A client reads an absent answer as `unknown`, never as
|
||||
* `not-supported`. */
|
||||
export type Support = "supported" | "not-supported" | "unknown";
|
||||
|
||||
/** Who established a fact: OpenRig observing it directly, OpenRig's own adapter reporting what it read from the
|
||||
* program, or an outside adapter. A member's own claim never makes a fact observed. */
|
||||
export type Reporter = "openrig-observed" | "openrig-adapter" | "outside-adapter";
|
||||
|
||||
/** What a delivery route says happened to one attempt. None of them means the member read the work or did it: pickup
|
||||
* is a claim on the row, and nothing else. */
|
||||
export type DeliveryFact =
|
||||
/** The route took it: text typed and submitted, a trigger answered 2xx. */
|
||||
| "accepted"
|
||||
/** Queued inside the program; no start seen yet. */
|
||||
| "queued-not-started"
|
||||
/** A turn start correlated to this attempt. */
|
||||
| "turn-started"
|
||||
/** Proven that nothing reached the member: no input was written, or the route refused before sending. */
|
||||
| "not-delivered"
|
||||
/** The route reported a failure after the attempt may already have reached the member (an error answer, a dropped
|
||||
* connection): failed, with its effect unknown. */
|
||||
| "failed-effect-unknown"
|
||||
/** Held on purpose by a guard or a setting; nothing was written. */
|
||||
| "held"
|
||||
| "unknown";
|
||||
|
||||
export interface ReportedFact<F extends string> {
|
||||
fact: F;
|
||||
reporter: Reporter;
|
||||
}
|
||||
|
||||
/** The route a wake took. `none` means no route can carry it (a pull-only member, an address nothing holds). */
|
||||
export type WakeRoute = "terminal" | "gateway" | "hosted-trigger" | "none";
|
||||
|
||||
/** One wake attempt's outcome, read from the queue's recorded nudge result. */
|
||||
export interface WakeOutcome extends ReportedFact<DeliveryFact> {
|
||||
/** Null when the recorded text names no route this reader knows. */
|
||||
route: WakeRoute | null;
|
||||
/** A terminal wake whose submission was seen rendered, not only sent. */
|
||||
renderVerified?: true;
|
||||
/** Which guard held it: the seat's hold or interruption level (retrying is held again), or the draft-aware check
|
||||
* (a draft or unreadable input, which can clear on its own). */
|
||||
heldBy?: "typing-guard" | "draft-aware";
|
||||
/** The recorded text, unchanged. */
|
||||
recorded: string;
|
||||
}
|
||||
|
||||
/** Read a queue row's recorded nudge result (`last_nudge_result`) as a delivery fact. Every form a producer writes
|
||||
* today is listed; anything else reads `unknown` with no route, so a consumer can tell "the wake's outcome can't be
|
||||
* read" from every known outcome. Null when no attempt was recorded. */
|
||||
export function readWakeOutcome(recorded: string | null | undefined): WakeOutcome | null {
|
||||
if (recorded === null || recorded === undefined) return null;
|
||||
const at = (fact: DeliveryFact, route: WakeRoute | null, extra: Partial<WakeOutcome> = {}): WakeOutcome =>
|
||||
({ fact, reporter: "openrig-observed", route, recorded, ...extra });
|
||||
if (recorded === "verified") return at("accepted", "terminal", { renderVerified: true });
|
||||
// Sent and submitted without render verification: the route took it, nothing more.
|
||||
if (recorded === "delivered-ack-pending") return at("accepted", "terminal");
|
||||
if (recorded === "retained:typing_guard") return at("held", "terminal", { heldBy: "typing-guard" });
|
||||
if (recorded.startsWith("retained:")) return at("held", "terminal", { heldBy: "draft-aware" });
|
||||
// A hosted member's trigger answered: the ring was taken, not the work.
|
||||
if (recorded.startsWith("wake-accepted:webhook")) return at("accepted", "hosted-trigger");
|
||||
if (recorded.startsWith("indeterminate:webhook")) return at("unknown", "hosted-trigger");
|
||||
// A ring that never left: no URL, no Authorization value, an invalid URL.
|
||||
if (/^failed:webhook \([^)]*\) (isn't set:|not rung:)/.test(recorded)) return at("not-delivered", "hosted-trigger");
|
||||
// Any other webhook failure came after the POST was sent. The trigger's own error answer is its report; a network
|
||||
// error is what OpenRig saw. Either way the member may have the ring.
|
||||
if (/^failed:webhook \([^)]*\) answered /.test(recorded)) {
|
||||
return at("failed-effect-unknown", "hosted-trigger", { reporter: "outside-adapter" });
|
||||
}
|
||||
if (recorded.startsWith("failed:webhook")) return at("failed-effect-unknown", "hosted-trigger");
|
||||
// The gateway's own ledger records whether the person was posted to; this attempt's own outcome is unknown.
|
||||
if (recorded.startsWith("gateway-owned:")) return at("unknown", "gateway");
|
||||
if (recorded.startsWith("unroutable:")) return at("not-delivered", "none");
|
||||
// `indeterminate:` covers input that may have been written (a paste, an unconfirmed Enter) and timeouts.
|
||||
if (recorded.startsWith("indeterminate:")) return at("unknown", "terminal");
|
||||
// `failed:` is written only when no input could have reached the seat.
|
||||
if (recorded.startsWith("failed:")) return at("not-delivered", "terminal");
|
||||
return at("unknown", null);
|
||||
}
|
||||
@@ -28,10 +28,18 @@ export interface NetworkBinding {
|
||||
updatedAt: string;
|
||||
lastContactAt: string | null;
|
||||
lastError: string | null;
|
||||
/** Set when the binding was made through an enrolled account: its installation (`neti-…`) and participant name. */
|
||||
installationId: string | null;
|
||||
participantName: string | null;
|
||||
/** The account's own id for the installation the name was listed under. A replaced enrollment keeps the local id and
|
||||
* moves to a new account installation, so only a list from this one can confirm the name was dropped. */
|
||||
accountInstallationId: string | null;
|
||||
/** When the account stopped listing this participant, after the binding ended here. */
|
||||
accountRemovedAt: string | null;
|
||||
}
|
||||
|
||||
export class NetworkBindingError extends Error {
|
||||
constructor(readonly code: string, message: string, readonly status: 400 | 404 | 409 | 500 = 400) {
|
||||
constructor(readonly code: string, message: string, readonly status: 400 | 401 | 404 | 409 | 500 | 502 | 503 = 400) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
@@ -39,12 +47,15 @@ export class NetworkBindingError extends Error {
|
||||
interface Row {
|
||||
id: string; node_id: string; local_session: string; hub_url: string; hub_seat: string; allow_http: number; state: NetworkBindingState;
|
||||
state_reason: string | null; created_at: string; updated_at: string; last_contact_at: string | null; last_error: string | null;
|
||||
installation_id: string | null; participant_name: string | null; account_removed_at: string | null; account_installation_id: string | null;
|
||||
}
|
||||
|
||||
const project = (row: Row): NetworkBinding => ({
|
||||
id: row.id, nodeId: row.node_id, localSession: row.local_session, hubUrl: row.hub_url, hubSeat: row.hub_seat,
|
||||
allowHttp: row.allow_http === 1, unencrypted: row.hub_url.startsWith("http:"), state: row.state, stateReason: row.state_reason, createdAt: row.created_at, updatedAt: row.updated_at,
|
||||
lastContactAt: row.last_contact_at, lastError: row.last_error,
|
||||
installationId: row.installation_id, participantName: row.participant_name, accountRemovedAt: row.account_removed_at,
|
||||
accountInstallationId: row.account_installation_id,
|
||||
});
|
||||
|
||||
const MAX_CREDENTIAL_BYTES = 16384;
|
||||
@@ -58,10 +69,15 @@ export function networkCredentialFile(home: string, bindingId: string): string {
|
||||
/** The participant's bearer, read from its private file. A file anyone else can read, a link or an oversized file is
|
||||
* refused, with the path named and the value never shown. */
|
||||
export function readNetworkCredential(home: string, bindingId: string): string {
|
||||
const file = networkCredentialFile(home, bindingId);
|
||||
return readPrivateToken(networkCredentialFile(home, bindingId),
|
||||
(file) => `The credential for ${bindingId} isn't at ${file}. Connect the seat again with its token file.`);
|
||||
}
|
||||
|
||||
/** A token from one of the network's private files (`{"version":1,"token":…}`), checked the same way for every kind. */
|
||||
export function readPrivateToken(file: string, missing: (file: string) => string): string {
|
||||
let stat;
|
||||
try { stat = lstatSync(file); } catch {
|
||||
throw new NetworkBindingError("credential_missing", `The credential for ${bindingId} isn't at ${file}. Connect the seat again with its token file.`, 404);
|
||||
throw new NetworkBindingError("credential_missing", missing(file), 404);
|
||||
}
|
||||
if (!stat.isFile() || stat.isSymbolicLink() || stat.size > MAX_CREDENTIAL_BYTES || (stat.mode & 0o077) !== 0) {
|
||||
throw new NetworkBindingError("credential_unsafe", `${file} must be a regular file readable only by this user.`);
|
||||
@@ -77,7 +93,29 @@ export function readNetworkCredential(home: string, bindingId: string): string {
|
||||
return value.token;
|
||||
}
|
||||
|
||||
function validHubUrl(raw: unknown, allowHttp: boolean): string {
|
||||
/** Write a token to its own private file: a 0700 folder and a 0600 file, opened exclusively. On failure, only what
|
||||
* this call made is removed: the file it opened, and the folder if it made it (`mkdirSync` returns the first folder it
|
||||
* made, so any value means this one is new). Returns the cleanup for a later step that fails. */
|
||||
export function writePrivateToken(file: string, token: string): () => void {
|
||||
let created = false;
|
||||
let madeFolder: string | undefined;
|
||||
const cleanup = () => {
|
||||
if (created) rmSync(file, { force: true });
|
||||
if (madeFolder !== undefined) rmSync(dirname(file), { recursive: true, force: true });
|
||||
};
|
||||
try {
|
||||
madeFolder = mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
|
||||
const fd = openSync(file, "wx", 0o600);
|
||||
created = true;
|
||||
try { writeFileSync(fd, `${JSON.stringify({ version: 1, token })}\n`); } finally { closeSync(fd); }
|
||||
} catch (error) {
|
||||
cleanup();
|
||||
throw error;
|
||||
}
|
||||
return cleanup;
|
||||
}
|
||||
|
||||
export function validHubUrl(raw: unknown, allowHttp: boolean): string {
|
||||
if (typeof raw !== "string" || !raw.trim()) throw new NetworkBindingError("invalid_hub_url", "A hub URL is required.");
|
||||
let url: URL;
|
||||
try { url = new URL(raw.trim()); } catch { throw new NetworkBindingError("invalid_hub_url", `'${raw}' isn't a URL.`); }
|
||||
@@ -140,37 +178,39 @@ export class NetworkBindingStore {
|
||||
/** Bind a local seat to a hub participant with that participant's token. The credential is written first, exclusively
|
||||
* and private, then the row. If either step fails, the credential this call created is removed, so a refused bind
|
||||
* leaves none behind; a file that was already at that path is never touched. */
|
||||
create(input: { session: unknown; hubUrl: unknown; hubSeat: unknown; token: unknown; allowHttp?: unknown }): NetworkBinding {
|
||||
const session = typeof input.session === "string" ? input.session.trim() : "";
|
||||
/** The seat to bind, by its node: it must exist and have no active binding. Checked before anything is sent anywhere. */
|
||||
freeSeat(rawSession: unknown): { session: string; nodeId: string } {
|
||||
const session = typeof rawSession === "string" ? rawSession.trim() : "";
|
||||
if (!session) throw new NetworkBindingError("invalid_seat", "Name the local seat to connect.");
|
||||
const nodeId = resolveSessionNodeId(this.db, session);
|
||||
if (!nodeId) throw new NetworkBindingError("seat_not_found", `No local seat is named '${session}'.`, 404);
|
||||
const bound = this.db.prepare("SELECT id FROM network_bindings WHERE node_id = ? AND state = 'active'").get(nodeId) as { id: string } | undefined;
|
||||
if (bound) throw new NetworkBindingError("seat_already_connected", `${session} is already connected (${bound.id}). Disconnect it first.`, 409);
|
||||
return { session, nodeId };
|
||||
}
|
||||
|
||||
create(input: { session: unknown; hubUrl: unknown; hubSeat: unknown; token: unknown; allowHttp?: unknown;
|
||||
installation?: { id: string; name: string; account: string } }): NetworkBinding {
|
||||
const { session, nodeId } = this.freeSeat(input.session);
|
||||
const allowHttp = input.allowHttp === true;
|
||||
const hubUrl = validHubUrl(input.hubUrl, allowHttp);
|
||||
const hubSeat = validHubSeat(input.hubSeat);
|
||||
const token = validToken(input.token);
|
||||
const bound = this.db.prepare("SELECT id FROM network_bindings WHERE node_id = ? AND state = 'active'").get(nodeId) as { id: string } | undefined;
|
||||
if (bound) throw new NetworkBindingError("seat_already_connected", `${session} is already connected (${bound.id}). Disconnect it first.`, 409);
|
||||
const taken = this.db.prepare("SELECT id FROM network_bindings WHERE hub_url = ? AND hub_seat = ? AND state = 'active'").get(hubUrl, hubSeat) as { id: string } | undefined;
|
||||
if (taken) throw new NetworkBindingError("participant_already_connected", `${hubSeat} at ${hubUrl} is already connected (${taken.id}).`, 409);
|
||||
|
||||
const id = this.newId();
|
||||
const file = networkCredentialFile(this.home, id);
|
||||
let created = false;
|
||||
let madeFolder: string | undefined;
|
||||
// The credential first, then the row; a failed row takes the credential with it. A file that was already at the path
|
||||
// is never touched, and a folder that was already there keeps its other files.
|
||||
const cleanup = writePrivateToken(networkCredentialFile(this.home, id), token);
|
||||
try {
|
||||
madeFolder = mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
|
||||
const fd = openSync(file, "wx", 0o600);
|
||||
created = true;
|
||||
try { writeFileSync(fd, `${JSON.stringify({ version: 1, token })}\n`); } finally { closeSync(fd); }
|
||||
const at = this.now().toISOString();
|
||||
this.db.prepare(`INSERT INTO network_bindings (id, node_id, local_session, hub_url, hub_seat, allow_http, state, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active', ?, ?)`).run(id, nodeId, session, hubUrl, hubSeat, allowHttp ? 1 : 0, at, at);
|
||||
this.db.prepare(`INSERT INTO network_bindings (id, node_id, local_session, hub_url, hub_seat, allow_http, state, created_at, updated_at,
|
||||
installation_id, participant_name, account_installation_id) VALUES (?, ?, ?, ?, ?, ?, 'active', ?, ?, ?, ?, ?)`)
|
||||
.run(id, nodeId, session, hubUrl, hubSeat, allowHttp ? 1 : 0, at, at, input.installation?.id ?? null, input.installation?.name ?? null,
|
||||
input.installation?.account ?? null);
|
||||
} catch (error) {
|
||||
// Only what this call made: the file it opened, and this binding's folder if it made it (`mkdirSync` returns the
|
||||
// first folder it made, so any value means this one is new). A folder that was already there keeps its other files.
|
||||
if (created) rmSync(file, { force: true });
|
||||
if (madeFolder !== undefined) rmSync(dirname(file), { recursive: true, force: true });
|
||||
cleanup();
|
||||
throw error;
|
||||
}
|
||||
return this.get(id)!;
|
||||
@@ -187,6 +227,18 @@ export class NetworkBindingStore {
|
||||
return this.end(id, "revoked", reason);
|
||||
}
|
||||
|
||||
/** Ends these active bindings' state only, so a caller can do it in one transaction with its own change. Their
|
||||
* credentials stay until the caller ends each one again with `disconnect`, which removes them. */
|
||||
markDisconnected(ids: string[], reason: string): void {
|
||||
const end = this.db.prepare("UPDATE network_bindings SET state = 'disconnected', state_reason = ?, updated_at = ? WHERE id = ? AND state = 'active'");
|
||||
for (const id of ids) end.run(reason, this.now().toISOString(), id);
|
||||
}
|
||||
|
||||
/** The account stopped listing this ended binding's participant. */
|
||||
markAccountRemoved(id: string): void {
|
||||
this.db.prepare("UPDATE network_bindings SET account_removed_at = ?, updated_at = ? WHERE id = ?").run(this.now().toISOString(), this.now().toISOString(), id);
|
||||
}
|
||||
|
||||
/** The connector reached the hub as this binding. */
|
||||
recordContact(id: string, at: Date = this.now()): void {
|
||||
this.db.prepare("UPDATE network_bindings SET last_contact_at = ?, last_error = NULL, updated_at = ? WHERE id = ?")
|
||||
|
||||
@@ -0,0 +1,378 @@
|
||||
// This daemon enrolled in an account, so the person connects a seat by a name instead of a hub URL and a token file.
|
||||
// Enrolling trades a one-time code from the account page for this installation's key. Connecting a seat lists the
|
||||
// installation's participants with the account, which answers a bearer for a newly listed name, and the account's MCP
|
||||
// front and rig, where the binding then reaches. A name the installation stops listing is revoked there.
|
||||
//
|
||||
// The account replaces the whole list on every change, so changes to one installation's list, and a replaced enrollment,
|
||||
// run one at a time, each reading what it needs inside its turn. One enroll, connect or disconnect spends at most one
|
||||
// deadline with the account, waiting for its turn included.
|
||||
import type Database from "better-sqlite3";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { existsSync, renameSync, rmSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { NetworkBindingError, readPrivateToken, writePrivateToken, type NetworkBinding, type NetworkBindingStore } from "./network-bindings.js";
|
||||
|
||||
export interface NetworkInstallation {
|
||||
id: string;
|
||||
site: string;
|
||||
/** The account's id for this installation. */
|
||||
installationId: string;
|
||||
name: string | null;
|
||||
allowHttp: boolean;
|
||||
createdAt: string;
|
||||
}
|
||||
|
||||
/** An enroll's answer. A replaced enrollment names the bindings it ended, whose names the old installation may still list,
|
||||
* and any cleanup after the switch that didn't finish. */
|
||||
export interface EnrollResult extends NetworkInstallation {
|
||||
replaced?: { endedBindings: Array<{ id: string; localSession: string; participantName: string | null }>; problems?: string[] };
|
||||
}
|
||||
|
||||
interface Row { id: string; site: string; installation_id: string; name: string | null; allow_http: number; created_at: string }
|
||||
|
||||
const project = (row: Row): NetworkInstallation => ({
|
||||
id: row.id, site: row.site, installationId: row.installation_id, name: row.name, allowHttp: row.allow_http === 1, createdAt: row.created_at,
|
||||
});
|
||||
|
||||
/** Where the installation key lives: `<home>/network-secrets/<installation>/key.json`, private to this user. */
|
||||
export function networkInstallationKeyFile(home: string, id: string): string {
|
||||
if (!/^neti-[0-9a-f]{20}$/.test(id)) throw new NetworkBindingError("invalid_installation_id", `'${id}' isn't a network installation id.`);
|
||||
return join(home, "network-secrets", id, "key.json");
|
||||
}
|
||||
|
||||
const NAME = /^[a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])?$/;
|
||||
|
||||
/** The state reason of a binding a replaced enrollment ended: its name belongs to the old installation. */
|
||||
export const REPLACED_REASON = "replaced by a new enrollment";
|
||||
|
||||
/** What's known about the account's side of an ended binding made through it: confirmed dropped, unconfirmed (a removal
|
||||
* that failed or whose answer was lost), or left with an earlier account installation after the enrollment was replaced,
|
||||
* which this daemon's key can neither drop nor confirm. `accountInstallationNow` is the account's id for the binding's
|
||||
* enrollment as it is now. */
|
||||
export function accountRemoval(binding: NetworkBinding, accountInstallationNow: string | null | undefined): "confirmed" | "unconfirmed" | "old-installation" | null {
|
||||
if (!binding.installationId || binding.state === "active") return null;
|
||||
if (binding.accountRemovedAt) return "confirmed";
|
||||
// The hub refused its credential, so nothing live is left there.
|
||||
if (binding.state === "revoked") return null;
|
||||
if (binding.accountInstallationId !== accountInstallationNow) return "old-installation";
|
||||
return "unconfirmed";
|
||||
}
|
||||
|
||||
/** The command's time with the account ran out before a call was sent. */
|
||||
class NotSent extends NetworkBindingError {}
|
||||
|
||||
/** What the account answers for the installation's participant list. */
|
||||
interface Identities { participants: Array<{ name: string; state?: string; bearer?: string }>; mcp: string; rig: string }
|
||||
|
||||
export type AccountFetch = (url: string, init: { method: string; headers: Record<string, string>; body: string; redirect: "error"; signal: AbortSignal })
|
||||
=> Promise<{ status: number; json(): Promise<unknown> }>;
|
||||
|
||||
/** How long one enroll, connect or disconnect may spend with the account in all, waiting for its turn included. */
|
||||
export const ACCOUNT_DEADLINE_MS = 30_000;
|
||||
|
||||
/** One change at a time to an installation's participant list, across the stores each request makes. */
|
||||
const listTurns = new Map<string, Promise<void>>();
|
||||
async function inListTurn<T>(installationId: string, run: () => Promise<T>): Promise<T> {
|
||||
const before = listTurns.get(installationId) ?? Promise.resolve();
|
||||
let release!: () => void;
|
||||
const mine = new Promise<void>((resolve) => { release = resolve; });
|
||||
const chain = before.then(() => mine);
|
||||
listTurns.set(installationId, chain);
|
||||
try {
|
||||
await before;
|
||||
return await run();
|
||||
} finally {
|
||||
release();
|
||||
if (listTurns.get(installationId) === chain) listTurns.delete(installationId);
|
||||
}
|
||||
}
|
||||
|
||||
export class NetworkInstallationStore {
|
||||
private readonly fetch: AccountFetch;
|
||||
constructor(
|
||||
private readonly db: Database.Database,
|
||||
private readonly home: string,
|
||||
private readonly opts: { fetch?: AccountFetch; selfHostId: () => string | null; now?: () => Date; newId?: () => string; timeoutMs?: number },
|
||||
) {
|
||||
this.fetch = opts.fetch ?? ((url, init) => fetch(url, init));
|
||||
}
|
||||
|
||||
list(): NetworkInstallation[] {
|
||||
return (this.db.prepare("SELECT * FROM network_installations ORDER BY created_at, id").all() as Row[]).map(project);
|
||||
}
|
||||
|
||||
/** Trade the account page's one-time code for this installation's key. The code is checked by the account only, so a
|
||||
* refused enroll here (a URL, a second enroll at the same site without `replace`) never spends it. With `replace`, the
|
||||
* installation's turn comes first, so its wait counts against the same deadline: a wait that runs out sends no code,
|
||||
* and the new key takes the old one's place only after the account accepted the new code. */
|
||||
async enroll(bindings: NetworkBindingStore, input: { code: unknown; site: unknown; allowHttp?: unknown; replace?: unknown }): Promise<EnrollResult> {
|
||||
const code = typeof input.code === "string" ? input.code.trim() : "";
|
||||
if (!code || /\s/.test(code)) throw new NetworkBindingError("invalid_code", "Give the one-time code the account page shows.");
|
||||
const site = validSite(input.site, input.allowHttp === true);
|
||||
const existing = this.db.prepare("SELECT * FROM network_installations WHERE site = ?").get(site) as Row | undefined;
|
||||
if (existing && input.replace !== true) {
|
||||
throw new NetworkBindingError("already_enrolled", `This daemon is already enrolled at ${site} (${existing.id}). To enroll again with a new code, add --replace.`, 409);
|
||||
}
|
||||
const instance = this.instance();
|
||||
const until = this.deadline();
|
||||
const allowHttp = input.allowHttp === true ? 1 : 0;
|
||||
if (existing) {
|
||||
return inListTurn(existing.id, async () => {
|
||||
const next = await this.exchange(site, code, instance, until);
|
||||
return this.replace(bindings, this.get(existing.id), next.installation, next.key, allowHttp);
|
||||
});
|
||||
}
|
||||
|
||||
const { installation, key } = await this.exchange(site, code, instance, until);
|
||||
const id = this.opts.newId?.() ?? `neti-${randomBytes(10).toString("hex")}`;
|
||||
const cleanup = writePrivateToken(networkInstallationKeyFile(this.home, id), key);
|
||||
try {
|
||||
this.db.prepare("INSERT INTO network_installations (id, site, installation_id, name, allow_http, created_at) VALUES (?, ?, ?, ?, ?, ?)")
|
||||
.run(id, site, installation.id, installation.name, allowHttp, (this.opts.now?.() ?? new Date()).toISOString());
|
||||
} catch (error) {
|
||||
cleanup();
|
||||
throw error;
|
||||
}
|
||||
return this.get(id);
|
||||
}
|
||||
|
||||
/** The code, with this daemon's host id, for an account installation and its key. */
|
||||
private async exchange(site: string, code: string, instance: string, until: number): Promise<{ installation: { id: string; name: string | null }; key: string }> {
|
||||
let answer;
|
||||
try {
|
||||
answer = await this.call(site, "POST", "/api/connector/enroll", null, { code, instance }, until);
|
||||
} catch (error) {
|
||||
if (error instanceof NotSent) throw new NetworkBindingError(error.code, `${error.message} The code wasn't sent, so it's still good while it's current: try again.`, 503);
|
||||
if (error instanceof NetworkBindingError && error.code === "account_unavailable") {
|
||||
throw new NetworkBindingError("account_unavailable", `${error.message} Whether the account took the code is unknown: try again with the same code, and if it's refused, get a new one.`, 503);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
if (answer.status !== 200) {
|
||||
const error = errorCode(answer.body);
|
||||
if (answer.status === 400 || answer.status === 409) {
|
||||
throw new NetworkBindingError(error ?? "enroll_refused", `The account refused the code (${error ?? answer.status}). Get a new one from the account page; each is single-use and short-lived.`, answer.status);
|
||||
}
|
||||
throw new NetworkBindingError("account_unavailable", `The account couldn't answer (${answer.status}). If the code is still current, try again with it; otherwise get a new one.`, 503);
|
||||
}
|
||||
const body = (answer.body ?? {}) as { installation?: { id?: unknown; name?: unknown }; key?: unknown };
|
||||
if (typeof body.installation?.id !== "string" || typeof body.key !== "string" || !body.key || /\s/.test(body.key)) {
|
||||
throw new NetworkBindingError("account_unexpected", "The account's answer to the enroll had no installation or key.", 502);
|
||||
}
|
||||
return { installation: { id: body.installation.id, name: typeof body.installation.name === "string" ? body.installation.name : null }, key: body.key };
|
||||
}
|
||||
|
||||
/** The new key goes in beside the old one and takes its place, then the row and the old installation's active bindings
|
||||
* change in one transaction. A failure before that commits puts the old key back (with no old key, it removes the new
|
||||
* one), so nothing here changes, though the account has already made the new installation and nothing here holds its
|
||||
* key. After the commit the replace has happened: the ended bindings' names are returned, since the old installation
|
||||
* may still list them and this key can't drop them, with any cleanup that didn't finish. */
|
||||
private replace(bindings: NetworkBindingStore, old: NetworkInstallation, next: { id: string; name: string | null }, key: string, allowHttp: number): EnrollResult {
|
||||
const file = networkInstallationKeyFile(this.home, old.id);
|
||||
const staged = `${file}.next`;
|
||||
const kept = `${file}.prev`;
|
||||
const ending = bindings.active().filter((b) => b.installationId === old.id);
|
||||
rmSync(staged, { force: true });
|
||||
const cleanup = writePrivateToken(staged, key);
|
||||
const hadKey = existsSync(file);
|
||||
try {
|
||||
if (hadKey) renameSync(file, kept);
|
||||
renameSync(staged, file);
|
||||
this.db.transaction(() => {
|
||||
this.db.prepare("UPDATE network_installations SET installation_id = ?, name = ?, allow_http = ? WHERE id = ?").run(next.id, next.name, allowHttp, old.id);
|
||||
bindings.markDisconnected(ending.map((b) => b.id), REPLACED_REASON);
|
||||
})();
|
||||
} catch (error) {
|
||||
if (!hadKey) rmSync(file, { force: true });
|
||||
else if (existsSync(kept)) renameSync(kept, file);
|
||||
cleanup();
|
||||
throw new NetworkBindingError("replace_failed",
|
||||
`The account accepted the new code, but this daemon couldn't switch to it (${(error as Error).message.slice(0, 200)}), so nothing here changed. The account now has a new installation whose key nothing here holds: remove it on the account page, then enroll again with a new code.`, 500);
|
||||
}
|
||||
const problems: string[] = [];
|
||||
try {
|
||||
rmSync(kept, { force: true });
|
||||
} catch (error) {
|
||||
problems.push(`The earlier key's backup at ${kept} couldn't be removed (${(error as Error).message.slice(0, 200)}). It's no longer used; remove it by hand.`);
|
||||
}
|
||||
for (const binding of ending) {
|
||||
// Already ended in the transaction; this removes its credential, and a failure says how to retry.
|
||||
try { bindings.disconnect(binding.id, REPLACED_REASON); } catch (error) { problems.push((error as Error).message); }
|
||||
}
|
||||
return { ...this.get(old.id), replaced: { endedBindings: ending.map((b) => ({ id: b.id, localSession: b.localSession, participantName: b.participantName })),
|
||||
...(problems.length ? { problems } : {}) } };
|
||||
}
|
||||
|
||||
/** The one installation a name-only connect goes through. */
|
||||
only(): NetworkInstallation {
|
||||
const all = this.list();
|
||||
if (all.length === 0) throw new NetworkBindingError("not_enrolled", "This daemon isn't enrolled in an account. Run rig network enroll <code> --site <site> first, or connect with --hub and --token-file.", 404);
|
||||
if (all.length > 1) throw new NetworkBindingError("installation_ambiguous", "This daemon is enrolled at more than one site; connect with --hub and --token-file.", 409);
|
||||
return all[0]!;
|
||||
}
|
||||
|
||||
/** Connect a seat as a participant name: list it with the account, then bind to the account's MCP front with the new
|
||||
* bearer. The seat is checked first, so a refusal here sends nothing. A name the account keeps without a bearer had
|
||||
* its answer lost earlier: it's dropped and listed again for a fresh one. */
|
||||
async connect(bindings: NetworkBindingStore, input: { session: unknown; name: unknown }): Promise<NetworkBinding> {
|
||||
const installation = this.only();
|
||||
bindings.freeSeat(input.session);
|
||||
const name = typeof input.name === "string" ? input.name.trim() : "";
|
||||
if (!NAME.test(name)) throw new NetworkBindingError("invalid_name", `'${name}' isn't a participant name: lowercase letters, digits and inner dashes, up to 32.`);
|
||||
const until = this.deadline();
|
||||
return inListTurn(installation.id, async () => {
|
||||
// In this installation's turn: checked again, with the enrollment as it is now (a replace may have run while this
|
||||
// waited), and every list sent reads the names to keep afresh.
|
||||
const current = this.get(installation.id);
|
||||
bindings.freeSeat(input.session);
|
||||
if (this.listedNames(bindings, current).includes(name)) throw new NetworkBindingError("participant_already_connected", `${name} is already connected here.`, 409);
|
||||
const others = () => this.listedNames(bindings, current).filter((n) => n !== name);
|
||||
let answer = await this.setParticipants(bindings, current, () => [...others(), name], until);
|
||||
let bearer = answer.participants.find((p) => p.name === name)?.bearer;
|
||||
if (!bearer) {
|
||||
await this.setParticipants(bindings, current, others, until);
|
||||
answer = await this.setParticipants(bindings, current, () => [...others(), name], until);
|
||||
bearer = answer.participants.find((p) => p.name === name)?.bearer;
|
||||
if (!bearer) throw new NetworkBindingError("account_unexpected", `The account listed ${name} without a bearer, twice.`, 502);
|
||||
}
|
||||
try {
|
||||
return bindings.create({ session: input.session, hubUrl: answer.mcp, hubSeat: `${name}@${answer.rig}`, token: bearer,
|
||||
allowHttp: current.allowHttp, installation: { id: current.id, name, account: current.installationId } });
|
||||
} catch (error) {
|
||||
// Best effort: nothing here holds that bearer, so the account is asked to stop listing the name. If that fails,
|
||||
// the error says so, whatever failed here; the account may still list it.
|
||||
let unconfirmed = "";
|
||||
try {
|
||||
await this.setParticipants(bindings, current, others, until);
|
||||
} catch (drop) {
|
||||
unconfirmed = ` OpenRig couldn't confirm the account dropped ${name} again (${(drop as Error).message}); it may still list it.`;
|
||||
}
|
||||
if (!unconfirmed) throw error;
|
||||
if (error instanceof NetworkBindingError) throw new NetworkBindingError(error.code, `${error.message}${unconfirmed}`, error.status);
|
||||
throw new NetworkBindingError("binding_write_failed", `The binding couldn't be saved here (${(error as Error).message.slice(0, 200)}).${unconfirmed}`, 500);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/** After a binding made through the account ended here, the account stops listing its name, which revokes it there.
|
||||
* Only an answered list that no longer has the name marks it removed; a failed or lost one leaves it unconfirmed. */
|
||||
async removeAtAccount(bindings: NetworkBindingStore, binding: NetworkBinding): Promise<NetworkBinding> {
|
||||
const installation = this.list().find((i) => i.id === binding.installationId);
|
||||
if (!installation || accountRemoval(binding, installation.installationId) !== "unconfirmed") return binding;
|
||||
const until = this.deadline();
|
||||
return inListTurn(installation.id, async () => {
|
||||
// A replace while this waited moved the enrollment to another account installation, which can't drop this name.
|
||||
const current = this.get(installation.id);
|
||||
if (accountRemoval(binding, current.installationId) !== "unconfirmed") return bindings.get(binding.id)!;
|
||||
try {
|
||||
await this.setParticipants(bindings, current, () => this.listedNames(bindings, current), until);
|
||||
} catch (error) {
|
||||
throw new NetworkBindingError("account_update_unconfirmed",
|
||||
`${binding.id} is ${binding.state} here, but OpenRig couldn't confirm that the account dropped ${binding.participantName} (${(error as Error).message}). Run \`rig network disconnect ${binding.id}\` again.`, 502);
|
||||
}
|
||||
return bindings.get(binding.id)!;
|
||||
});
|
||||
}
|
||||
|
||||
/** `accountRemoval` against this binding's enrollment as it is now. */
|
||||
removalOf(binding: NetworkBinding): ReturnType<typeof accountRemoval> {
|
||||
const now = binding.installationId ? this.list().find((i) => i.id === binding.installationId) : undefined;
|
||||
return accountRemoval(binding, now?.installationId);
|
||||
}
|
||||
|
||||
private get(id: string): NetworkInstallation {
|
||||
return project(this.db.prepare("SELECT * FROM network_installations WHERE id = ?").get(id) as Row);
|
||||
}
|
||||
|
||||
private listedNames(bindings: NetworkBindingStore, installation: NetworkInstallation): string[] {
|
||||
return bindings.active().filter((b) => b.installationId === installation.id && b.participantName).map((b) => b.participantName!);
|
||||
}
|
||||
|
||||
private instance(): string {
|
||||
const instance = this.opts.selfHostId();
|
||||
if (!instance) throw new NetworkBindingError("instance_unknown", "This daemon hasn't resolved its host id yet. Try again once it's fully up.", 503);
|
||||
return instance;
|
||||
}
|
||||
|
||||
private deadline(): number {
|
||||
return Date.now() + (this.opts.timeoutMs ?? ACCOUNT_DEADLINE_MS);
|
||||
}
|
||||
|
||||
private async setParticipants(bindings: NetworkBindingStore, installation: NetworkInstallation, names: () => string[], until: number): Promise<Identities> {
|
||||
const instance = this.instance();
|
||||
const repair = `Get a new code from the account page and run rig network enroll <code> --site ${installation.site} --replace.`;
|
||||
const key = readPrivateToken(networkInstallationKeyFile(this.home, installation.id),
|
||||
(file) => `The installation key for ${installation.id} isn't at ${file}. ${repair}`);
|
||||
const put = () => this.call(installation.site, "PUT", "/api/connector/identities", key, { instance, participants: names() }, until);
|
||||
let answer = await put();
|
||||
// The account couldn't apply a drop while something else changed this installation: the list is read and sent once more.
|
||||
if (answer.status === 409 && errorCode(answer.body) === "drop_failed") answer = await put();
|
||||
if (answer.status !== 200) {
|
||||
const error = errorCode(answer.body);
|
||||
if (answer.status === 409 && error === "drop_failed") {
|
||||
throw new NetworkBindingError("account_busy", "The account was changing this installation's participants at the same time. Try again.", 503);
|
||||
}
|
||||
if (answer.status === 409) throw new NetworkBindingError(error ?? "name_taken", "Another connection to the account already uses that name. Choose another.", 409);
|
||||
if (answer.status === 401) throw new NetworkBindingError(error ?? "not_enrolled", `The account doesn't recognize this installation any more. ${repair}`, 401);
|
||||
if (answer.status === 400) throw new NetworkBindingError(error ?? "bad_names", "The account refused the participant names.", 400);
|
||||
throw new NetworkBindingError("account_unavailable", `The account couldn't answer (${answer.status}). Try again.`, 503);
|
||||
}
|
||||
const body = (answer.body ?? {}) as Partial<Identities>;
|
||||
if (!Array.isArray(body.participants) || typeof body.mcp !== "string" || typeof body.rig !== "string") {
|
||||
throw new NetworkBindingError("account_unexpected", "The account's answer had no participants, MCP front or rig.", 502);
|
||||
}
|
||||
// The answer is what this account installation now lists: a binding that ended while listed under it, whose name
|
||||
// isn't in the answer, is confirmed dropped. A name listed under an earlier installation is never confirmed here.
|
||||
const listed = new Set(body.participants.map((p) => p.name));
|
||||
for (const ended of bindings.list()) {
|
||||
if (ended.installationId === installation.id && ended.participantName && !listed.has(ended.participantName)
|
||||
&& accountRemoval(ended, installation.installationId) === "unconfirmed") {
|
||||
bindings.markAccountRemoved(ended.id);
|
||||
}
|
||||
}
|
||||
return body as Identities;
|
||||
}
|
||||
|
||||
private async call(site: string, method: string, path: string, key: string | null, payload: unknown, until: number): Promise<{ status: number; body: unknown }> {
|
||||
const left = until - Date.now();
|
||||
if (left <= 0) throw new NotSent("account_unavailable", `The account at ${site} wasn't reached in time: this command's time with the account ran out.`, 503);
|
||||
let res;
|
||||
try {
|
||||
// The key goes only to the site the person enrolled at: a redirect isn't followed with it, and a stalled call ends.
|
||||
res = await this.fetch(new URL(path, site).toString(), {
|
||||
method,
|
||||
headers: { "Content-Type": "application/json", ...(key ? { Authorization: `Bearer ${key}` } : {}) },
|
||||
body: JSON.stringify(payload),
|
||||
redirect: "error",
|
||||
signal: AbortSignal.timeout(left),
|
||||
});
|
||||
} catch (error) {
|
||||
const cause = (error as { cause?: { message?: unknown } }).cause?.message;
|
||||
if (typeof cause === "string" && /redirect/i.test(cause)) {
|
||||
throw new NetworkBindingError("account_redirected", `The account at ${site} answered with a redirect, which isn't followed with the installation key. Use the site's own URL.`, 502);
|
||||
}
|
||||
throw new NetworkBindingError("account_unavailable", `The account at ${site} couldn't be reached: ${(error as Error).message.slice(0, 200)}.`, 503);
|
||||
}
|
||||
let body: unknown = null;
|
||||
try { body = await res.json(); } catch { /* no JSON body */ }
|
||||
return { status: res.status, body };
|
||||
}
|
||||
}
|
||||
|
||||
function errorCode(body: unknown): string | null {
|
||||
const error = (body as { error?: unknown } | null)?.error;
|
||||
return typeof error === "string" && /^[a-z][a-z0-9_]*$/.test(error) ? error : null;
|
||||
}
|
||||
|
||||
/** The account's site: https, or plain http on the person's choice, with no credential in it. */
|
||||
function validSite(raw: unknown, allowHttp: boolean): string {
|
||||
if (typeof raw !== "string" || !raw.trim()) throw new NetworkBindingError("invalid_site", "Give the account's site URL with --site.");
|
||||
let url: URL;
|
||||
try { url = new URL(raw.trim()); } catch { throw new NetworkBindingError("invalid_site", `'${raw}' isn't a URL.`); }
|
||||
if (url.protocol !== "https:" && url.protocol !== "http:") throw new NetworkBindingError("invalid_site", "The site must use https (or http).");
|
||||
if (url.username || url.password) throw new NetworkBindingError("invalid_site", "The site URL can't hold a credential.");
|
||||
if (url.protocol === "http:" && !allowHttp) {
|
||||
throw new NetworkBindingError("http_not_allowed", `${url.origin} is plain http, so the installation key and every participant's token would cross the network unencrypted, where they can be read. Use the site's https URL, or enroll again with --allow-http to accept that.`);
|
||||
}
|
||||
return url.origin + "/";
|
||||
}
|
||||
@@ -34,6 +34,7 @@ import { createLinkedSuccessorLookup, deriveCrossHostSuccessorId, hasLinkedSucce
|
||||
import { stalledPickupFinding } from "./queue-pickup.js";
|
||||
import { SettingsStore } from "./user-settings/settings-store.js";
|
||||
import { loadHostRegistry } from "./hosts/hosts-registry-reader.js";
|
||||
import { readWakeOutcome } from "./connection-contract.js";
|
||||
|
||||
export const STUCK_SWEEP_INTERVAL_KEY = "queue.stuck_sweep_interval_seconds";
|
||||
export const DEFAULT_STUCK_SWEEP_INTERVAL_SECONDS = 300;
|
||||
@@ -470,8 +471,9 @@ function deliveryResolutionEvidence(
|
||||
if (row.claimedAt) return `source claimed at ${row.claimedAt}`;
|
||||
if (row.state === "blocked") return "source parked";
|
||||
if (kind === "undelivered-wake") {
|
||||
if (row.lastNudgeResult === "verified") return "source wake verified";
|
||||
if (row.lastNudgeResult?.startsWith("wake-accepted:")) return "source wake accepted by its hosted trigger";
|
||||
const wake = readWakeOutcome(row.lastNudgeResult);
|
||||
if (wake?.fact === "accepted" && wake.renderVerified) return "source wake verified";
|
||||
if (wake?.fact === "accepted" && wake.route === "hosted-trigger") return "source wake accepted by its hosted trigger";
|
||||
}
|
||||
}
|
||||
if (deps.queueRepo.humanNotificationPostedThisEpisode(qitemId)) return "current human ask posted";
|
||||
|
||||
@@ -12,6 +12,7 @@ import type { OperatorDeliveryEngine } from "./gateway/operator-delivery-engine.
|
||||
import { openPromptRefusals, CLOSE_PREFIX, PARKED_OWNER_POLICY_NAME, REFUSED_PREFIX } from "./policies/parked-owner-consumer.js";
|
||||
import { findQueueRecovery, recoveryTag } from "./queue-recovery.js";
|
||||
import { lastMeaningfulTransition, type WaitingView } from "./queue-waiting.js";
|
||||
import { readWakeOutcome } from "./connection-contract.js";
|
||||
// S01 (OPR.0.5.5.1) — WAKE OR ESCALATE ON BATONS. A handoff whose wake fails must never
|
||||
// silently park: today one failed nudge is recorded and nothing follows (the measured
|
||||
// dominant 0.5.3 failure class — a perfect surviving packet, a recipient never woken).
|
||||
@@ -383,28 +384,44 @@ function hasPickupEvidence(db: Database.Database, row: Pick<QueueItem, "qitemId"
|
||||
type WakeMode = "failed" | "unconfirmed" | "held";
|
||||
|
||||
function classifyWakeResult(lastNudgeResult: string | null): WakeMode | null {
|
||||
if (!lastNudgeResult) return null;
|
||||
if (lastNudgeResult.startsWith("failed:")) return "failed";
|
||||
// #344 follow-up — a typing-guard refusal (retained:) wrote NO pane input, so an
|
||||
// unclaimed baton keeps a retry path instead of stranding behind a terminal
|
||||
// recovery. Admission is scoped to unclaimed batons by unclaimedWakeMode(): the
|
||||
// claimed parked-owner arm's retry readback still selects `failed:%` only, so
|
||||
// admitting retained there would make recovery OWN a row that nothing retries.
|
||||
// Retry behaviour, as it actually runs: the first retry re-probes the guard and
|
||||
// records a fresh attempt; later retries reuse the same delivery id, whose
|
||||
// retained readback returns `retained` again even after the guard has cleared —
|
||||
// so the row re-probes once and then escalates rather than looping probes.
|
||||
// A draft-aware hold can clear on its own (the draft is sent or cleared), so it keeps that retry path. A hold or a
|
||||
// level that holds the wake is the person's choice: retrying is held again, so it reads as held.
|
||||
if (lastNudgeResult === "retained:typing_guard") return "held";
|
||||
if (lastNudgeResult.startsWith("retained:")) return "failed";
|
||||
if (
|
||||
lastNudgeResult === "delivered-ack-pending" ||
|
||||
lastNudgeResult.startsWith("indeterminate:") ||
|
||||
lastNudgeResult.startsWith("gateway-owned:")
|
||||
)
|
||||
return "unconfirmed";
|
||||
return null; // verified (or unknown vocabulary) — never enters the ladder
|
||||
// A claimed row's unconfirmed result is presented as `claimed` (presentedNudgeResult): the claim is pickup, which
|
||||
// outranks any wake outcome.
|
||||
if (lastNudgeResult === "claimed") return null;
|
||||
const outcome = readWakeOutcome(lastNudgeResult);
|
||||
if (!outcome) return null;
|
||||
switch (outcome.fact) {
|
||||
// Nothing reached the seat, so a retry can't double-deliver. An address no route can carry (`none`) has nothing
|
||||
// to retry; its own notice covers it.
|
||||
case "not-delivered":
|
||||
return outcome.route === "none" ? null : "failed";
|
||||
// A webhook failure after the POST: its effect is unknown, but it keeps the failed path and retry policy it has
|
||||
// always had.
|
||||
case "failed-effect-unknown":
|
||||
return "failed";
|
||||
// #344 follow-up — a typing-guard refusal (retained:) wrote NO pane input, so an
|
||||
// unclaimed baton keeps a retry path instead of stranding behind a terminal
|
||||
// recovery. Admission is scoped to unclaimed batons by unclaimedWakeMode(): the
|
||||
// claimed parked-owner arm's retry readback still selects `failed:%` only, so
|
||||
// admitting retained there would make recovery OWN a row that nothing retries.
|
||||
// Retry behaviour, as it actually runs: the first retry re-probes the guard and
|
||||
// records a fresh attempt; later retries reuse the same delivery id, whose
|
||||
// retained readback returns `retained` again even after the guard has cleared —
|
||||
// so the row re-probes once and then escalates rather than looping probes.
|
||||
// A draft-aware hold can clear on its own (the draft is sent or cleared), so it keeps that retry path. A hold or a
|
||||
// level that holds the wake is the person's choice: retrying is held again, so it reads as held.
|
||||
case "held":
|
||||
return outcome.heldBy === "typing-guard" ? "held" : "failed";
|
||||
// A rendered terminal wake, or a hosted trigger that took the ring (the stuck sweep owns a hosted member's
|
||||
// unclaimed work), never enters. An unverified terminal send rides the confirmation path.
|
||||
case "accepted":
|
||||
return outcome.renderVerified || outcome.route === "hosted-trigger" ? null : "unconfirmed";
|
||||
// Unknown, including a recorded result no reader recognises: it never retries (input may have arrived), and it
|
||||
// escalates past the window when nothing picked the work up, instead of reading as delivered.
|
||||
case "queued-not-started":
|
||||
case "turn-started":
|
||||
case "unknown":
|
||||
return "unconfirmed";
|
||||
}
|
||||
}
|
||||
|
||||
/** A retained (typing-guard) result is retryable ONLY for an unclaimed baton. The
|
||||
@@ -416,7 +433,7 @@ function classifyWakeResult(lastNudgeResult: string | null): WakeMode | null {
|
||||
function unclaimedWakeMode(row: Pick<QueueItem, "state" | "claimedAt" | "handedOffFrom" | "lastNudgeResult">): WakeMode | null {
|
||||
const mode = classifyWakeResult(row.lastNudgeResult);
|
||||
if (mode === null) return null;
|
||||
const retained = row.lastNudgeResult?.startsWith("retained:") ?? false;
|
||||
const retained = readWakeOutcome(row.lastNudgeResult)?.fact === "held";
|
||||
if (retained && !(row.state === "pending" && !row.claimedAt && row.handedOffFrom)) return null;
|
||||
return mode;
|
||||
}
|
||||
@@ -863,7 +880,9 @@ export async function runWakeLadderTick(deps: WakeLadderDeps): Promise<WakeLadde
|
||||
? `wake failed ${view.attempts} times over ${minutesSince(view.firstMarkerTs ?? row.tsCreated, now)} min`
|
||||
: mode === "held"
|
||||
? `held by ${row.destinationSession}'s typing guard for ${minutesSince(row.tsCreated, now)} min: nothing was typed, and it isn't retried`
|
||||
: `unconfirmed delivery with no pickup evidence over ${minutesSince(row.tsCreated, now)} min`;
|
||||
: readWakeOutcome(row.lastNudgeResult)?.route === null
|
||||
? `wake outcome unknown (unrecognised result "${row.lastNudgeResult!.slice(0, 80)}") with no pickup evidence over ${minutesSince(row.tsCreated, now)} min`
|
||||
: `unconfirmed delivery with no pickup evidence over ${minutesSince(row.tsCreated, now)} min`;
|
||||
if (hasConsumedEscalationHop(deps.db, row.qitemId)) {
|
||||
if (!due || suspended) continue;
|
||||
const consumedReason = `${reason}; escalates_to one-hop delivery already used; existing fallback retained`;
|
||||
|
||||
@@ -124,14 +124,25 @@ export interface CurrentSessionAddress {
|
||||
* Physical names win over derived names. Archived/historical bindings cannot
|
||||
* advertise a new alias, and two current matches never pick an arbitrary seat. */
|
||||
export function resolveCurrentSessionAddress(db: Database.Database, address: string): CurrentSessionAddress | null {
|
||||
const rows = db.prepare(`SELECT r.*, n.id AS nodeId, n.logical_id AS logicalId,
|
||||
const current = `SELECT r.name, r.archived_at, n.id AS nodeId, n.logical_id AS logicalId,
|
||||
s.id AS sessionId, s.session_name AS sessionName, s.status,
|
||||
replace(n.logical_id, '.', '-') || '@' || r.name AS senderAddress
|
||||
FROM nodes n JOIN rigs r ON r.id=n.rig_id
|
||||
JOIN bindings b ON b.node_id=n.id AND b.tmux_session IS NOT NULL
|
||||
JOIN sessions s ON s.node_id=n.id AND s.session_name=b.tmux_session
|
||||
AND s.id=(SELECT id FROM sessions WHERE node_id=n.id ORDER BY id DESC LIMIT 1)
|
||||
WHERE s.session_name=? OR replace(n.logical_id, '.', '-') || '@' || r.name=?`)
|
||||
CROSS JOIN sessions s ON s.node_id=n.id AND s.session_name=b.tmux_session
|
||||
AND s.id=(SELECT id FROM sessions WHERE node_id=n.id ORDER BY id DESC LIMIT 1)`;
|
||||
// Inventory passes physical names for every seat. The OR with a derived alias
|
||||
// made each lookup walk all nodes (or all historical sessions). Seek the exact
|
||||
// name first; only an absent active exact match needs the alias walk. Keep this
|
||||
// in one statement so both alternatives observe the same database snapshot.
|
||||
const rows = db.prepare(`WITH exact AS MATERIALIZED (
|
||||
${current} WHERE b.tmux_session=? AND r.archived_at IS NULL
|
||||
AND COALESCE(s.status, '') NOT IN ('superseded', 'detached', 'exited')
|
||||
) SELECT * FROM exact
|
||||
UNION ALL
|
||||
${current}
|
||||
WHERE replace(n.logical_id, '.', '-') || '@' || r.name=?
|
||||
AND NOT EXISTS (SELECT 1 FROM exact)`)
|
||||
.all(address, address) as Array<CurrentSessionAddress & { name: string; archived_at?: string | null; status: string }>;
|
||||
const active = rows.filter(row => row.archived_at == null && !['superseded', 'detached', 'exited'].includes(row.status));
|
||||
const exact = active.filter(row => row.sessionName === address);
|
||||
@@ -783,7 +794,7 @@ export class SessionRegistry {
|
||||
`SELECT s.* FROM sessions s
|
||||
JOIN nodes n ON s.node_id = n.id
|
||||
WHERE n.rig_id = ?
|
||||
ORDER BY s.created_at`
|
||||
ORDER BY s.created_at, s.rowid`
|
||||
)
|
||||
.all(rigId) as SessionRow[];
|
||||
|
||||
|
||||
@@ -282,6 +282,7 @@ function findCurrentClaudeQuestion(paneContent: string): string | null {
|
||||
const CLAUDE_STATUS_WARNINGS = [
|
||||
/^✘ Auto-update failed(?:: [^·\r\n]+)? · Run claude doctor$/,
|
||||
/^✘ Auto-update failed · Try claude doctor or (?:npm i -g @anthropic-ai\/claude-code|cd ~\/\.claude\/local && npm update @anthropic-ai\/claude-code)$/,
|
||||
/^✘ Auto-update failed · Try claude doctor or npm i -g @anthropic-ai\/claude-c…$/,
|
||||
/^tmux focus-events off · add 'set -g focus-events on' to ~\/\.tmux\.conf and re…$/,
|
||||
/^You've used (?:\d|[1-9]\d)% of your weekly limit · resets \d{1,2}(?::\d{2})?(?:am|pm) \(UTC\)$/,
|
||||
// #1641: Claude 2.1.293's update banner, whatever installer it names.
|
||||
|
||||
@@ -2,48 +2,76 @@ import { Hono, type Context } from "hono";
|
||||
import type { RigRepository } from "../domain/rig-repository.js";
|
||||
import { NetworkBindingError, NetworkBindingStore } from "../domain/network/network-bindings.js";
|
||||
import { networkClosedElsewhere } from "../domain/network/network-connector.js";
|
||||
import { NetworkInstallationStore, type AccountFetch } from "../domain/network/network-installations.js";
|
||||
import { getSelfHostId } from "../domain/hosts/fanout-contract.js";
|
||||
import { authBearerTokenMiddleware } from "../middleware/auth-bearer-token.js";
|
||||
import { OPENRIG_HOME } from "../openrig-compat.js";
|
||||
|
||||
/** A local seat's connections to a remote hub (`rig network`). Reads list every binding with its state and last contact;
|
||||
* the token is accepted on connect and never returned. Writes are gated like the hosts routes: enforced when an operator
|
||||
* bearer is configured, passed through on a loopback or tailnet-trusted daemon. */
|
||||
export function networkRoutes(opts?: { bearerToken?: string | null; home?: string }): Hono {
|
||||
/** A local seat's connections to a remote hub (`rig network`). Reads list every binding with its state and last contact,
|
||||
* and the account enrollments; tokens and keys are accepted and never returned. Writes are gated like the hosts routes:
|
||||
* enforced when an operator bearer is configured, passed through on a loopback or tailnet-trusted daemon. */
|
||||
export function networkRoutes(opts?: { bearerToken?: string | null; home?: string; accountFetch?: AccountFetch; selfHostId?: () => string | null; accountTimeoutMs?: number }): Hono {
|
||||
const router = new Hono();
|
||||
const requireAuth = authBearerTokenMiddleware({ expectedToken: opts?.bearerToken ?? null });
|
||||
router.use("/bindings", async (c, next) => (c.req.method === "GET" ? next() : requireAuth(c, next)));
|
||||
router.use("/bindings/:id/disconnect", requireAuth);
|
||||
router.use("/installations", requireAuth);
|
||||
|
||||
const store = (c: Context) => new NetworkBindingStore((c.get("rigRepo" as never) as RigRepository).db, opts?.home ?? OPENRIG_HOME);
|
||||
const home = opts?.home ?? OPENRIG_HOME;
|
||||
const db = (c: Context) => (c.get("rigRepo" as never) as RigRepository).db;
|
||||
const store = (c: Context) => new NetworkBindingStore(db(c), home);
|
||||
const installations = (c: Context) => new NetworkInstallationStore(db(c), home, { fetch: opts?.accountFetch, selfHostId: opts?.selfHostId ?? getSelfHostId, timeoutMs: opts?.accountTimeoutMs });
|
||||
const refused = (c: Context, error: unknown) => {
|
||||
if (error instanceof NetworkBindingError) return c.json({ error: error.code, message: error.message }, error.status);
|
||||
throw error;
|
||||
};
|
||||
const json = async (c: Context): Promise<Record<string, unknown> | null> => {
|
||||
try { return await c.req.json(); } catch { return null; }
|
||||
};
|
||||
|
||||
// The connector's own view (next pass, backoff, last pass) when it runs in this daemon; null otherwise.
|
||||
router.get("/bindings", (c) => {
|
||||
const connector = c.get("networkConnector" as never) as { snapshot(): unknown } | undefined;
|
||||
const db = (c.get("rigRepo" as never) as RigRepository).db;
|
||||
// Hub items someone else closed before a binding's result reached them: a conflict the person sees.
|
||||
const bindings = store(c).list().map((b) => ({ ...b, closedElsewhere: networkClosedElsewhere(db, b.id) }));
|
||||
return c.json({ bindings, connector: connector?.snapshot() ?? null });
|
||||
const enrolled = installations(c);
|
||||
const bindings = store(c).list().map((b) => ({ ...b, closedElsewhere: networkClosedElsewhere(db(c), b.id), accountRemoval: enrolled.removalOf(b) }));
|
||||
return c.json({ bindings, installations: enrolled.list(), connector: connector?.snapshot() ?? null });
|
||||
});
|
||||
|
||||
// With a hub URL and token, bind to that hub directly; with only a participant name, through the enrolled account.
|
||||
router.post("/bindings", async (c) => {
|
||||
let body: Record<string, unknown>;
|
||||
try { body = await c.req.json(); } catch { return c.json({ error: "invalid_body", message: "The body must be JSON." }, 400); }
|
||||
const body = await json(c);
|
||||
if (!body) return c.json({ error: "invalid_body", message: "The body must be JSON." }, 400);
|
||||
try {
|
||||
if (body.hubUrl === undefined && body.token === undefined && body.participantName !== undefined) {
|
||||
return c.json(await installations(c).connect(store(c), { session: body.session, name: body.participantName }), 201);
|
||||
}
|
||||
return c.json(store(c).create({ session: body.session, hubUrl: body.hubUrl, hubSeat: body.hubSeat, token: body.token, allowHttp: body.allowHttp }), 201);
|
||||
} catch (error) { return refused(c, error); }
|
||||
});
|
||||
|
||||
router.post("/bindings/:id/disconnect", async (c) => {
|
||||
let body: Record<string, unknown> = {};
|
||||
try { body = await c.req.json(); } catch { /* no reason given */ }
|
||||
router.post("/installations", async (c) => {
|
||||
const body = await json(c);
|
||||
if (!body) return c.json({ error: "invalid_body", message: "The body must be JSON." }, 400);
|
||||
try {
|
||||
return c.json(store(c).disconnect(c.req.param("id"), typeof body.reason === "string" ? body.reason : undefined));
|
||||
return c.json(await installations(c).enroll(store(c), { code: body.code, site: body.site, allowHttp: body.allowHttp, replace: body.replace }), 201);
|
||||
} catch (error) { return refused(c, error); }
|
||||
});
|
||||
|
||||
// The binding ends here first, then a binding made through the account is dropped there, which revokes its name.
|
||||
router.post("/bindings/:id/disconnect", async (c) => {
|
||||
const body = (await json(c)) ?? {};
|
||||
const bindings = store(c);
|
||||
const target = bindings.find(c.req.param("id"));
|
||||
if (!target) return refused(c, new NetworkBindingError("binding_not_found", `No network binding or connected seat is named '${c.req.param("id")}'.`, 404));
|
||||
let cleanup: unknown = null;
|
||||
try { bindings.disconnect(target.id, typeof body.reason === "string" ? body.reason : undefined); } catch (error) { cleanup = error; }
|
||||
try {
|
||||
const enrolled = installations(c);
|
||||
const ended = await enrolled.removeAtAccount(bindings, bindings.get(target.id)!);
|
||||
return cleanup ? refused(c, cleanup) : c.json({ ...ended, accountRemoval: enrolled.removalOf(ended) });
|
||||
} catch (error) { return refused(c, cleanup ?? error); }
|
||||
});
|
||||
|
||||
return router;
|
||||
}
|
||||
@@ -89,6 +89,23 @@ describe("cursor-bound composer input", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["", "person's draft"])("reads the observed clipped auto-update footer with input %j", body => {
|
||||
// A complete synthetic frame; the retained partial capture cannot supply physical cursor rows.
|
||||
const snapshot = claude(body);
|
||||
snapshot.screen = snapshot.screen.replace("? for shortcuts", "⏵⏵ accept edits on (shift+tab to cycle)\n✘ Auto-update failed · Try claude doctor or npm i -g @anthropic-ai/claude-c…");
|
||||
expect(inspectComposerInput(snapshot, "claude-code").state).toBe(body ? "text" : "empty");
|
||||
});
|
||||
|
||||
it("keeps near misses unknown below the footer and warning text inside a draft", () => {
|
||||
const warning = "✘ Auto-update failed · Try claude doctor or npm i -g @anthropic-ai/claude-c…";
|
||||
for (const nearMiss of [warning + " now", warning.replace("doctor", "doctors"), warning.replace("claude-c…", "other…")]) {
|
||||
const snapshot = claude("");
|
||||
snapshot.screen = snapshot.screen.replace("? for shortcuts", `? for shortcuts\n${nearMiss}`);
|
||||
expect(inspectComposerInput(snapshot, "claude-code").state).toBe("unknown");
|
||||
}
|
||||
expect(inspectComposerInput(claude(warning, undefined, 100), "claude-code").state).toBe("text");
|
||||
});
|
||||
|
||||
it("reads Claude's context hint below a named composer without skipping arbitrary rows", () => {
|
||||
const frame = (body: string, styled = false, hint = "~66k uncached · /clear to start fresh"): ComposerSnapshot => ({
|
||||
screen: [...Array<string>(22).fill(""),
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { readWakeOutcome } from "../src/domain/connection-contract.js";
|
||||
import { NUDGE_FAIL_PREFIX } from "../src/domain/policies/parked-owner-consumer.js";
|
||||
|
||||
describe("readWakeOutcome: every recorded nudge result reads as a delivery fact", () => {
|
||||
// One row per form a producer writes today (queue-repository, hosted-wake, the destination resolver, the parked-owner
|
||||
// consumer). A new producer form belongs here too, or it reads unknown.
|
||||
const cases: Array<[string, Record<string, unknown>]> = [
|
||||
["verified", { fact: "accepted", route: "terminal", renderVerified: true }],
|
||||
["delivered-ack-pending", { fact: "accepted", route: "terminal" }],
|
||||
["retained:typing_guard", { fact: "held", route: "terminal", heldBy: "typing-guard" }],
|
||||
["retained:draft_aware", { fact: "held", route: "terminal", heldBy: "draft-aware" }],
|
||||
["wake-accepted:webhook (hook) answered 202; pickup is the member's claim on this row", { fact: "accepted", route: "hosted-trigger" }],
|
||||
["indeterminate:webhook (hook) didn't answer within 5000 ms", { fact: "unknown", route: "hosted-trigger" }],
|
||||
// A webhook failure after the POST doesn't prove the member missed the ring; one before it does.
|
||||
["failed:webhook (hook) answered 500", { fact: "failed-effect-unknown", route: "hosted-trigger", reporter: "outside-adapter" }],
|
||||
["failed:webhook (hook) answered 302, a redirect, which isn't followed", { fact: "failed-effect-unknown", route: "hosted-trigger", reporter: "outside-adapter" }],
|
||||
["failed:webhook (hook) couldn't be reached (ECONNRESET)", { fact: "failed-effect-unknown", route: "hosted-trigger" }],
|
||||
["failed:webhook (hook) isn't set: no webhook URL to ring", { fact: "not-delivered", route: "hosted-trigger" }],
|
||||
["failed:webhook (hook) not rung: its Authorization value (auth) isn't set", { fact: "not-delivered", route: "hosted-trigger" }],
|
||||
["failed:webhook (hook) not rung: the URL is invalid (bad scheme)", { fact: "not-delivered", route: "hosted-trigger" }],
|
||||
["gateway-owned: 'slack:U1@external' is a virtual @external destination", { fact: "unknown", route: "gateway" }],
|
||||
["unroutable: 'm@r' is a hosted member with no wake trigger", { fact: "not-delivered", route: "none" }],
|
||||
["indeterminate:wake send timed out", { fact: "unknown", route: "terminal" }],
|
||||
["indeterminate:earlier delivery unconfirmed; retry failed:x", { fact: "unknown", route: "terminal" }],
|
||||
["indeterminate: parked-owner wake delivery — timed out", { fact: "unknown", route: "terminal" }],
|
||||
["failed:tmux session not found", { fact: "not-delivered", route: "terminal" }],
|
||||
["failed:terminal-seat: 'sh@r' is a terminal seat", { fact: "not-delivered", route: "terminal" }],
|
||||
["failed:wake not retained: disk full", { fact: "not-delivered", route: "terminal" }],
|
||||
[`${NUDGE_FAIL_PREFIX} — refused`, { fact: "not-delivered", route: "terminal" }],
|
||||
];
|
||||
|
||||
it.each(cases)("%s", (recorded, expected) => {
|
||||
const outcome = readWakeOutcome(recorded)!;
|
||||
expect(outcome).toMatchObject({ reporter: "openrig-observed", ...expected, recorded });
|
||||
});
|
||||
|
||||
it("only a rendered terminal wake is render-verified; an unverified send is accepted, nothing more", () => {
|
||||
expect(readWakeOutcome("verified")?.renderVerified).toBe(true);
|
||||
expect(readWakeOutcome("delivered-ack-pending")?.renderVerified).toBeUndefined();
|
||||
});
|
||||
|
||||
it("an unrecognised result reads unknown with no route, never success or failure", () => {
|
||||
for (const recorded of ["accepted-by-some-new-route", "claimed", "", "VERIFIED", "ok"]) {
|
||||
expect(readWakeOutcome(recorded)).toEqual({ fact: "unknown", reporter: "openrig-observed", route: null, recorded });
|
||||
}
|
||||
});
|
||||
|
||||
it("no recorded attempt reads as no outcome", () => {
|
||||
expect(readWakeOutcome(null)).toBeNull();
|
||||
expect(readWakeOutcome(undefined)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -6,6 +6,7 @@ import Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { coreSchema } from "../src/db/migrations/001_core_schema.js";
|
||||
import { rigArchiveSchema } from "../src/db/migrations/042_rig_archive.js";
|
||||
import { bindingsSessionsSchema } from "../src/db/migrations/002_bindings_sessions.js";
|
||||
import { eventsSchema } from "../src/db/migrations/003_events.js";
|
||||
import { queueItemSummarySchema } from "../src/db/migrations/044_queue_item_summary.js";
|
||||
@@ -106,7 +107,7 @@ describe("the projector's execution queue", () => {
|
||||
const NOW = new Date("2026-10-25T12:00:00.000Z");
|
||||
function fixture(file?: string) {
|
||||
const db = track(file ? createDb(file) : createDb());
|
||||
migrate(db, [coreSchema, bindingsSessionsSchema, eventsSchema, queueItemSummarySchema, queueItemsSchema, queueTransitionsSchema, queueTransitionWakesSchema, viewsCustomSchema]);
|
||||
migrate(db, [coreSchema, bindingsSessionsSchema, rigArchiveSchema, eventsSchema, queueItemSummarySchema, queueItemsSchema, queueTransitionsSchema, queueTransitionWakesSchema, viewsCustomSchema]);
|
||||
return db;
|
||||
}
|
||||
const insertRow = (db: Database.Database, id: string, state: string, tags: string[], body: string, claimedAt: string | null = "2026-10-25T11:00:00.000Z") =>
|
||||
|
||||
@@ -19,6 +19,7 @@ import type Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { coreSchema } from "../src/db/migrations/001_core_schema.js";
|
||||
import { rigArchiveSchema } from "../src/db/migrations/042_rig_archive.js";
|
||||
import { bindingsSessionsSchema } from "../src/db/migrations/002_bindings_sessions.js";
|
||||
import { eventsSchema } from "../src/db/migrations/003_events.js";
|
||||
import { queueItemSummarySchema } from "../src/db/migrations/044_queue_item_summary.js";
|
||||
@@ -167,7 +168,7 @@ describe("execution view — S27 (OPR.0.5.6.27)", () => {
|
||||
db = createDb();
|
||||
migrate(db, [
|
||||
coreSchema,
|
||||
bindingsSessionsSchema,
|
||||
bindingsSessionsSchema, rigArchiveSchema,
|
||||
eventsSchema,
|
||||
queueItemsSchema,
|
||||
queueItemSummarySchema,
|
||||
|
||||
@@ -6,6 +6,7 @@ import { queueInboxCheckinsSchema } from "../../src/db/migrations/158_queue_inbo
|
||||
import { queueItemsHandedOffFromIndexSchema } from "../../src/db/migrations/159_queue_items_handed_off_from_index.js";
|
||||
import { networkBindingsSchema } from "../../src/db/migrations/160_network_bindings.js";
|
||||
import { networkReceiptsSchema } from "../../src/db/migrations/161_network_receipts.js";
|
||||
import { networkInstallationsSchema } from "../../src/db/migrations/163_network_installations.js";
|
||||
import { activityGrantsSchema } from "../../src/db/migrations/143_activity_grants.js";
|
||||
import { environmentBindingsSchema } from "../../src/db/migrations/144_environment_bindings.js";
|
||||
import { nodePermissionsToSchema } from "../../src/db/migrations/145_node_permissions_to.js";
|
||||
@@ -108,6 +109,7 @@ import { humanQuestionsSchema } from "../../src/db/migrations/091_human_question
|
||||
import { nodeEffortSchema } from "../../src/db/migrations/092_node_effort.js";
|
||||
import { rigPolicySchema } from "../../src/db/migrations/041_rig_policy.js";
|
||||
import { rigArchiveSchema } from "../../src/db/migrations/042_rig_archive.js";
|
||||
import { sessionAddressReadIndexesSchema } from "../../src/db/migrations/162_session_address_read_indexes.js";
|
||||
import { resumeProvenanceSchema } from "../../src/db/migrations/043_resume_provenance.js";
|
||||
import { resumeVerificationSchema } from "../../src/db/migrations/045_resume_verification.js";
|
||||
import { seatIdentityVerdictsSchema } from "../../src/db/migrations/046_seat_identity_verdicts.js";
|
||||
@@ -168,7 +170,7 @@ import fs from "node:fs";
|
||||
|
||||
/** Seam B R6: the canonical full-fixture migration list, exported so file-backed
|
||||
* DB-reopen tests migrate IDENTICALLY to createFullTestDb. */
|
||||
export const migrationsForFullTestDb = [coreSchema, bindingsSessionsSchema, eventsSchema, snapshotsSchema, checkpointsSchema, resumeMetadataSchema, nodeSpecFieldsSchema, packagesSchema, installJournalSchema, journalSeqSchema, bootstrapSchema, discoverySchema, discoveryFkFix, agentspecRebootSchema, startupContextSchema, chatMessagesSchema, podNamespaceSchema, contextUsageSchema, externalCliAttachmentSchema, rigServicesSchema, seatHandoverObservabilitySchema, nodeCodexConfigProfileSchema, nodePermissionPolicySchema, rigPermissionPolicySchema, nodePolicyProvenanceSchema, rigPolicyProvenanceSchema, streamItemsSchema, queueItemsSchema, queueTransitionsSchema, rigPolicySchema, rigArchiveSchema, resumeProvenanceSchema, resumeVerificationSchema, seatIdentityVerdictsSchema, selfHostIdentitySchema, occupantTenuresSchema, daemonLifecycleSchema, watchdogJobsSchema, occupantGenerationStampsSchema, projectionManifestSchema, watchdogTargetGenerationSchema, appliedLaunchObservationsSchema, appliedLaunchObservationInvalidationsSchema, threadSeatMapSchema, queueTransitionWakesSchema, nodeSessionSourceSchema, scopedOperatingPostureSchema, humanNotificationIntentSchema, reviewReadIndexesSchema, inventoryEventIndexesSchema, rigClaudeManagedBlockFileSchema, nodePermissionSelectionsSchema, humanReplyToSchema, humanQuestionsSchema, nodeEffortSchema, rigNonInterruptiveSchema, rigInstallRootSchema, threadPartMapSchema, queueTransitionsQitemIdOrderSchema, queueItemsRecoveryTagsSchema, resumeRotationSchema, sessionStopIntentSchema, nodeEffortChoiceSchema, modePostureCoexistSchema, seatEnvSchema, appliedLaunchDaemonOptOutSchema, sessionCommunityHarnessSchema, threadSeatMapChannelKeySchema, queueItemWakeInputBoxSchema, providerAccountReadingsSchema, humanQuietWindowsSchema, seatIdentityPanePidSchema, seatIdentityProcessEvidenceSchema, rigStartupLayersSchema, humanOperatingModesSchema, queueDelegatedDecisionSchema, nodeAdvisorSchema, operationCorrelationSchema, terminalPolicySchema, seatRuntimeSwitchesSchema, claudeTokenReadingsSchema, nodeLaunchRecipeSchema, nodeClaudeTokenRefSchema, nodeClaudePermissionsSchema, nodeCodexPermissionsSchema, escalationTargetRefsSchema, sessionAgentRunnerSchema, nodeAccountHomeSchema, nodeClaudeMcpSchema, activityGrantsSchema, environmentBindingsSchema, nodePermissionsToSchema, nodeStartupGuidanceFileSchema, managedBlockWritesSchema, streamIntakeCursorSchema, nodeLaunchWrapperSchema, piAccountSwitchesSchema, eventsRetentionMarksSchema, queueErasureSchema, queueErasureBatchesSchema, queueInboxCheckinsSchema, queueItemsHandedOffFromIndexSchema, networkBindingsSchema, networkReceiptsSchema];
|
||||
export const migrationsForFullTestDb = [coreSchema, bindingsSessionsSchema, eventsSchema, snapshotsSchema, checkpointsSchema, resumeMetadataSchema, nodeSpecFieldsSchema, packagesSchema, installJournalSchema, journalSeqSchema, bootstrapSchema, discoverySchema, discoveryFkFix, agentspecRebootSchema, startupContextSchema, chatMessagesSchema, podNamespaceSchema, contextUsageSchema, externalCliAttachmentSchema, rigServicesSchema, seatHandoverObservabilitySchema, nodeCodexConfigProfileSchema, nodePermissionPolicySchema, rigPermissionPolicySchema, nodePolicyProvenanceSchema, rigPolicyProvenanceSchema, streamItemsSchema, queueItemsSchema, queueTransitionsSchema, rigPolicySchema, rigArchiveSchema, resumeProvenanceSchema, resumeVerificationSchema, seatIdentityVerdictsSchema, selfHostIdentitySchema, occupantTenuresSchema, daemonLifecycleSchema, watchdogJobsSchema, occupantGenerationStampsSchema, projectionManifestSchema, watchdogTargetGenerationSchema, appliedLaunchObservationsSchema, appliedLaunchObservationInvalidationsSchema, threadSeatMapSchema, queueTransitionWakesSchema, nodeSessionSourceSchema, scopedOperatingPostureSchema, humanNotificationIntentSchema, reviewReadIndexesSchema, inventoryEventIndexesSchema, rigClaudeManagedBlockFileSchema, nodePermissionSelectionsSchema, humanReplyToSchema, humanQuestionsSchema, nodeEffortSchema, rigNonInterruptiveSchema, rigInstallRootSchema, threadPartMapSchema, queueTransitionsQitemIdOrderSchema, queueItemsRecoveryTagsSchema, resumeRotationSchema, sessionStopIntentSchema, nodeEffortChoiceSchema, modePostureCoexistSchema, seatEnvSchema, appliedLaunchDaemonOptOutSchema, sessionCommunityHarnessSchema, threadSeatMapChannelKeySchema, queueItemWakeInputBoxSchema, providerAccountReadingsSchema, humanQuietWindowsSchema, seatIdentityPanePidSchema, seatIdentityProcessEvidenceSchema, rigStartupLayersSchema, humanOperatingModesSchema, queueDelegatedDecisionSchema, nodeAdvisorSchema, operationCorrelationSchema, terminalPolicySchema, seatRuntimeSwitchesSchema, claudeTokenReadingsSchema, nodeLaunchRecipeSchema, nodeClaudeTokenRefSchema, nodeClaudePermissionsSchema, nodeCodexPermissionsSchema, escalationTargetRefsSchema, sessionAgentRunnerSchema, nodeAccountHomeSchema, nodeClaudeMcpSchema, activityGrantsSchema, environmentBindingsSchema, nodePermissionsToSchema, nodeStartupGuidanceFileSchema, managedBlockWritesSchema, streamIntakeCursorSchema, nodeLaunchWrapperSchema, piAccountSwitchesSchema, eventsRetentionMarksSchema, queueErasureSchema, queueErasureBatchesSchema, queueInboxCheckinsSchema, queueItemsHandedOffFromIndexSchema, networkBindingsSchema, networkReceiptsSchema, sessionAddressReadIndexesSchema, networkInstallationsSchema];
|
||||
|
||||
/**
|
||||
* P24 — the DECLARED exclusions for {@link migrationsForFullTestDb}. That list is deliberately a
|
||||
|
||||
@@ -16,7 +16,7 @@ import { NetworkBindingStore, networkCredentialFile } from "../src/domain/networ
|
||||
import { Hono } from "hono";
|
||||
import { connectMcpHub, HubError, type HubConnect, type HubItem, type HubSession } from "../src/domain/network/hub-client.js";
|
||||
import { networkRoutes } from "../src/routes/network.js";
|
||||
import { NetworkConnectorService, networkLocalItemId, runNetworkBindingPass, type NetworkConnectorDeps } from "../src/domain/network/network-connector.js";
|
||||
import { NetworkConnectorService, networkClosedElsewhere, networkLocalItemId, runNetworkBindingPass, type NetworkConnectorDeps } from "../src/domain/network/network-connector.js";
|
||||
|
||||
const PARTICIPANT = "alex@team";
|
||||
|
||||
@@ -424,6 +424,16 @@ describe("network connector", () => {
|
||||
expect(repo.getById(local!.qitemId)!.resolution).toBe("Here it is.");
|
||||
});
|
||||
|
||||
it("closed-elsewhere conflicts are listed by when they settled, and ones that settled in the same millisecond by local item", () => {
|
||||
const add = db.prepare(`INSERT INTO network_receipts (local_qitem_id, binding_id, hub_qitem_id, created_at, settled_at, outcome)
|
||||
VALUES (?, 'netb-order', ?, '2026-11-11T19:00:00.000Z', ?, 'closed_on_hub')`);
|
||||
// The latest has the first id, and the two that tie were recorded, and name hub items, out of local-item order.
|
||||
add.run("qitem-net-a", "hub-1", "2026-11-11T19:00:00.001Z");
|
||||
add.run("qitem-net-c", "hub-2", "2026-11-11T19:00:00.000Z");
|
||||
add.run("qitem-net-b", "hub-3", "2026-11-11T19:00:00.000Z");
|
||||
expect(networkClosedElsewhere(db, "netb-order").map((c) => c.localItem)).toEqual(["qitem-net-b", "qitem-net-c", "qitem-net-a"]);
|
||||
});
|
||||
|
||||
it("a hub item already failed elsewhere gets no write and isn't our result, and a close that loses a race is read back as closed elsewhere", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-f1");
|
||||
@@ -451,10 +461,12 @@ describe("network connector", () => {
|
||||
// A later pass's ordinary contact doesn't clear it, and it's read from the receipts, so a restart keeps it.
|
||||
expect((await pass(b.id)).error).toBeUndefined();
|
||||
const listed = await (await app.request("/api/network/bindings")).json() as { bindings: Array<{ id: string; closedElsewhere: unknown[] }> };
|
||||
expect(listed.bindings.find((x) => x.id === b.id)!.closedElsewhere).toEqual([
|
||||
// Listed in the order they settled, which can fall in one millisecond or two, so compared by local item.
|
||||
const byLocal = (x: { localItem: string }, y: { localItem: string }) => x.localItem.localeCompare(y.localItem);
|
||||
expect([...listed.bindings.find((x) => x.id === b.id)!.closedElsewhere as Array<{ localItem: string }>].sort(byLocal)).toEqual([
|
||||
{ hubItem: "hub-f1", localItem: f1!.qitemId, note: `The hub item was closed elsewhere (failed) before this result reached it. The result stays on ${f1!.qitemId}.` },
|
||||
{ hubItem: "hub-f2", localItem: f2!.qitemId, note: `The hub item was closed elsewhere (done, no-follow-on) before this result reached it. The result stays on ${f2!.qitemId}.` },
|
||||
].sort((x, y) => x.localItem.localeCompare(y.localItem)));
|
||||
].sort(byLocal));
|
||||
});
|
||||
|
||||
it("a close without a resolution sends only the closer's own closing note, never another note on the item", async () => {
|
||||
|
||||
@@ -0,0 +1,481 @@
|
||||
// Enrolling in an account and connecting seats by name, through the daemon's real network routes, against an account
|
||||
// that answers the way the account's connector routes do: a one-time code for an installation key, then a participant
|
||||
// list that answers a bearer for each newly listed name and revokes a name dropped from it.
|
||||
import { afterEach, beforeEach, expect, it, vi } from "vitest";
|
||||
import Database from "better-sqlite3";
|
||||
import { Hono } from "hono";
|
||||
import { mkdtempSync, readdirSync, rmSync, statSync } from "node:fs";
|
||||
import { setTimeout as sleep } from "node:timers/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { ALL_MIGRATIONS } from "../src/db/all-migrations.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { RigRepository } from "../src/domain/rig-repository.js";
|
||||
import { readNetworkCredential, readPrivateToken } from "../src/domain/network/network-bindings.js";
|
||||
import { networkInstallationKeyFile, type AccountFetch } from "../src/domain/network/network-installations.js";
|
||||
import { networkRoutes } from "../src/routes/network.js";
|
||||
|
||||
/** Makes removing a path with this ending fail, the way a busy file does. */
|
||||
const failRemoval = vi.hoisted(() => ({ suffix: null as string | null }));
|
||||
vi.mock("node:fs", async (importOriginal) => {
|
||||
const real = await importOriginal<typeof import("node:fs")>();
|
||||
return {
|
||||
...real,
|
||||
rmSync: (...args: Parameters<typeof real.rmSync>) => {
|
||||
if (failRemoval.suffix && String(args[0]).endsWith(failRemoval.suffix)) throw Object.assign(new Error("EBUSY: resource busy or locked, rm"), { code: "EBUSY" });
|
||||
return real.rmSync(...args);
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
const KEY = `oik1_cust_inst-1_${"k".repeat(43)}`;
|
||||
|
||||
class FakeAccount {
|
||||
readonly codes = new Set(["ec1.good", "ec1.next"]);
|
||||
/** Each enrollment is a new installation with its own key and an empty list. */
|
||||
enrollments = 0;
|
||||
key = KEY;
|
||||
readonly used = new Set<string>();
|
||||
instance: string | null = null;
|
||||
names = new Map<string, { gen: number; state: "connected" | "revoked" }>();
|
||||
readonly elsewhere = new Set(["taken"]);
|
||||
readonly puts: string[][] = [];
|
||||
calls = 0;
|
||||
readonly inits: Array<{ redirect: string; signal: AbortSignal }> = [];
|
||||
/** Makes the next call fail the way fetch does on a redirect it won't follow. */
|
||||
redirectNext = false;
|
||||
/** The next call never answers. */
|
||||
stallNext = false;
|
||||
/** The next participant list is applied, then its answer is lost. */
|
||||
loseNext = false;
|
||||
down = false;
|
||||
mcp = "https://account.example/mcp";
|
||||
/** The next participant lists the account can't apply, because something else changed the installation. */
|
||||
dropFails = 0;
|
||||
/** How long a participant list waits before it's applied, so concurrent requests overlap. */
|
||||
delayMs = 0;
|
||||
/** After this many applied lists, the account goes down. */
|
||||
downAfterPuts: number | null = null;
|
||||
|
||||
/** The owner removed this installation on the account page. */
|
||||
forget(): void { this.instance = null; }
|
||||
|
||||
readonly fetch: AccountFetch = async (url, init) => {
|
||||
this.calls++;
|
||||
this.inits.push({ redirect: init.redirect, signal: init.signal });
|
||||
if (this.down) throw new Error("connect ECONNREFUSED");
|
||||
if (this.redirectNext) { this.redirectNext = false; throw Object.assign(new TypeError("fetch failed"), { cause: new Error("unexpected redirect") }); }
|
||||
if (this.stallNext) { this.stallNext = false; await new Promise((_resolve, reject) => init.signal.addEventListener("abort", () => reject(init.signal.reason), { once: true })); }
|
||||
const path = new URL(url).pathname;
|
||||
const body = JSON.parse(init.body) as { code?: string; instance?: string; participants?: string[] };
|
||||
const reply = (status: number, json: unknown) => ({ status, json: async () => json });
|
||||
if (path === "/api/connector/enroll" && init.method === "POST") {
|
||||
if (this.used.has(body.code!)) return reply(409, { error: "code_not_current" });
|
||||
if (!this.codes.has(body.code!)) return reply(400, { error: "code_not_current" });
|
||||
this.used.add(body.code!);
|
||||
this.instance = body.instance!;
|
||||
this.enrollments++;
|
||||
this.key = `${KEY}${this.enrollments}`;
|
||||
this.names = new Map();
|
||||
return reply(200, { installation: { id: `inst-${this.enrollments}`, name: "lab" }, key: this.key });
|
||||
}
|
||||
if (path === "/api/connector/identities" && init.method === "PUT") {
|
||||
if (init.headers.Authorization !== `Bearer ${this.key}` || body.instance !== this.instance) return reply(401, { error: "not_enrolled" });
|
||||
if (this.delayMs) await sleep(this.delayMs);
|
||||
const list = body.participants!;
|
||||
if (list.some((n) => this.elsewhere.has(n))) return reply(409, { error: "name_taken" });
|
||||
if (this.dropFails > 0) { this.dropFails--; return reply(409, { error: "drop_failed" }); }
|
||||
this.puts.push(list);
|
||||
for (const [name, entry] of this.names) if (!list.includes(name)) entry.state = "revoked";
|
||||
const participants = list.map((name) => {
|
||||
const entry = this.names.get(name);
|
||||
if (entry?.state === "connected") return { name, state: "connected" };
|
||||
const gen = (entry?.gen ?? 0) + 1;
|
||||
this.names.set(name, { gen, state: "connected" });
|
||||
return { name, state: "connected", bearer: `bear-${name}-${gen}` };
|
||||
});
|
||||
if (this.downAfterPuts !== null && this.puts.length >= this.downAfterPuts) this.down = true;
|
||||
if (this.loseNext) { this.loseNext = false; throw new Error("socket hang up"); }
|
||||
return reply(200, { participants, mcp: this.mcp, rig: "acct" });
|
||||
}
|
||||
return reply(404, { error: "not_found" });
|
||||
};
|
||||
}
|
||||
|
||||
let db: Database.Database;
|
||||
let home: string;
|
||||
let rigs: RigRepository;
|
||||
let account: FakeAccount;
|
||||
let app: Hono;
|
||||
let seq = 0;
|
||||
let hostId: string | null = "host-a";
|
||||
beforeEach(() => {
|
||||
hostId = "host-a";
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
rigs = new RigRepository(db);
|
||||
home = mkdtempSync(join(tmpdir(), "openrig-network-account-"));
|
||||
account = new FakeAccount();
|
||||
app = new Hono();
|
||||
app.use("*", async (c, next) => { c.set("rigRepo" as never, rigs as never); await next(); });
|
||||
app.route("/api/network", networkRoutes({ home, accountFetch: account.fetch, selfHostId: () => hostId, accountTimeoutMs: 1000 }));
|
||||
});
|
||||
afterEach(() => { failRemoval.suffix = null; db.close(); rmSync(home, { recursive: true, force: true }); });
|
||||
|
||||
function seat(session: string): void {
|
||||
const rig = rigs.listRigs().find((r) => r.name === "lab") ?? rigs.createRig("lab");
|
||||
const node = rigs.addNode(rig.id, `n${++seq}`, { runtime: "claude-code" });
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES (?, ?, ?, 'running')").run(`s${seq}`, node.id, session);
|
||||
}
|
||||
const post = async (path: string, body: unknown) => {
|
||||
const res = await app.request(`/api/network${path}`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(body) });
|
||||
return { status: res.status, data: await res.json() as Record<string, any> };
|
||||
};
|
||||
const enroll = () => post("/installations", { code: "ec1.good", site: "https://account.example" });
|
||||
const connect = (session: string, name: string) => post("/bindings", { session, participantName: name });
|
||||
const replace = () => post("/installations", { code: "ec1.next", site: "https://account.example", replace: true });
|
||||
const status = async () => (await (await app.request("/api/network/bindings")).json()) as {
|
||||
bindings: Array<{ id: string; state: string; accountRemoval: string | null; accountRemovedAt: string | null }>; installations: Array<{ installationId: string }> };
|
||||
|
||||
it("enrolling trades the code for a key kept privately; a used code is refused with its reason; a second enroll at the site sends nothing", async () => {
|
||||
const enrolled = await enroll();
|
||||
expect(enrolled).toMatchObject({ status: 201, data: { site: "https://account.example/", installationId: "inst-1", name: "lab" } });
|
||||
expect(enrolled.data.id).toMatch(/^neti-[0-9a-f]{20}$/);
|
||||
expect(account.instance).toBe("host-a");
|
||||
const keyFile = networkInstallationKeyFile(home, enrolled.data.id);
|
||||
expect(statSync(keyFile).mode & 0o777).toBe(0o600);
|
||||
expect(readPrivateToken(keyFile, () => "missing")).toBe(account.key);
|
||||
const listed = await (await app.request("/api/network/bindings")).text();
|
||||
expect(listed).toContain(enrolled.data.id);
|
||||
expect(listed).not.toContain(KEY);
|
||||
|
||||
const calls = account.calls;
|
||||
expect(await enroll()).toMatchObject({ status: 409, data: { error: "already_enrolled" } });
|
||||
expect(account.calls).toBe(calls);
|
||||
expect(await post("/installations", { code: "ec1.good", site: "http://other.example", allowHttp: false })).toMatchObject({ status: 400, data: { error: "http_not_allowed" } });
|
||||
expect(account.calls).toBe(calls);
|
||||
|
||||
// The same code at another site: the account says it's spent.
|
||||
expect(await post("/installations", { code: "ec1.good", site: "https://second.example" })).toMatchObject({ status: 409, data: { error: "code_not_current" } });
|
||||
});
|
||||
|
||||
it("connecting by name lists it with the account and binds to the account's MCP front with its bearer; a second seat keeps the first listed", async () => {
|
||||
seat("a@lab");
|
||||
seat("b@lab");
|
||||
const enrolled = await enroll();
|
||||
const alpha = await connect("a@lab", "alpha");
|
||||
expect(alpha).toMatchObject({ status: 201, data: { localSession: "a@lab", hubUrl: "https://account.example/mcp", hubSeat: "alpha@acct",
|
||||
installationId: enrolled.data.id, participantName: "alpha", state: "active" } });
|
||||
expect(readNetworkCredential(home, alpha.data.id)).toBe("bear-alpha-1");
|
||||
const beta = await connect("b@lab", "beta");
|
||||
expect(beta.status).toBe(201);
|
||||
expect(account.puts.at(-1)).toEqual(["alpha", "beta"]);
|
||||
expect(readNetworkCredential(home, beta.data.id)).toBe("bear-beta-1");
|
||||
expect(account.names.get("alpha")).toEqual({ gen: 1, state: "connected" });
|
||||
});
|
||||
|
||||
it("a lost answer to the participant list is recovered on the next connect: the name is dropped and listed again for a fresh bearer", async () => {
|
||||
seat("a@lab");
|
||||
await enroll();
|
||||
account.loseNext = true;
|
||||
expect(await connect("a@lab", "alpha")).toMatchObject({ status: 503, data: { error: "account_unavailable" } });
|
||||
expect(account.names.get("alpha")).toEqual({ gen: 1, state: "connected" });
|
||||
const again = await connect("a@lab", "alpha");
|
||||
expect(again.status).toBe(201);
|
||||
expect(account.puts.slice(-2)).toEqual([[], ["alpha"]]);
|
||||
expect(readNetworkCredential(home, again.data.id)).toBe("bear-alpha-2");
|
||||
});
|
||||
|
||||
it("one deadline covers all of a command's account calls: a connect that needs three lists runs out of time instead of waiting for each", async () => {
|
||||
seat("a@lab");
|
||||
await enroll();
|
||||
account.loseNext = true;
|
||||
await connect("a@lab", "alpha");
|
||||
// Each list now takes 600 ms against this daemon's 1 s deadline, and recovering the lost answer needs three.
|
||||
account.delayMs = 600;
|
||||
const late = await connect("a@lab", "alpha");
|
||||
expect(late).toMatchObject({ status: 503, data: { error: "account_unavailable" } });
|
||||
expect(late.data.message).toContain("this command's time with the account ran out");
|
||||
expect(account.puts.at(-1)).toEqual([]);
|
||||
});
|
||||
|
||||
it("a name another connection holds is refused, and a seat that can't be connected sends nothing to the account", async () => {
|
||||
seat("a@lab");
|
||||
await enroll();
|
||||
expect(await connect("a@lab", "taken")).toMatchObject({ status: 409, data: { error: "name_taken" } });
|
||||
const calls = account.calls;
|
||||
expect(await connect("nobody@lab", "alpha")).toMatchObject({ status: 404, data: { error: "seat_not_found" } });
|
||||
expect(await connect("a@lab", "Not A Name")).toMatchObject({ status: 400, data: { error: "invalid_name" } });
|
||||
expect(account.calls).toBe(calls);
|
||||
const listed = await (await app.request("/api/network/bindings")).json() as { bindings: unknown[] };
|
||||
expect(listed.bindings).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("disconnect drops the name at the account; when the account can't be reached the binding still ends, and the retry it names drops it", async () => {
|
||||
seat("a@lab");
|
||||
seat("b@lab");
|
||||
await enroll();
|
||||
const alpha = await connect("a@lab", "alpha");
|
||||
await connect("b@lab", "beta");
|
||||
account.down = true;
|
||||
const failed = await post("/bindings/a@lab/disconnect", {});
|
||||
expect(failed).toMatchObject({ status: 502, data: { error: "account_update_unconfirmed" } });
|
||||
expect(failed.data.message).toContain("couldn't confirm that the account dropped alpha");
|
||||
expect(failed.data.message).toContain(`Run \`rig network disconnect ${alpha.data.id}\` again.`);
|
||||
expect(account.names.get("alpha")!.state).toBe("connected");
|
||||
// Status keeps saying it's unconfirmed until an answered list shows otherwise.
|
||||
const status = await (await app.request("/api/network/bindings")).json() as { bindings: Array<{ id: string; accountRemoval: string | null }> };
|
||||
expect(status.bindings.find((b) => b.id === alpha.data.id)!.accountRemoval).toBe("unconfirmed");
|
||||
account.down = false;
|
||||
const ended = await post(`/bindings/${alpha.data.id}/disconnect`, {});
|
||||
expect(ended).toMatchObject({ status: 200, data: { state: "disconnected", accountRemoval: "confirmed" } });
|
||||
expect(ended.data.accountRemovedAt).not.toBeNull();
|
||||
expect(account.puts.at(-1)).toEqual(["beta"]);
|
||||
expect(account.names.get("alpha")!.state).toBe("revoked");
|
||||
expect(account.names.get("beta")!.state).toBe("connected");
|
||||
});
|
||||
|
||||
it("a bind that fails here after the account listed the name has the account drop it again", async () => {
|
||||
seat("a@lab");
|
||||
await enroll();
|
||||
// The account answers a plain-http front, which this binding has no --allow-http choice for.
|
||||
account.mcp = "http://account.example/mcp";
|
||||
expect(await connect("a@lab", "alpha")).toMatchObject({ status: 400, data: { error: "http_not_allowed" } });
|
||||
expect(account.puts).toEqual([["alpha"], []]);
|
||||
expect(account.names.get("alpha")!.state).toBe("revoked");
|
||||
});
|
||||
|
||||
it("a participant list the account couldn't apply is retried once, and a second failure says to try again, not that the name is taken", async () => {
|
||||
seat("a@lab");
|
||||
seat("b@lab");
|
||||
await enroll();
|
||||
account.dropFails = 1;
|
||||
expect((await connect("a@lab", "alpha")).status).toBe(201);
|
||||
expect(account.names.get("alpha")).toEqual({ gen: 1, state: "connected" });
|
||||
account.dropFails = 2;
|
||||
expect(await connect("b@lab", "beta")).toMatchObject({ status: 503, data: { error: "account_busy" } });
|
||||
const listed = await (await app.request("/api/network/bindings")).json() as { bindings: Array<{ localSession: string }> };
|
||||
expect(listed.bindings.map((b) => b.localSession)).toEqual(["a@lab"]);
|
||||
});
|
||||
|
||||
it("the installation key goes only to the enrolled site: a redirect isn't followed with it, and a stalled call ends", async () => {
|
||||
seat("a@lab");
|
||||
account.redirectNext = true;
|
||||
expect(await enroll()).toMatchObject({ status: 502, data: { error: "account_redirected" } });
|
||||
expect(account.inits.every((i) => i.redirect === "error" && i.signal instanceof AbortSignal)).toBe(true);
|
||||
expect(account.used.size).toBe(0);
|
||||
await enroll();
|
||||
account.stallNext = true;
|
||||
const stalled = await connect("a@lab", "alpha");
|
||||
expect(stalled).toMatchObject({ status: 503, data: { error: "account_unavailable" } });
|
||||
expect(account.inits.every((i) => i.redirect === "error")).toBe(true);
|
||||
});
|
||||
|
||||
it("concurrent account writes keep every active name: two connects started together, and a disconnect racing a connect", async () => {
|
||||
seat("a@lab"); seat("b@lab"); seat("c@lab");
|
||||
await enroll();
|
||||
account.delayMs = 20;
|
||||
const [alpha, beta] = await Promise.all([connect("a@lab", "alpha"), connect("b@lab", "beta")]);
|
||||
expect([alpha.status, beta.status]).toEqual([201, 201]);
|
||||
expect(account.names.get("alpha")!.state).toBe("connected");
|
||||
expect(account.names.get("beta")!.state).toBe("connected");
|
||||
const [dropped, gamma] = await Promise.all([post(`/bindings/${alpha.data.id}/disconnect`, {}), connect("c@lab", "gamma")]);
|
||||
expect(dropped).toMatchObject({ status: 200, data: { accountRemoval: "confirmed" } });
|
||||
expect(gamma.status).toBe(201);
|
||||
expect(account.names.get("alpha")!.state).toBe("revoked");
|
||||
expect(account.names.get("beta")!.state).toBe("connected");
|
||||
expect(account.names.get("gamma")!.state).toBe("connected");
|
||||
expect(readNetworkCredential(home, gamma.data.id)).toBe(`bear-gamma-1`);
|
||||
});
|
||||
|
||||
it("enroll --replace repairs an installation the account forgot: the old key works until the new code is accepted, and old bindings are ended and named", async () => {
|
||||
seat("a@lab"); seat("b@lab");
|
||||
const first = await enroll();
|
||||
const alpha = await connect("a@lab", "alpha");
|
||||
account.forget();
|
||||
const refused = await connect("b@lab", "beta");
|
||||
expect(refused).toMatchObject({ status: 401, data: { error: "not_enrolled" } });
|
||||
expect(refused.data.message).toContain("rig network enroll <code> --site https://account.example/ --replace");
|
||||
expect(await enroll()).toMatchObject({ status: 409, data: { error: "already_enrolled" } });
|
||||
|
||||
// A refused new code changes nothing.
|
||||
const keyFile = networkInstallationKeyFile(home, first.data.id);
|
||||
const oldKey = readPrivateToken(keyFile, () => "missing");
|
||||
expect(await post("/installations", { code: "ec1.spent", site: "https://account.example", replace: true })).toMatchObject({ status: 400 });
|
||||
expect(readPrivateToken(keyFile, () => "missing")).toBe(oldKey);
|
||||
|
||||
const replaced = await post("/installations", { code: "ec1.next", site: "https://account.example", replace: true });
|
||||
expect(replaced).toMatchObject({ status: 201, data: { id: first.data.id, installationId: "inst-2",
|
||||
replaced: { endedBindings: [{ id: alpha.data.id, localSession: "a@lab", participantName: "alpha" }] } } });
|
||||
expect(readPrivateToken(keyFile, () => "missing")).toBe(account.key);
|
||||
const status = await (await app.request("/api/network/bindings")).json() as { bindings: Array<{ id: string; state: string; accountRemoval: string | null }> };
|
||||
expect(status.bindings.find((b) => b.id === alpha.data.id)).toMatchObject({ state: "disconnected", accountRemoval: "old-installation" });
|
||||
expect((await connect("b@lab", "beta")).status).toBe(201);
|
||||
});
|
||||
|
||||
it("a lost installation key is repaired the same way, and a failed replace keeps the old enrollment working", async () => {
|
||||
seat("a@lab"); seat("b@lab");
|
||||
const first = await enroll();
|
||||
await connect("a@lab", "alpha");
|
||||
// A refused replace leaves the old key in use.
|
||||
expect(await post("/installations", { code: "ec1.spent", site: "https://account.example", replace: true })).toMatchObject({ status: 400 });
|
||||
expect((await connect("b@lab", "beta")).status).toBe(201);
|
||||
|
||||
rmSync(join(networkInstallationKeyFile(home, first.data.id), ".."), { recursive: true });
|
||||
seat("c@lab");
|
||||
const missing = await connect("c@lab", "gamma");
|
||||
expect(missing).toMatchObject({ status: 404, data: { error: "credential_missing" } });
|
||||
expect(missing.data.message).toContain("--replace");
|
||||
expect((await post("/installations", { code: "ec1.next", site: "https://account.example", replace: true })).status).toBe(201);
|
||||
expect((await connect("c@lab", "gamma")).status).toBe(201);
|
||||
});
|
||||
|
||||
it("a replace that fails here after the account accepted the new code puts the old key back", async () => {
|
||||
const first = await enroll();
|
||||
const keyFile = networkInstallationKeyFile(home, first.data.id);
|
||||
const oldKey = readPrivateToken(keyFile, () => "missing");
|
||||
db.exec("CREATE TRIGGER fail_replace BEFORE UPDATE ON network_installations BEGIN SELECT RAISE(ABORT, 'disk I/O error'); END");
|
||||
const res = await app.request("/api/network/installations", { method: "POST", headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ code: "ec1.next", site: "https://account.example", replace: true }) });
|
||||
expect(res.status).toBe(500);
|
||||
expect(account.used.has("ec1.next")).toBe(true);
|
||||
expect(readPrivateToken(keyFile, () => "missing")).toBe(oldKey);
|
||||
expect(readdirSync(dirname(keyFile))).toEqual(["key.json"]);
|
||||
const status = await (await app.request("/api/network/bindings")).json() as { installations: Array<{ installationId: string }> };
|
||||
expect(status.installations.map((i) => i.installationId)).toEqual(["inst-1"]);
|
||||
});
|
||||
|
||||
it("an unresolved host id reads as try again, not enroll again, and sends nothing", async () => {
|
||||
seat("a@lab");
|
||||
await enroll();
|
||||
hostId = null;
|
||||
const calls = account.calls;
|
||||
expect(await connect("a@lab", "alpha")).toMatchObject({ status: 503, data: { error: "instance_unknown" } });
|
||||
expect(account.calls).toBe(calls);
|
||||
});
|
||||
|
||||
it("a failed bind whose drop at the account also fails says the account may still list the name", async () => {
|
||||
seat("a@lab");
|
||||
await enroll();
|
||||
account.mcp = "http://account.example/mcp";
|
||||
account.downAfterPuts = 1;
|
||||
const refused = await connect("a@lab", "alpha");
|
||||
expect(refused).toMatchObject({ status: 400, data: { error: "http_not_allowed" } });
|
||||
expect(refused.data.message).toContain("OpenRig couldn't confirm the account dropped alpha again");
|
||||
expect(account.names.get("alpha")!.state).toBe("connected");
|
||||
});
|
||||
|
||||
it("an unconfirmed removal is confirmed by a later answered list that no longer has the name", async () => {
|
||||
seat("a@lab"); seat("b@lab");
|
||||
await enroll();
|
||||
const alpha = await connect("a@lab", "alpha");
|
||||
account.down = true;
|
||||
expect((await post(`/bindings/${alpha.data.id}/disconnect`, {})).data.error).toBe("account_update_unconfirmed");
|
||||
account.down = false;
|
||||
expect((await connect("b@lab", "beta")).status).toBe(201);
|
||||
const status = await (await app.request("/api/network/bindings")).json() as { bindings: Array<{ id: string; accountRemoval: string | null }> };
|
||||
expect(status.bindings.find((b) => b.id === alpha.data.id)!.accountRemoval).toBe("confirmed");
|
||||
});
|
||||
|
||||
it("a name-only connect without an enrollment says how to enroll", async () => {
|
||||
seat("a@lab");
|
||||
const refused = await connect("a@lab", "alpha");
|
||||
expect(refused).toMatchObject({ status: 404, data: { error: "not_enrolled" } });
|
||||
expect(refused.data.message).toContain("rig network enroll <code> --site <site>");
|
||||
expect(account.calls).toBe(0);
|
||||
});
|
||||
|
||||
it("an unconfirmed drop from before a replace stays with the old installation: the new installation's lists never confirm it", async () => {
|
||||
seat("a@lab"); seat("b@lab");
|
||||
await enroll();
|
||||
const alpha = await connect("a@lab", "alpha");
|
||||
account.down = true;
|
||||
expect(await post(`/bindings/${alpha.data.id}/disconnect`, {})).toMatchObject({ status: 502, data: { error: "account_update_unconfirmed" } });
|
||||
account.down = false;
|
||||
expect(await replace()).toMatchObject({ status: 201, data: { installationId: "inst-2", replaced: { endedBindings: [] } } });
|
||||
const alphaNow = async () => (await status()).bindings.find((b) => b.id === alpha.data.id);
|
||||
expect(await alphaNow()).toMatchObject({ accountRemoval: "old-installation", accountRemovedAt: null });
|
||||
|
||||
// The new installation's list doesn't have alpha, which was never listed there.
|
||||
expect((await connect("b@lab", "beta")).status).toBe(201);
|
||||
expect(account.puts.at(-1)).toEqual(["beta"]);
|
||||
expect(await alphaNow()).toMatchObject({ accountRemoval: "old-installation", accountRemovedAt: null });
|
||||
// Disconnecting it again sends nothing this key could use and claims no drop.
|
||||
const puts = account.puts.length;
|
||||
expect(await post(`/bindings/${alpha.data.id}/disconnect`, {})).toMatchObject({ status: 200, data: { accountRemoval: "old-installation", accountRemovedAt: null } });
|
||||
expect(account.puts.length).toBe(puts);
|
||||
});
|
||||
|
||||
it("a plain local failure to save the binding, with the account's drop failing too, still says the account may list the name", async () => {
|
||||
seat("a@lab");
|
||||
await enroll();
|
||||
db.exec("CREATE TRIGGER fail_bind BEFORE INSERT ON network_bindings BEGIN SELECT RAISE(ABORT, 'disk I/O error'); END");
|
||||
account.downAfterPuts = 1;
|
||||
const failed = await connect("a@lab", "alpha");
|
||||
expect(failed).toMatchObject({ status: 500, data: { error: "binding_write_failed" } });
|
||||
expect(failed.data.message).toContain("disk I/O error");
|
||||
expect(failed.data.message).toContain("couldn't confirm the account dropped alpha again");
|
||||
expect(JSON.stringify(failed.data)).not.toContain("bear-alpha");
|
||||
expect(account.names.get("alpha")!.state).toBe("connected");
|
||||
});
|
||||
|
||||
it("a replace whose wait for the installation's turn uses up its deadline sends no code and changes nothing", async () => {
|
||||
seat("a@lab");
|
||||
const first = await enroll();
|
||||
const keyFile = networkInstallationKeyFile(home, first.data.id);
|
||||
const oldKey = readPrivateToken(keyFile, () => "missing");
|
||||
// A list that overruns holds the installation's turn past the replace's 1 s deadline.
|
||||
account.delayMs = 1800;
|
||||
const slow = connect("a@lab", "alpha");
|
||||
await sleep(50);
|
||||
const late = await replace();
|
||||
expect(late).toMatchObject({ status: 503, data: { error: "account_unavailable" } });
|
||||
expect(late.data.message).toContain("The code wasn't sent");
|
||||
expect(account.used.has("ec1.next")).toBe(false);
|
||||
expect(readPrivateToken(keyFile, () => "missing")).toBe(oldKey);
|
||||
await slow;
|
||||
});
|
||||
|
||||
it("a replace that fails here with only the key file missing leaves no key behind, as before it, and says what the account now has", async () => {
|
||||
seat("a@lab");
|
||||
const first = await enroll();
|
||||
const keyFile = networkInstallationKeyFile(home, first.data.id);
|
||||
rmSync(keyFile);
|
||||
db.exec("CREATE TRIGGER fail_replace BEFORE UPDATE ON network_installations BEGIN SELECT RAISE(ABORT, 'disk I/O error'); END");
|
||||
const failed = await replace();
|
||||
expect(failed).toMatchObject({ status: 500, data: { error: "replace_failed" } });
|
||||
expect(failed.data.message).toContain("nothing here changed");
|
||||
expect(failed.data.message).toContain("a new installation whose key nothing here holds");
|
||||
expect(readdirSync(dirname(keyFile))).toEqual([]);
|
||||
expect((await status()).installations.map((i) => i.installationId)).toEqual(["inst-1"]);
|
||||
db.exec("DROP TRIGGER fail_replace");
|
||||
expect(await connect("a@lab", "alpha")).toMatchObject({ status: 404, data: { error: "credential_missing" } });
|
||||
});
|
||||
|
||||
it("a replace whose old binding can't be ended changes nothing here: the switch and the bindings' end commit together", async () => {
|
||||
seat("a@lab");
|
||||
const first = await enroll();
|
||||
const alpha = await connect("a@lab", "alpha");
|
||||
const keyFile = networkInstallationKeyFile(home, first.data.id);
|
||||
const oldKey = readPrivateToken(keyFile, () => "missing");
|
||||
db.exec("CREATE TRIGGER fail_end BEFORE UPDATE OF state ON network_bindings BEGIN SELECT RAISE(ABORT, 'disk I/O error'); END");
|
||||
expect(await replace()).toMatchObject({ status: 500, data: { error: "replace_failed" } });
|
||||
expect(readPrivateToken(keyFile, () => "missing")).toBe(oldKey);
|
||||
expect(readdirSync(dirname(keyFile))).toEqual(["key.json"]);
|
||||
const now = await status();
|
||||
expect(now.installations.map((i) => i.installationId)).toEqual(["inst-1"]);
|
||||
expect(now.bindings.find((b) => b.id === alpha.data.id)).toMatchObject({ state: "active" });
|
||||
});
|
||||
|
||||
it("a replace whose old key's backup can't be removed after the switch still ends the old bindings, and reports the cleanup", async () => {
|
||||
seat("a@lab");
|
||||
const first = await enroll();
|
||||
const alpha = await connect("a@lab", "alpha");
|
||||
failRemoval.suffix = "key.json.prev";
|
||||
const replaced = await replace();
|
||||
failRemoval.suffix = null;
|
||||
expect(replaced).toMatchObject({ status: 201, data: { installationId: "inst-2", replaced: { endedBindings: [{ id: alpha.data.id, participantName: "alpha" }] } } });
|
||||
expect(replaced.data.replaced.problems).toEqual([expect.stringMatching(/backup at .*key\.json\.prev couldn't be removed \(EBUSY/)]);
|
||||
expect(readPrivateToken(networkInstallationKeyFile(home, first.data.id), () => "missing")).toBe(account.key);
|
||||
expect((await status()).bindings.find((b) => b.id === alpha.data.id)).toMatchObject({ state: "disconnected", accountRemoval: "old-installation" });
|
||||
});
|
||||
@@ -275,6 +275,56 @@ describe("S01 wake-or-escalate — retry ladder, named rungs, derived suspension
|
||||
expect(calls.some((c) => c.target === "worker@r")).toBe(false);
|
||||
});
|
||||
|
||||
it("UNKNOWN READS UNKNOWN: a baton whose recorded wake result no reader recognises escalates past the window, never re-sent, and the notice says the outcome is unknown", async () => {
|
||||
const baton = await mkBaton();
|
||||
setNudgeResult(baton.qitemId, "accepted-by-some-new-route", 45);
|
||||
ageCreated(baton.qitemId, 45);
|
||||
await tick();
|
||||
expect(markersOf(baton.qitemId, LADDER_ATTEMPT_PREFIX)).toHaveLength(0);
|
||||
const rungs = markersOf(baton.qitemId, LADDER_RUNG_PREFIX);
|
||||
expect(rungs).toHaveLength(1);
|
||||
expect(rungs[0]).toMatch(/wake outcome unknown \(unrecognised result "accepted-by-some-new-route"\)/);
|
||||
expect(calls.some((c) => c.target === "worker@r")).toBe(false);
|
||||
});
|
||||
|
||||
it("UNKNOWN READS UNKNOWN (negative controls): inside the window, or once claimed, an unrecognised result gets nothing", async () => {
|
||||
const fresh = await mkBaton();
|
||||
setNudgeResult(fresh.qitemId, "accepted-by-some-new-route", 5);
|
||||
const claimed = await mkBaton("other@r");
|
||||
setNudgeResult(claimed.qitemId, "accepted-by-some-new-route", 45);
|
||||
ageCreated(claimed.qitemId, 45);
|
||||
repo.claim({ qitemId: claimed.qitemId, destinationSession: "other@r" });
|
||||
await tick();
|
||||
expect(calls).toHaveLength(0);
|
||||
expect(markersOf(fresh.qitemId, LADDER_RUNG_PREFIX)).toHaveLength(0);
|
||||
expect(markersOf(claimed.qitemId, LADDER_RUNG_PREFIX)).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("KNOWN OUTCOMES KEEP THEIR PATH: a verified wake, a hosted trigger's accepted ring and an unroutable address never enter the ladder", async () => {
|
||||
const batons = [];
|
||||
for (const [idx, result] of ["verified", "wake-accepted:webhook (hook) answered 202; pickup is the member's claim on this row",
|
||||
"unroutable: 'w2@r' has no terminal transport on this daemon"].entries()) {
|
||||
const baton = await mkBaton(`w${idx}@r`);
|
||||
setNudgeResult(baton.qitemId, result, 45);
|
||||
ageCreated(baton.qitemId, 45);
|
||||
batons.push(baton);
|
||||
}
|
||||
await tick();
|
||||
expect(calls).toHaveLength(0);
|
||||
for (const baton of batons) {
|
||||
expect(markersOf(baton.qitemId, LADDER_ATTEMPT_PREFIX)).toHaveLength(0);
|
||||
expect(markersOf(baton.qitemId, LADDER_RUNG_PREFIX)).toHaveLength(0);
|
||||
}
|
||||
});
|
||||
|
||||
it("A WEBHOOK FAILURE AFTER THE POST KEEPS THE FAILED PATH: its effect is unknown, but it is retried as it always was", async () => {
|
||||
const baton = await mkBaton();
|
||||
setNudgeResult(baton.qitemId, "failed:webhook (hook) answered 500", 10);
|
||||
await tick();
|
||||
expect(markersOf(baton.qitemId, LADDER_ATTEMPT_PREFIX)).toHaveLength(1);
|
||||
expect(calls).toEqual([{ qitemId: baton.qitemId, target: "worker@r" }]);
|
||||
});
|
||||
|
||||
it("DELIVERY MACHINERY IS NOT PICKUP: a relay-owner-notification-transport-failed note on an unconfirmed or held baton does not count as pickup evidence", async () => {
|
||||
const mod = await ladderMod();
|
||||
for (const [idx, result] of ["gateway-owned: virtual destination", "retained:typing_guard"].entries()) {
|
||||
|
||||
@@ -13,6 +13,7 @@ import type Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { coreSchema } from "../src/db/migrations/001_core_schema.js";
|
||||
import { rigArchiveSchema } from "../src/db/migrations/042_rig_archive.js";
|
||||
import { bindingsSessionsSchema } from "../src/db/migrations/002_bindings_sessions.js";
|
||||
import { eventsSchema } from "../src/db/migrations/003_events.js";
|
||||
import { streamItemsSchema } from "../src/db/migrations/023_stream_items.js";
|
||||
@@ -70,7 +71,7 @@ describe("GET /api/review/*", () => {
|
||||
db = createDb(":memory:");
|
||||
migrate(db, [
|
||||
coreSchema,
|
||||
bindingsSessionsSchema,
|
||||
bindingsSessionsSchema, rigArchiveSchema,
|
||||
eventsSchema,
|
||||
streamItemsSchema,
|
||||
queueItemsSchema,
|
||||
@@ -335,7 +336,7 @@ describe("GET /api/review/rig (OPR.0.4.4.22)", () => {
|
||||
db = createDb(":memory:");
|
||||
migrate(db, [
|
||||
coreSchema,
|
||||
bindingsSessionsSchema,
|
||||
bindingsSessionsSchema, rigArchiveSchema,
|
||||
eventsSchema,
|
||||
streamItemsSchema,
|
||||
queueItemsSchema,
|
||||
@@ -668,7 +669,7 @@ describe("qitem-ccf87c0d — composeMission MEMBERSHIP-scan load contract", () =
|
||||
it("one composeMission over 40 indexed slices executes <= 4 membership queue scans AND composes all 40 ledger entries", () => {
|
||||
const db = createDb();
|
||||
migrate(db, [
|
||||
coreSchema, bindingsSessionsSchema, eventsSchema, streamItemsSchema,
|
||||
coreSchema, bindingsSessionsSchema, rigArchiveSchema, eventsSchema, streamItemsSchema,
|
||||
queueItemsSchema, queueTransitionsSchema, missionControlActionsSchema,
|
||||
queueItemSummarySchema,
|
||||
]);
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import type Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { ALL_MIGRATIONS } from "../src/db/all-migrations.js";
|
||||
import { resolveCurrentSessionAddress, SessionRegistry } from "../src/domain/session-registry.js";
|
||||
import { getNodeInventory } from "../src/domain/node-inventory.js";
|
||||
|
||||
const databases: Database.Database[] = [];
|
||||
afterEach(() => { for (const db of databases.splice(0)) db.close(); });
|
||||
|
||||
function fixture(indexed = true) {
|
||||
const db = createDb(); databases.push(db);
|
||||
migrate(db, ALL_MIGRATIONS.filter(m => indexed || m.name !== "162_session_address_read_indexes.sql"));
|
||||
db.exec("INSERT INTO rigs(id,name) VALUES('r','team')");
|
||||
const node = db.prepare("INSERT INTO nodes(id,rig_id,logical_id,runtime) VALUES(?,'r',?,'claude-code')");
|
||||
const session = db.prepare("INSERT INTO sessions(id,node_id,session_name,status,created_at) VALUES(?,?,?,'running',?)");
|
||||
const binding = db.prepare("INSERT INTO bindings(id,node_id,tmux_session,attachment_type) VALUES(?,?,?,'tmux')");
|
||||
for (let n = 0; n < 4; n++) {
|
||||
node.run(`n${n}`, `dev.worker${n}`);
|
||||
for (let i = 0; i < 20; i++) session.run(`${n}-${String(i).padStart(3, '0')}`, `n${n}`, `native-${n}@team`, "2099-01-01");
|
||||
// Session id defines recency even when the wall clock moves backwards.
|
||||
session.run(`${n}-999`, `n${n}`, `native-${n}@team`, "2026-01-01");
|
||||
binding.run(`b${n}`, `n${n}`, `native-${n}@team`);
|
||||
}
|
||||
return db;
|
||||
}
|
||||
|
||||
describe("current session address read cost", () => {
|
||||
it("seeks a physical name and the latest registration instead of scanning session history", () => {
|
||||
const db = fixture(), sql: string[] = [], prepare = db.prepare;
|
||||
db.prepare = function (text: string) { sql.push(text); return prepare.call(this, text); } as typeof db.prepare;
|
||||
try {
|
||||
expect(resolveCurrentSessionAddress(db, "native-0@team")?.nodeId).toBe("n0");
|
||||
expect(getNodeInventory(db, "r").map(n => n.canonicalSessionName)).toContain("native-0@team");
|
||||
} finally { db.prepare = prepare; }
|
||||
const address = sql.find(s => s.includes("AS senderAddress"))!;
|
||||
const plan = db.prepare(`EXPLAIN QUERY PLAN ${address}`).all("native-0@team", "native-0@team") as { detail: string }[];
|
||||
expect(plan.map(p => p.detail).join("\n")).toMatch(/SEARCH b USING INDEX idx_bindings_tmux_session/);
|
||||
expect(plan.map(p => p.detail).join("\n")).not.toMatch(/SCAN s\b/);
|
||||
const latest = db.prepare("EXPLAIN QUERY PLAN SELECT id FROM sessions WHERE node_id=? ORDER BY id DESC LIMIT 1").all("n0") as { detail: string }[];
|
||||
expect(latest.map(p => p.detail).join("\n")).not.toContain("TEMP B-TREE");
|
||||
});
|
||||
|
||||
it("keeps exact-name priority, derived aliases, and fresh binding changes", () => {
|
||||
const db = fixture();
|
||||
expect(resolveCurrentSessionAddress(db, "dev-worker0@team")?.sessionName).toBe("native-0@team");
|
||||
// A physical spelling takes priority over another seat's derived spelling.
|
||||
db.prepare("UPDATE sessions SET session_name='dev-worker0@team' WHERE id='1-999'").run();
|
||||
db.prepare("UPDATE bindings SET tmux_session='dev-worker0@team' WHERE node_id='n1'").run();
|
||||
expect(resolveCurrentSessionAddress(db, "dev-worker0@team")?.nodeId).toBe("n1");
|
||||
db.prepare("UPDATE sessions SET status='exited' WHERE id='1-999'").run();
|
||||
// That other seat's saved spelling still reserves the name.
|
||||
expect(resolveCurrentSessionAddress(db, "dev-worker0@team")).toBeNull();
|
||||
db.prepare("UPDATE rigs SET archived_at='2026-01-02' WHERE id='r'").run();
|
||||
expect(resolveCurrentSessionAddress(db, "native-0@team")).toBeNull();
|
||||
});
|
||||
|
||||
it("seeks the latest named registration when the generation reader resolves a session", () => {
|
||||
const db = fixture(), sql: string[] = [], prepare = db.prepare;
|
||||
db.prepare = function (text: string) { sql.push(text); return prepare.call(this, text); } as typeof db.prepare;
|
||||
try { new SessionRegistry(db).currentOccupantGenerationForSession("native-0@team"); }
|
||||
finally { db.prepare = prepare; }
|
||||
const read = sql.find(s => s.includes("SELECT node_id FROM sessions WHERE session_name"))!;
|
||||
expect(read).toBeDefined();
|
||||
const plan = db.prepare(`EXPLAIN QUERY PLAN ${read}`).all("native-0@team") as { detail: string }[];
|
||||
expect(plan.map(p => p.detail).join("\n")).toContain("SEARCH sessions USING INDEX idx_sessions_name_latest_id");
|
||||
expect(plan.map(p => p.detail).join("\n")).not.toContain("TEMP B-TREE");
|
||||
});
|
||||
|
||||
it("upgrades populated databases without changing registrations or inventory", () => {
|
||||
const db = fixture(false);
|
||||
const before = { inventory: getNodeInventory(db, "r"),
|
||||
physical: resolveCurrentSessionAddress(db, "native-0@team"),
|
||||
alias: resolveCurrentSessionAddress(db, "dev-worker0@team"),
|
||||
registrations: db.prepare("SELECT * FROM sessions ORDER BY id").all() };
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
expect({ inventory: getNodeInventory(db, "r"),
|
||||
physical: resolveCurrentSessionAddress(db, "native-0@team"),
|
||||
alias: resolveCurrentSessionAddress(db, "dev-worker0@team"),
|
||||
registrations: db.prepare("SELECT * FROM sessions ORDER BY id").all() }).toEqual(before);
|
||||
expect(db.pragma("integrity_check", { simple: true })).toBe("ok");
|
||||
expect(db.pragma("foreign_key_check")).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -370,6 +370,23 @@ describe("SessionRegistry", () => {
|
||||
expect(names).toContain("r01-dev1-qa");
|
||||
});
|
||||
|
||||
it("getSessionsForRig preserves insertion order for timestamp ties with the session indexes installed", () => {
|
||||
const indexes = db.prepare("PRAGMA index_list(sessions)").all() as { name: string }[];
|
||||
expect(indexes.map((index) => index.name)).toContain("idx_sessions_node_latest_id");
|
||||
const insert = db.prepare("INSERT INTO sessions(id, node_id, session_name, created_at) VALUES (?, ?, ?, ?)");
|
||||
// Non-monotonic ids distinguish insertion order from either id ordering.
|
||||
insert.run("m-first", "node-1", "first@rig", "2026-01-02");
|
||||
insert.run("z-second", "node-1", "second@rig", "2026-01-02");
|
||||
insert.run("a-third", "node-1", "third@rig", "2026-01-02");
|
||||
insert.run("n-fourth", "node-2", "fourth@rig", "2026-01-02");
|
||||
// Timestamp remains the primary order even when this row was inserted last.
|
||||
insert.run("earlier", "node-2", "earlier@rig", "2026-01-01");
|
||||
|
||||
expect(registry.getSessionsForRig("rig-1").map((session) => session.id)).toEqual([
|
||||
"earlier", "m-first", "z-second", "a-third", "n-fourth",
|
||||
]);
|
||||
});
|
||||
|
||||
it("getBindingForNode returns null when unbound", () => {
|
||||
const binding = registry.getBindingForNode("node-1");
|
||||
expect(binding).toBeNull();
|
||||
|
||||
@@ -7,6 +7,7 @@ import type Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { coreSchema } from "../src/db/migrations/001_core_schema.js";
|
||||
import { rigArchiveSchema } from "../src/db/migrations/042_rig_archive.js";
|
||||
import { outboxEntriesSchema } from "../src/db/migrations/027_outbox_entries.js";
|
||||
import { bindingsSessionsSchema } from "../src/db/migrations/002_bindings_sessions.js";
|
||||
import { eventsSchema } from "../src/db/migrations/003_events.js";
|
||||
@@ -133,7 +134,7 @@ describe("workflow route (WF3 FR-4 — close+recreate+rebind)", () => {
|
||||
db = createDb();
|
||||
migrate(db, [
|
||||
outboxEntriesSchema,
|
||||
coreSchema, bindingsSessionsSchema, eventsSchema,
|
||||
coreSchema, bindingsSessionsSchema, rigArchiveSchema, eventsSchema,
|
||||
queueItemsSchema, queueTransitionsSchema,
|
||||
queueItemSummarySchema, queueItemEvidenceRefSchema,
|
||||
workflowSpecsSchema, workflowInstancesSchema, workflowStepTrailsSchema,
|
||||
|
||||
@@ -7,6 +7,7 @@ import type Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { coreSchema } from "../src/db/migrations/001_core_schema.js";
|
||||
import { rigArchiveSchema } from "../src/db/migrations/042_rig_archive.js";
|
||||
import { outboxEntriesSchema } from "../src/db/migrations/027_outbox_entries.js";
|
||||
import { eventsSchema } from "../src/db/migrations/003_events.js";
|
||||
import { queueItemsSchema } from "../src/db/migrations/024_queue_items.js";
|
||||
@@ -582,7 +583,7 @@ describe("workflow routes — WF-2 structured error boundaries", () => {
|
||||
migrate(db, [
|
||||
outboxEntriesSchema,
|
||||
coreSchema,
|
||||
bindingsSessionsSchema,
|
||||
bindingsSessionsSchema, rigArchiveSchema,
|
||||
eventsSchema,
|
||||
queueItemsSchema,
|
||||
queueTransitionsSchema,
|
||||
|
||||
@@ -13,6 +13,7 @@ import type Database from "better-sqlite3";
|
||||
import { createDb } from "../src/db/connection.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { coreSchema } from "../src/db/migrations/001_core_schema.js";
|
||||
import { rigArchiveSchema } from "../src/db/migrations/042_rig_archive.js";
|
||||
import { outboxEntriesSchema } from "../src/db/migrations/027_outbox_entries.js";
|
||||
import { bindingsSessionsSchema } from "../src/db/migrations/002_bindings_sessions.js";
|
||||
import { eventsSchema } from "../src/db/migrations/003_events.js";
|
||||
@@ -188,7 +189,7 @@ describe("OPR.0.4.6.WF2 — spec language", () => {
|
||||
migrate(db, [
|
||||
outboxEntriesSchema,
|
||||
coreSchema,
|
||||
bindingsSessionsSchema,
|
||||
bindingsSessionsSchema, rigArchiveSchema,
|
||||
eventsSchema,
|
||||
queueItemsSchema,
|
||||
queueTransitionsSchema,
|
||||
|
||||
+49
-30
@@ -48,6 +48,8 @@ interface ContentLine {
|
||||
zones?: Array<{ start: number; end: number; action: Action }>;
|
||||
/** slice-17: token segments for canvas-rendered rows (graph view) */
|
||||
segs?: Array<{ text: string; token?: import("./theme.js").Token; bold?: boolean; bg?: import("./theme.js").Token }>;
|
||||
/** Row of a labelled group. The first visible row of each group shows `text` when set, so a scrolled group keeps its label. */
|
||||
sticky?: { group: string; text?: string };
|
||||
}
|
||||
|
||||
function pad(text: string | number | null | undefined, width: number): string {
|
||||
@@ -94,17 +96,24 @@ function agentColumns(contentWidth: number): AgentColumn[] {
|
||||
];
|
||||
}
|
||||
|
||||
function instanceAgentColumns(contentWidth: number): AgentColumn[] {
|
||||
if (contentWidth >= 110) return [
|
||||
["rig", "RIG", 13, "left"], ["pod", "POD", 8, "left"], ["seat", "SEAT", 14, "left"],
|
||||
["runtime", "RT", 3, "left"], ["context", "CTX", 7, "right"], ["status", "STATE", 10, "left"],
|
||||
["queue", "Q", 2, "right"], ["work", "WORK", 12, "left"], ["now", "NOW", Math.max(18, contentWidth - 78), "left"],
|
||||
];
|
||||
if (contentWidth >= 78) return [
|
||||
["rig", "RIG", 10, "left"], ["pod", "POD", 6, "left"], ["seat", "SEAT", 12, "left"],
|
||||
["runtime", "RT", 3, "left"], ["context", "CTX", 5, "right"], ["status", "STATE", 9, "left"],
|
||||
["queue", "Q", 2, "right"], ["work", "WORK", Math.max(8, contentWidth - 54), "left"],
|
||||
];
|
||||
/** RIG fits the longest rig name shown, between the tier's old width and its cap; NOW or WORK gives up the difference. */
|
||||
function instanceAgentColumns(contentWidth: number, longestRigName = 0): AgentColumn[] {
|
||||
if (contentWidth >= 110) {
|
||||
const rig = Math.min(Math.max(longestRigName, 13), 24);
|
||||
return [
|
||||
["rig", "RIG", rig, "left"], ["pod", "POD", 8, "left"], ["seat", "SEAT", 14, "left"],
|
||||
["runtime", "RT", 3, "left"], ["context", "CTX", 7, "right"], ["status", "STATE", 10, "left"],
|
||||
["queue", "Q", 2, "right"], ["work", "WORK", 12, "left"], ["now", "NOW", Math.max(18, contentWidth - 78 - (rig - 13)), "left"],
|
||||
];
|
||||
}
|
||||
if (contentWidth >= 78) {
|
||||
const rig = Math.min(Math.max(longestRigName, 10), 16);
|
||||
return [
|
||||
["rig", "RIG", rig, "left"], ["pod", "POD", 6, "left"], ["seat", "SEAT", 12, "left"],
|
||||
["runtime", "RT", 3, "left"], ["context", "CTX", 5, "right"], ["status", "STATE", 9, "left"],
|
||||
["queue", "Q", 2, "right"], ["work", "WORK", Math.max(8, contentWidth - 54 - (rig - 10)), "left"],
|
||||
];
|
||||
}
|
||||
const fixed = 8 + 5 + 11 + 2 + 5 + 8 + 2 + 7;
|
||||
return [
|
||||
["rig", "RIG", 8, "left"], ["pod", "POD", 5, "left"], ["seat", "SEAT", 11, "left"],
|
||||
@@ -709,7 +718,7 @@ function noRigsText(snap: FleetSnapshot, host: FleetSnapshot["hosts"][number], l
|
||||
: ` (no rigs served by ${host.name})`;
|
||||
}
|
||||
|
||||
/** Rig boundaries take a solid rule and pod boundaries a dotted one; RIG repeats on every row so scrolled rows keep it. */
|
||||
/** Rig boundaries take a solid rule and pod boundaries a dotted one; RIG shows on a rig's first visible row so scrolled rows keep it. */
|
||||
function instanceContentLines(
|
||||
state: ViewState,
|
||||
snap: FleetSnapshot,
|
||||
@@ -801,7 +810,12 @@ function instanceContentLines(
|
||||
lines.push(healthSummaryLine(snap, scope, contentWidth));
|
||||
lines.push(...queueSampleLines(snap, contentWidth));
|
||||
lines.push({ text: state.filter ? `/ filter instance rows: ${state.filter} · / replace · esc clear` : "/ filter instance rows…" });
|
||||
const columns = instanceAgentColumns(contentWidth);
|
||||
const shownRigs = host.rigs.map((rig) => ({
|
||||
rig,
|
||||
agents: rig.pods.flatMap((pod) => pod.agents.map((agent) => ({ pod: pod.name, agent })))
|
||||
.filter(({ pod, agent }) => !state.filter || rig.name.includes(state.filter) || pod.includes(state.filter) || agent.name.includes(state.filter)),
|
||||
})).filter(({ rig, agents }) => !state.filter || agents.length > 0 || rig.name.includes(state.filter));
|
||||
const columns = instanceAgentColumns(contentWidth, Math.max(0, ...shownRigs.map(({ rig }) => rig.name.length)));
|
||||
lines.push({ text: tableRow(columns, { rig: "RIG", pod: "POD", seat: "SEAT", runtime: "RT", context: "CTX", status: "STATE", queue: "Q", work: "WORK", now: "NOW" }) });
|
||||
lines.push({ text: "━".repeat(columnsWidth(columns)) });
|
||||
let seatCount = 0;
|
||||
@@ -810,10 +824,7 @@ function instanceContentLines(
|
||||
let attentionUnknown = 0;
|
||||
let openCount = 0;
|
||||
let firstRig = true;
|
||||
for (const rig of host.rigs) {
|
||||
const agents = rig.pods.flatMap((pod) => pod.agents.map((agent) => ({ pod: pod.name, agent })))
|
||||
.filter(({ pod, agent }) => !state.filter || rig.name.includes(state.filter) || pod.includes(state.filter) || agent.name.includes(state.filter));
|
||||
if (state.filter && agents.length === 0 && !rig.name.includes(state.filter)) continue;
|
||||
for (const { rig, agents } of shownRigs) {
|
||||
if (!firstRig) lines.push({ text: "─".repeat(columnsWidth(columns)) });
|
||||
firstRig = false;
|
||||
const rigAction: Action = { type: "drill", resource: "rig", name: rig.name, target: { host: host.name } };
|
||||
@@ -825,7 +836,7 @@ function instanceContentLines(
|
||||
continue;
|
||||
}
|
||||
let previousPod: string | null = null;
|
||||
for (const { pod, agent } of agents) {
|
||||
for (const [index, { pod, agent }] of agents.entries()) {
|
||||
const firstInPod = pod !== previousPod;
|
||||
if (firstInPod && previousPod != null) lines.push({ text: "┈".repeat(columnsWidth(columns)) });
|
||||
previousPod = pod;
|
||||
@@ -836,18 +847,21 @@ function instanceContentLines(
|
||||
if (agent.needsAttention === true) attentionCount += 1;
|
||||
else if (agent.needsAttention == null) attentionUnknown += 1;
|
||||
openCount += queue.count;
|
||||
const cells = {
|
||||
rig: rig.name,
|
||||
pod: firstInPod ? pod : "",
|
||||
seat: `${stateCell.mark} ${seatName(pod, agent.name)}`,
|
||||
runtime: runtimeShort(agent.runtime),
|
||||
context: contextCompact(agent.context, contentWidth < 110),
|
||||
status: stateCell.word,
|
||||
queue: queueCount(snap, queue.count),
|
||||
work: queue.work,
|
||||
now: queue.now,
|
||||
};
|
||||
const named = tableRow(columns, cells);
|
||||
lines.push({
|
||||
text: tableRow(columns, {
|
||||
rig: rig.name,
|
||||
pod: firstInPod ? pod : "",
|
||||
seat: `${stateCell.mark} ${seatName(pod, agent.name)}`,
|
||||
runtime: runtimeShort(agent.runtime),
|
||||
context: contextCompact(agent.context, contentWidth < 110),
|
||||
status: stateCell.word,
|
||||
queue: queueCount(snap, queue.count),
|
||||
work: queue.work,
|
||||
now: queue.now,
|
||||
}),
|
||||
text: index === 0 ? named : tableRow(columns, { ...cells, rig: "" }),
|
||||
sticky: { group: `rig:${rig.name}`, ...(index === 0 ? {} : { text: named }) },
|
||||
action: { type: "drill", resource: "agent", name: agent.name, target: { host: host.name, rig: rig.name, pod } },
|
||||
zones: [{ start: 0, end: columns[0]![2], action: rigAction }],
|
||||
});
|
||||
@@ -1944,7 +1958,12 @@ function renderBody(state: ViewState, snap: FleetSnapshot, options: RenderOption
|
||||
const contentRows = content.length > bodyRows ? Math.max(bodyRows - 1, 0) : bodyRows;
|
||||
const maxContentOffset = Math.max(content.length - contentRows, 0);
|
||||
const contentStart = Math.min(state.contentOffset, maxContentOffset);
|
||||
const visibleContent = content.slice(contentStart, contentStart + contentRows);
|
||||
const labelledGroups = new Set<string>();
|
||||
const visibleContent = content.slice(contentStart, contentStart + contentRows).map((line) => {
|
||||
if (!line.sticky || labelledGroups.has(line.sticky.group)) return line;
|
||||
labelledGroups.add(line.sticky.group);
|
||||
return line.sticky.text == null ? line : { ...line, text: line.sticky.text };
|
||||
});
|
||||
if (content.length > bodyRows) {
|
||||
const scrollText = `scroll ↑/↓ · ${contentStart + 1}-${contentStart + visibleContent.length} of ${content.length}`;
|
||||
const up = scrollText.indexOf("↑");
|
||||
|
||||
@@ -6,6 +6,7 @@ import { decodeInput, resolveKeyAction, sgrClick } from "../src/input.js";
|
||||
import { renderScreen } from "../src/render.js";
|
||||
import { computeExplorerRows, createViewState } from "../src/state.js";
|
||||
import { createStyle, stripAnsi } from "../src/theme.js";
|
||||
import { explorerWidth } from "../src/visual-layout.js";
|
||||
import { stylizeLines } from "../src/stylize.js";
|
||||
import type { AgentRow, FleetSnapshot } from "../src/types.js";
|
||||
|
||||
@@ -118,16 +119,20 @@ describe("S12 instance mission control", () => {
|
||||
let scrolledSeatRows = 0;
|
||||
for (let offset = 0; offset <= screen.contentMaxOffset; offset += 1) {
|
||||
screen = renderScreen(view.get(), snap, { cols, rows, nowMs: 0, colorMode: "none" });
|
||||
const namedRigs = new Set<string>();
|
||||
for (const target of screen.contentTargets) {
|
||||
if (target.action.type !== "drill" || target.action.resource !== "agent") continue;
|
||||
reached.add(target.action.name);
|
||||
const rig = target.action.target?.rig;
|
||||
const content = stripAnsi(screen.lines[target.y - 1] ?? "").slice(screen.explorerWidth + 2);
|
||||
const firstVisible = !!rig && !namedRigs.has(rig);
|
||||
if (rig) namedRigs.add(rig);
|
||||
if (!rig || !/\b(working|needs you|idle|detached|failed|unknown|blocked)\b/.test(content)) continue;
|
||||
const headingVisible = screen.lines.some((line) => stripAnsi(line).slice(screen.explorerWidth + 2).includes(`RIG ${rig} ·`));
|
||||
if (headingVisible) continue;
|
||||
scrolledSeatRows += 1;
|
||||
expect(content.trimStart().startsWith(rig), `${cols}: scrolled seat row retains rig identity`).toBe(true);
|
||||
if (firstVisible) expect(content.startsWith(rig), `${cols}: first visible seat row of a rig retains rig identity`).toBe(true);
|
||||
else expect(content.startsWith(" ".repeat(rig.length)), `${cols}: later seat rows of a rig leave RIG blank`).toBe(true);
|
||||
}
|
||||
view.dispatch({ type: "content-scroll", delta: 1 });
|
||||
}
|
||||
@@ -140,7 +145,7 @@ describe("S12 instance mission control", () => {
|
||||
expect(whole).toMatch(/empty\s+—\s+\(no seats\).*stopped/);
|
||||
expect(whole).toMatch(/dev\s+[\s\S]*┈{8,}[\s\S]*qa\s+/);
|
||||
expect((whole.match(/┈{8,}/g) ?? [])).toHaveLength(1);
|
||||
expect(whole).toMatch(/build\s+qa\s+[^\n]*\n[^\n]*─{8,}[^\n]*\n[^\n]*docs\s+write/);
|
||||
expect(whole).toMatch(/\s+qa\s+[^\n]*\n[^\n]*─{8,}[^\n]*\n[^\n]*docs\s+write/);
|
||||
expect(whole).toMatch(/docs\s+write\s+[^\n]*\n[^\n]*─{8,}[^\n]*\n[^\n]*empty\s+—/);
|
||||
expect(whole).not.toMatch(/\+\d+|more hidden/i);
|
||||
expect(reached).toEqual(expectedAgents);
|
||||
@@ -319,3 +324,138 @@ describe("S12 view-aware hydration", () => {
|
||||
expect(snap.hosts[0]).toMatchObject({ id: "local", name: "local" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("instance table RIG column (#2848)", () => {
|
||||
const LONG = "RIG-session-review";
|
||||
const OVER_CAP = "rig-name-thirty-characters-xx";
|
||||
|
||||
function colsFor(contentWidth: number): number {
|
||||
for (let cols = contentWidth; cols < contentWidth + 80; cols += 1) {
|
||||
if (cols - explorerWidth(cols) - 2 === contentWidth) return cols;
|
||||
}
|
||||
throw new Error(`no terminal width gives content width ${contentWidth}`);
|
||||
}
|
||||
|
||||
function snapshotWithRigs(names: string[]): FleetSnapshot {
|
||||
const snap = multiRigSnapshot();
|
||||
snap.hosts[0]!.rigs.push(...names.map((name) => ({
|
||||
id: `r-${name}`,
|
||||
name,
|
||||
lifecycleState: "running",
|
||||
pods: [{ name: "ops", agents: [agent("ops.worker", `ops-worker@${name}`, "idle", 10)] }],
|
||||
})));
|
||||
return snap;
|
||||
}
|
||||
|
||||
function table(snap: FleetSnapshot, contentWidth: number, filter = "") {
|
||||
const view = openInstance(snap);
|
||||
if (filter) view.dispatch({ type: "filter", text: filter });
|
||||
const cols = colsFor(contentWidth);
|
||||
const screen = renderScreen(view.get(), snap, { cols, rows: 160, nowMs: 0, colorMode: "none" });
|
||||
const content = screen.lines.map((line) => stripAnsi(line).slice(screen.explorerWidth + 2));
|
||||
const header = content.find((line) => /^RIG\s+POD\s+SEAT/.test(line));
|
||||
expect(header, `${contentWidth}: table header`).toBeDefined();
|
||||
const heavyRule = content.find((line) => /^━{20,}/.test(line))!.match(/^━+/)![0].length;
|
||||
const rigWidth = header!.indexOf("POD") - 1;
|
||||
return { screen, content, header: header!, rigWidth, rowWidth: heavyRule };
|
||||
}
|
||||
|
||||
it.each([
|
||||
// [content width, expected RIG width, flexible column, expected flexible width]
|
||||
[77, 8, "WORK", 77 - (8 + 5 + 11 + 2 + 5 + 8 + 2 + 7)],
|
||||
[78, 16, "WORK", 78 - 54 - (16 - 10)],
|
||||
[109, 16, "WORK", 109 - 54 - (16 - 10)],
|
||||
[110, LONG.length, "NOW", 110 - 78 - (LONG.length - 13)],
|
||||
[190, LONG.length, "NOW", 190 - 78 - (LONG.length - 13)],
|
||||
] as const)("fits RIG to the longest shown name at content width %i", (contentWidth, rigWidth, flexible, flexibleWidth) => {
|
||||
const { header, rigWidth: actual, rowWidth } = table(snapshotWithRigs([LONG]), contentWidth);
|
||||
expect(actual).toBe(rigWidth);
|
||||
expect(rowWidth).toBeLessThanOrEqual(contentWidth);
|
||||
expect(header.trimEnd().endsWith(flexible)).toBe(true);
|
||||
expect(rowWidth - header.lastIndexOf(flexible)).toBe(flexibleWidth);
|
||||
});
|
||||
|
||||
it("keeps the old RIG width as the minimum when every name is short", () => {
|
||||
expect(table(multiRigSnapshot(), 160).rigWidth).toBe(13);
|
||||
expect(table(multiRigSnapshot(), 90).rigWidth).toBe(10);
|
||||
});
|
||||
|
||||
it.each([[110, 24], [160, 24], [78, 16], [100, 16]])("caps RIG and truncates a longer name at content width %i", (contentWidth, cap) => {
|
||||
const { content, rigWidth, rowWidth } = table(snapshotWithRigs([OVER_CAP]), contentWidth);
|
||||
expect(rigWidth).toBe(cap);
|
||||
expect(rowWidth).toBeLessThanOrEqual(contentWidth);
|
||||
expect(content.some((line) => line.startsWith(`${OVER_CAP.slice(0, cap - 1)}…`))).toBe(true);
|
||||
for (const line of content) expect(line.trimEnd().length).toBeLessThanOrEqual(contentWidth);
|
||||
});
|
||||
|
||||
function seatRows(screen: ReturnType<typeof renderScreen>) {
|
||||
return screen.contentTargets
|
||||
.filter((target) => target.action.type === "drill" && target.action.resource === "agent")
|
||||
.map((target) => ({
|
||||
y: target.y,
|
||||
rig: target.action.type === "drill" ? target.action.target?.rig ?? "" : "",
|
||||
name: target.action.type === "drill" ? target.action.name : "",
|
||||
text: stripAnsi(screen.lines[target.y - 1] ?? "").slice(screen.explorerWidth + 2),
|
||||
}));
|
||||
}
|
||||
|
||||
it("names every rig on its single row when each rig has one seat", () => {
|
||||
const names = Array.from({ length: 8 }, (_, index) => `solo-${index}`);
|
||||
const { screen } = table(snapshotWithRigs(names), 160);
|
||||
const rows = seatRows(screen).filter((row) => names.includes(row.rig));
|
||||
expect(rows.map((row) => row.rig)).toEqual(names);
|
||||
for (const row of rows) expect(row.text.startsWith(`${row.rig} `)).toBe(true);
|
||||
});
|
||||
|
||||
it("names a rig on its first row only, across pod dividers, while POD still leads each pod", () => {
|
||||
const { screen, content, rigWidth } = table(multiRigSnapshot(), 160);
|
||||
const build = seatRows(screen).filter((row) => row.rig === "build");
|
||||
expect(build.map((row) => row.name)).toEqual(["dev.driver", "dev.guard", "qa.checker"]);
|
||||
expect(build[0]!.text).toMatch(/^build\s+dev\s/);
|
||||
expect(build[1]!.text.slice(0, rigWidth + 4).trim()).toBe("");
|
||||
expect(build[2]!.text).toMatch(new RegExp(`^ {${rigWidth}} qa\\s`));
|
||||
expect(content[build[2]!.y - 2]).toMatch(/^┈{8,}/);
|
||||
expect(content.filter((line) => line.startsWith("build "))).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("drills into the row's rig from a blank RIG cell", () => {
|
||||
const { screen, rigWidth } = table(multiRigSnapshot(), 160);
|
||||
const guard = seatRows(screen).find((row) => row.name === "dev.guard")!;
|
||||
expect(guard.text.slice(0, rigWidth).trim()).toBe("");
|
||||
for (const x of [screen.explorerWidth + 3, screen.explorerWidth + 2 + rigWidth]) {
|
||||
const hit = screen.hitMap.find((target) => target.y === guard.y && x >= target.x1 && x <= target.x2);
|
||||
expect(hit?.action).toEqual({ type: "drill", resource: "rig", name: "build", target: { host: "mm2-openrig1" } });
|
||||
}
|
||||
const pastRig = screen.explorerWidth + 3 + rigWidth;
|
||||
const seatHit = screen.hitMap.find((target) => target.y === guard.y && pastRig >= target.x1 && pastRig <= target.x2);
|
||||
expect(seatHit?.action).toMatchObject({ type: "drill", resource: "agent", name: "dev.guard" });
|
||||
});
|
||||
|
||||
it("names a scrolled rig on its top visible row and leaves the rows under it blank", () => {
|
||||
const snap = multiRigSnapshot();
|
||||
snap.hosts[0]!.rigs[0]!.pods[0]!.agents.push(...Array.from({ length: 36 }, (_, index) =>
|
||||
agent(`dev.extra-${index}`, `dev-extra-${index}@build`, "active", 20 + index),
|
||||
));
|
||||
const view = openInstance(snap);
|
||||
let screen = renderScreen(view.get(), snap, { cols: 160, rows: 30, nowMs: 0, colorMode: "none" });
|
||||
view.dispatch({ type: "layout", contentMaxOffset: screen.contentMaxOffset, contentTargetCount: screen.contentTargets.length });
|
||||
let checked = false;
|
||||
for (let offset = 0; offset <= screen.contentMaxOffset && !checked; offset += 1) {
|
||||
screen = renderScreen(view.get(), snap, { cols: 160, rows: 30, nowMs: 0, colorMode: "none" });
|
||||
const build = seatRows(screen).filter((row) => row.rig === "build");
|
||||
if (build.length >= 3 && build[0]!.name !== "dev.driver") {
|
||||
expect(build[0]!.text.startsWith("build ")).toBe(true);
|
||||
for (const row of build.slice(1)) expect(row.text.startsWith(" ")).toBe(true);
|
||||
checked = true;
|
||||
}
|
||||
view.dispatch({ type: "content-scroll", delta: 1 });
|
||||
}
|
||||
expect(checked).toBe(true);
|
||||
});
|
||||
|
||||
it("sizes RIG from the rigs the filter leaves on screen", () => {
|
||||
const snap = snapshotWithRigs([LONG, "short-rig"]);
|
||||
expect(table(snap, 160).rigWidth).toBe(LONG.length);
|
||||
expect(table(snap, 160, "short").rigWidth).toBe(13);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user