mirror of
https://github.com/mvschwarz/openrig.git
synced 2026-11-11 17:32:22 -10:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bdaa8717bd | ||
|
|
b68742c3c2 | ||
|
|
80cfb7d01a | ||
|
|
f1a06c4f1b | ||
|
|
a7b8a27eb8 | ||
|
|
be2690179d | ||
|
|
a71ce16dd7 | ||
|
|
0834a16949 | ||
|
|
f1668d6907 | ||
|
|
89999f6ebf | ||
|
|
f3a3b1711b | ||
|
|
7886582f9d | ||
|
|
cf6d633667 | ||
|
|
8dfacd2604 | ||
|
|
4adc8872f2 | ||
|
|
94241e6c45 | ||
|
|
dfd4b6dbf4 | ||
|
|
c3abaaf00d |
No files matched your search
+4
-5
@@ -26,7 +26,7 @@ OpenRig. Two repairs aren't in this release: see Known limits.
|
||||
seat's own `OPENRIG_NODE_ID`, on the server and session it read. A terminal
|
||||
OpenRig didn't create, a mismarked one or a replaced one is left running, and
|
||||
`rig down` exits 2 naming it and how to stop it yourself. A seat joined with
|
||||
`rig attach --self` from a tmux pane is in that group. ([#2962](https://github.com/mvschwarz/openrig/pull/2962))
|
||||
`rig attach --self` is in that group. ([#2962](https://github.com/mvschwarz/openrig/pull/2962))
|
||||
- **openrig-core is 0.1.28.** A Codex session kept running across the upgrade
|
||||
can ask for hook approval until it's relaunched.
|
||||
|
||||
@@ -62,10 +62,9 @@ OpenRig. Two repairs aren't in this release: see Known limits.
|
||||
|
||||
- Queue content can be erased for closed-work retention and account deletion,
|
||||
and a retried create can't bring erased work back. ([#2837](https://github.com/mvschwarz/openrig/pull/2837), [#2843](https://github.com/mvschwarz/openrig/pull/2843))
|
||||
- A send records what's known about its delivery in one
|
||||
- Each send records what's known about its delivery in one
|
||||
`transport.delivery_evidence` event: what was admitted, what reached the
|
||||
input, what the screen check showed, and for which occupant. The record is
|
||||
best effort, so a missing one doesn't show there was no send. ([#2937](https://github.com/mvschwarz/openrig/pull/2937), [#2943](https://github.com/mvschwarz/openrig/pull/2943))
|
||||
input, what the screen check showed, and for which occupant. ([#2937](https://github.com/mvschwarz/openrig/pull/2937), [#2943](https://github.com/mvschwarz/openrig/pull/2943))
|
||||
- `rig queue create` advises on an unresolved scope, slice lists show observed
|
||||
queue counts, a proof policy can require every listed judge, the execution
|
||||
ladder shows each slice's recorded status, `rig queue show` names a handed-off
|
||||
@@ -151,7 +150,7 @@ OpenRig. Two repairs aren't in this release: see Known limits.
|
||||
It doesn't keep a stored host or boot identity, doesn't guard against another
|
||||
program running as you, and a daemon restart between losing the tmux server
|
||||
and the stop wasn't exercised. For an adopted session, such as a seat joined
|
||||
with `rig attach --self` from a tmux pane, `rig seat stop` still suggests `rig down`, which
|
||||
with `rig attach --self`, `rig seat stop` still suggests `rig down`, which
|
||||
leaves that terminal running too. Release it with `rig unclaim <session>`
|
||||
(that leaves its process running), then close it in its own terminal. ([#2962](https://github.com/mvschwarz/openrig/pull/2962))
|
||||
- **Codex working counter:** a capture that doesn't name its pane keeps the old
|
||||
|
||||
+2
-2
@@ -6,9 +6,9 @@ This isn't a schedule or a promise: we release often, and an item ships when a g
|
||||
To see what's being worked on, by release and theme, open the [OpenRig roadmap board](https://github.com/users/mvschwarz/projects/1).
|
||||
Issues labelled `good first issue` are also listed on the [contribute page](https://github.com/mvschwarz/openrig/contribute).
|
||||
|
||||
## Released in 0.7.10
|
||||
## Released in 0.7.11
|
||||
|
||||
[0.7.10 is published](https://github.com/mvschwarz/openrig/releases/tag/v0.7.10): it is a patch for people running their own agent teams. A step held after a managed compaction now goes in instead of waiting forever. See its release notes for what shipped, known issues, upgrade instructions and contributor credits.
|
||||
[0.7.11 is published](https://github.com/mvschwarz/openrig/releases/tag/v0.7.11): OpenRig 0.7.11 is a patch for people running their own agent teams. An ordinary send or work notice now goes into a person's typed draft by default instead of being refused (see Behaviour that changed). See its release notes for what shipped, known issues, upgrade instructions and contributor credits.
|
||||
|
||||
## How to help
|
||||
|
||||
|
||||
@@ -116,5 +116,3 @@ When a regression reaches a release, add one line: date, issue, artery, and whic
|
||||
- 2026-10-24, B6 (0.7.1 candidate `7badb557`), launch and resume (Codex hook trust): after an upgrade from 0.7.0, a Codex seat stopped at "Hooks need review: 6", because Codex 0.164 asks to approve a plugin's whole content again. Caught at the cut, before publication, by a native upgrade run with a real Codex. Missed by source tests and stub scenarios (a stub can't see the real Codex binary) and by the installed-upgrade check (no real Codex). Fixed in #1948. Scenario owed: a cheap real-agent Codex upgrade with approval renewal.
|
||||
- 2026-10-24, pane-only attention (#1876 on the 0.7.1 candidate), native observation: a Codex seat waiting at a prompt that only its pane showed read as working, with no attention signal. Caught at the cut, before publication, by real work on a test machine. Missed by fixtures that all carried hook signals. Fixed in #1946. Scenario owed: a stub pane showing a selector with no hook event must read as needing attention.
|
||||
- 2026-10-25, #1955, delivery (send readiness): approval-prompt text that a working Claude seat quoted in its own output read as a waiting prompt, so sends and queue wakes to that seat were refused. This one was present since 0.7.0, so it did reach a release; it was found during the 0.7.1 cut by real work on a test machine. Missed by classifier fixtures that had no working pane quoting prompt text. Fixed in #1955 for 0.7.2.
|
||||
- 2026-11-10, F4 (0.7.11, #2662 leg (a)), delivery (draft-aware, under Claude's update banner): beneath Claude's bare update banner, a draft-aware delivery pasted the message and withheld Enter, leaving it in the composer, where 0.7.10 held it. Caught at the cut by a native delivery run and accepted with disclosure, so it reached 0.7.11. Missed by footer-grammar fixtures that had no paste hint beside the update notice. Fixed in #2786 for 0.7.12.
|
||||
- 2026-11-10, Pi case 1c (0.7.11, #2630), launch (Pi account home): a Pi seat with its own Pi login and a declared absolute `PI_CODING_AGENT_DIR` stopped at launch even with an `auth.json` there, with a misleading missing-account-file error, where 0.7.10 started it. Found at the cut by source and function-level tests, not natively, and accepted with disclosure, so it reached 0.7.11. Not fixed: #2803 (0.7.12) makes the launch error name the real cause, and the stop stands. Scenario owed: a Pi seat with its own login and a declared account home.
|
||||
@@ -75,20 +75,20 @@ in the rig's `attentionCount` and the TUI's attention totals.
|
||||
After a full restore of a Claude or Codex seat, answering the prompt is enough. When
|
||||
the seat's activity next turns to running or idle, or when the whole restore completes
|
||||
after the seat already reads running or idle, the daemon checks the live seat: its tmux
|
||||
session is alive, and the pane runs the agent resuming the conversation the seat
|
||||
records. When the check passes, the restore outcome becomes `operator_recovered`, so
|
||||
the restore no longer makes the seat need attention. When it fails, the restore outcome
|
||||
stays and the seat still needs attention, which is true when the pane runs another
|
||||
conversation. A check that started before a newer restore of the seat doesn't settle
|
||||
the newer outcome.
|
||||
session is alive, and the agent in its pane was started to resume the conversation the
|
||||
seat records. When the check passes, the restore outcome becomes `operator_recovered`, so
|
||||
the restore no longer makes the seat need attention. When the check fails, the restore
|
||||
outcome stays and the seat still needs attention. The check uses launch evidence; it
|
||||
does not prove which conversation is open now.
|
||||
|
||||
`rig seat clear-attention <seat>` runs the same check on demand. When it fails, the
|
||||
command says which check failed, and `--reason` doesn't bypass it. If
|
||||
the seat records the wrong conversation and you know the right one, set it with
|
||||
`rig seat set-resume-token <seat> --token-stdin --reason <text>`, then run
|
||||
`clear-attention` again. After a restore of only some seats, or a community
|
||||
harness's assisted startup, `--reason "<what you saw>"` can acknowledge the
|
||||
outcome instead.
|
||||
command says which check failed, and `--reason` doesn't bypass it. If the seat records
|
||||
the wrong conversation and the agent in its pane was started with the right one, set it
|
||||
with `rig seat set-resume-token <seat> --token-stdin --reason <text>`, then run
|
||||
`clear-attention` again; a token the running agent wasn't started with fails the same
|
||||
check. After a restore of only some seats, or a community harness's assisted startup,
|
||||
`--reason "<what you saw>"` can acknowledge the outcome instead, once any startup text
|
||||
left in the input box is submitted (not sent again) and the seat has responded.
|
||||
|
||||
## Two Categories of Startup
|
||||
|
||||
@@ -278,7 +278,10 @@ with an old Claude header or input frame above it and Claude already running. A
|
||||
input frame also keeps the check pending. If the current screen or process cannot be read, or the process identity is unconfirmed, the handover keeps
|
||||
waiting within the readiness window and fails if that window expires. Old screen text or the `claude` process label
|
||||
alone cannot complete this check. A healthy agent can therefore remain unconfirmed while those observations are
|
||||
unavailable; inspect its pane before deciding how to recover.
|
||||
unavailable; inspect its pane before deciding how to recover. When the launch was already sent, the failure's
|
||||
guidance names that pane and the result carries `replacementStarted` and `successorPane`: the previous agent was
|
||||
already ended, the new one may be working there without being registered as the seat's occupant, and another
|
||||
handover or `rig seat launch <seat> --fresh --stop --reason <why>` ends it if it goes ahead.
|
||||
|
||||
**Claude submission check.** For each startup message to a Claude seat, OpenRig makes a bounded check that the prompt
|
||||
was submitted. When the capture can't tell, it records the submission as unverified and carries on. When it sees the
|
||||
|
||||
@@ -84,11 +84,23 @@ so its rate-limit hook is attributed to that seat too. This is appropriate when
|
||||
the helper uses the same login, since the quota belongs to the account. A
|
||||
helper on a different login is attributed the same way, so its rate limit can
|
||||
hold pending handoffs to every seat in that seat's Claude pool for one retry
|
||||
interval, even when the seat's own login still has room. This attribution does
|
||||
not bind the hook to Claude's own native session.
|
||||
interval, even when the seat's own login still has room. A reset time the
|
||||
helper's message states is not used: OpenRig keeps a stated reset only when the
|
||||
hook's Claude `session_id` is the seat's own resume token, so a helper never
|
||||
holds the pool past that one interval.
|
||||
|
||||
The event carries no usage percentage, account identity or reset timestamp.
|
||||
It marks the Claude provider pool as limited and preserves a current
|
||||
The event carries no usage percentage or account identity. When the seat's own
|
||||
Claude states when the limit resets (for example `resets Nov 10, 4pm
|
||||
(America/Sao_Paulo)`, `resets in 2h 30m`, or the older `usage limit
|
||||
reached|<epoch>`), the hook relay reads that time on the seat's machine and
|
||||
forwards only the resulting timestamp (`resetsAt`), never the message text. It
|
||||
reads the reset only after the typed `rate_limit` failure; the text never
|
||||
decides that a seat is limited. A reset it can't read completely (an unknown or
|
||||
truncated time zone, an unsupported unit, a date that doesn't exist such as Feb
|
||||
30) or that is in the past or more than eight days away gives no reset. The
|
||||
daemon applies the same eight-day bound to any sender.
|
||||
|
||||
The event marks the Claude provider pool as limited and preserves a current
|
||||
`agentActivity.reason` of `usage_limit` over a cached idle reading. It does not
|
||||
count as a request for human input; a newer permission prompt still takes
|
||||
precedence in the seat's activity display. Conflicting seat windows retain the
|
||||
@@ -99,13 +111,15 @@ Pending work handled by the wake ladder uses the existing local Claude pool
|
||||
declare different Claude config directories. If reset windows conflict within
|
||||
a proposed shared pool, only the seats reporting exhaustion are held; another
|
||||
seat's pending handoffs can still retry. An exhausted status line window with
|
||||
a known reset supplies the timer. Otherwise a typed failure uses
|
||||
a known reset supplies the timer, and so does a reset the seat's own typed
|
||||
failure stated (`provider-reset`). Otherwise a typed failure uses
|
||||
`queue.wake_retry_interval_seconds` (default 300) from the event time, plus the
|
||||
existing wake jitter, and records `config-fallback`. That is a retry time, not a
|
||||
claim that the provider's quota will reset then. The timer releases the shared
|
||||
blocker once and retains the existing durable wake delivery behavior.
|
||||
|
||||
Authentication, billing and overloaded errors are not classified as usage
|
||||
limits. Claude limit banners are not scraped, and older runtimes or failures
|
||||
that do not emit this hook still depend on status line data. Without either
|
||||
limits. Claude limit banners are not scraped from the terminal, and older
|
||||
runtimes or failures that do not emit this hook still depend on status line
|
||||
data. Without either
|
||||
source, provider usage remains unknown.
|
||||
@@ -866,8 +866,8 @@ What it costs:
|
||||
## Stop your teams
|
||||
|
||||
`rig down <rig-name>` ends that rig's agent sessions and work in progress.
|
||||
A terminal OpenRig didn't start, such as a seat joined with `rig attach --self`
|
||||
from a tmux pane, is left running: `rig down` names it with the commands to stop it by hand, and
|
||||
A terminal OpenRig didn't start, such as a seat joined with `rig attach --self`,
|
||||
is left running: `rig down` names it with the commands to stop it by hand, and
|
||||
exits 2.
|
||||
Read `rig ps --nodes --rig <rig-name>` first so you can see who will stop;
|
||||
`rig down` also prints the rig's recorded agent sessions and their activity
|
||||
@@ -898,7 +898,7 @@ the lifecycle API without booting a new kernel, then stop the remaining rigs.
|
||||
| --- | --- |
|
||||
| Tool missing or login fails | Use the specific setup/auth hint; recheck that executable in the launch shell. Do not send work to an unready seat. |
|
||||
| "Startup attention" or "Startup details" names a prompt | Answer the login, trust or bypass prompt in that seat's terminal, then run the `rig seat continue <seat>` it shows. |
|
||||
| A restored seat still counts as needing attention after its prompt was answered | Its restore outcome is still `attention_required`, and answering the prompt or running `rig seat continue` doesn't change it. Once the agent is up in its pane, run `rig seat clear-attention <seat>`; see [After a restored seat's prompt](agent-startup-guide.md#after-a-restored-seats-prompt). |
|
||||
| A restored seat still counts as needing attention after its prompt was answered | After a full restore of a Claude or Codex seat, the daemon clears this by itself once the seat reads running or idle and passes its check. If it still counts, check its `restoreOutcome` in `rig ps --nodes --json` (the RESTORE column): when it is `attention_required`, run `rig seat clear-attention <seat>`, which says which check failed; answering the prompt again or running `rig seat continue` doesn't change it. For a partial restore, submit any startup text left in the input box without sending it again, and wait for the agent to respond before clearing attention. See [After a restored seat's prompt](agent-startup-guide.md#after-a-restored-seats-prompt). |
|
||||
| "Readiness timeout after 30s — harness did not become interactive" | The harness took longer than the readiness window. Raise it for the next launch with `rig config set runtime.readiness_timeout_seconds <1-600>`, then relaunch that seat. |
|
||||
| "Claude resume did not finish within Ns; Claude may still be starting, so the launch was kept" | A restored Claude seat was not interactive when the wait ended. Its pane and resume token are kept. Look at the pane; once Claude is up, run `rig seat clear-attention <seat>`. The wait is about `runtime.readiness_timeout_seconds` in total for each seat (default 30 s), counting the first 3.2 s, and the time adds up seat by seat in a restore. Raising the setting for slow fresh launches lengthens it. If `rig up` or `rig launch` times out at 120 s, that is not a rollback: check `rig ps` before running it again. |
|
||||
| Daemon is healthy, kernel is still starting | Read `rig status` and `rig ps --nodes --rig kernel`; kernel readiness is separate. |
|
||||
|
||||
@@ -296,3 +296,31 @@ Unknown: not said yet.
|
||||
delivery guarantee.
|
||||
- **Transcript bytes as "usage".** `context-usage-threshold` measures an agent's transcript, not an account's
|
||||
allowance. For an allowance, see "A usage warning" above.
|
||||
|
||||
## Recurring reminders for agents
|
||||
|
||||
An agent-to-agent reminder can repeat until stopped. This is separate from
|
||||
[a person's one-shot watch](#setting-a-watch), which uses `rig queue block --wake-after`.
|
||||
|
||||
Save this as `periodic-reminder.yaml`, replacing the example session names with your own:
|
||||
|
||||
```yaml
|
||||
policy: periodic-reminder
|
||||
target:
|
||||
session: worker@my-team
|
||||
message: "Finish the current action, then check the pending work."
|
||||
```
|
||||
|
||||
Register it with all five required flags:
|
||||
|
||||
```bash
|
||||
rig watchdog register --spec periodic-reminder.yaml --policy periodic-reminder \
|
||||
--target-session worker@my-team --interval-seconds 600 \
|
||||
--registered-by coordinator@my-team
|
||||
```
|
||||
|
||||
At registration, `--policy` selects the policy; the YAML `policy` field is not the
|
||||
selector. YAML `target.session` takes precedence over `--target-session`, so keep them
|
||||
aligned. If YAML omits the target, the CLI target is the fallback. Put a non-empty
|
||||
`message` in YAML: `--message` is a shortcut only for `context-usage-threshold`.
|
||||
The first reminder is due after the interval; this example repeats every 600 seconds.
|
||||
@@ -0,0 +1,114 @@
|
||||
# Network connector: a local seat as a hub participant
|
||||
|
||||
Source of truth: `packages/daemon/src/domain/network/` (`network-bindings.ts`, `network-connector.ts`, `hub-client.ts`),
|
||||
`packages/daemon/src/routes/network.ts`, `packages/cli/src/commands/network.ts`.
|
||||
|
||||
A hub is another OpenRig queue that its participants reach over MCP, such as a daemon served by `rig mcp serve --http`
|
||||
with one grant per participant. The network connector puts one selected local seat on a hub as one of its participants.
|
||||
The connector opens no port: your daemon makes every connection outward, and the hub never connects to your machine. Work the hub
|
||||
sends that participant runs on your seat, under your accounts, and the result goes back to the hub item.
|
||||
|
||||
## Connect
|
||||
|
||||
```bash
|
||||
rig network connect <seat> --hub <url> --as <participant> --token-file <path>
|
||||
```
|
||||
|
||||
- `<participant>` is the participant's address in the hub (`member@rig`), and the token file holds its bearer.
|
||||
- The binding follows the seat's node, so a relaunched or renamed seat keeps it.
|
||||
- The token is stored at `<OpenRig home>/network-secrets/<binding>/credential.json` (a 0700 folder and a 0600 file). It
|
||||
is never returned by the API or shown in status. A token in the hub URL is refused.
|
||||
- A plain-http hub URL needs `--allow-http` (see Limits).
|
||||
- One active binding per local seat, and one per hub participant.
|
||||
- On a daemon with an operator bearer configured, `connect` and `disconnect` send `OPENRIG_AUTH_BEARER_TOKEN`, as the
|
||||
`rig host` writes do.
|
||||
|
||||
## What the connector does
|
||||
|
||||
The daemon runs the connector for every active binding. It normally polls every 30 seconds
|
||||
(`OPENRIG_NETWORK_POLL_SECONDS`, at least 5); a failing hub, backoff or a backlog can delay it. Each binding's pass has
|
||||
a deadline of 60 seconds, so one slow hub doesn't hold the others. With no active binding it makes no network call.
|
||||
|
||||
1. It lists the participant's own items on the hub, claims each pending one, and creates a local queue item for the
|
||||
seat's current address. The local item comes from OpenRig itself, and it says where the work came from first: its
|
||||
summary starts "From <requester> via network hub", and its body opens with the requester, the hub and the hub item,
|
||||
before the requester's own words.
|
||||
2. It reports each stage on the hub item, kept apart:
|
||||
- **Queued:** the hub claim means the connector holds the item. The note says "Not picked up yet."
|
||||
- **Picked up:** once the seat claims the local item.
|
||||
- **The result:** once the local item closes, following any local handoffs to the item that finished. The hub item
|
||||
closes the way its queue closes one: done with the resolution and the closure reason (`no-follow-on`, `canceled`
|
||||
or `denied`), or failed when the local item failed. A close with no resolution sends the closer's own note on that
|
||||
close, or "Closed without a stated result." No other note on the item is sent.
|
||||
|
||||
To answer, close the local item with your result:
|
||||
|
||||
```bash
|
||||
rig queue update <local item> --state done --closure-reason no-follow-on --resolution "<your answer>"
|
||||
```
|
||||
|
||||
## Retries, restarts and unknown outcomes
|
||||
|
||||
- Each hub item maps to one local item, under an id derived from the seat's node, the hub, the participant and the hub
|
||||
item. A lost answer, a retry, a daemon restart, or disconnecting and connecting the same seat to the same hub and
|
||||
participant again lands on that same item, so it isn't queued twice. A new binding reports what the earlier one
|
||||
took.
|
||||
- If the local record of an item is gone, or it was handed on without one successor here, the hub item says its outcome
|
||||
is unknown, and it is never run again.
|
||||
- If someone else closed the hub item first (its requester canceled it, say), the local result stays here, the connector
|
||||
doesn't count it as delivered, and `rig network status` (and its `--json`) shows it under the binding as closed
|
||||
elsewhere. The connector checks the hub item just before closing it. It does not write over a close it has already
|
||||
seen, and reads the item back after a close attempt. A hub item someone there handed on counts as closed elsewhere
|
||||
too.
|
||||
- The derived id makes the queue item unique, not the work's outside effects. Anything your agent did outside OpenRig
|
||||
before a crash is neither undone nor deduplicated.
|
||||
|
||||
## Status
|
||||
|
||||
```bash
|
||||
rig network status [--json]
|
||||
```
|
||||
|
||||
Each binding shows its seat, participant, hub, state, and its last contact or last error. `--json` adds the
|
||||
connector's view: each binding's next pass, backoff and last pass.
|
||||
|
||||
## When things go wrong
|
||||
|
||||
- **The seat has no session:** its hub items stay pending and unclaimed, and the last error says why. They're taken
|
||||
once the seat has a session again. A stopped seat still has its session, so it still takes items; they wait for it.
|
||||
- **The hub can't be reached, or a pass runs past its deadline:** the binding stays active, the error is recorded, and
|
||||
retries back off, doubling up to 5 minutes. A failure on this machine is recorded as local, not as the hub's.
|
||||
- **The hub refuses the credential** (HTTP 401): the binding is marked revoked and its credential removed. No new work
|
||||
comes in; local items and results stay.
|
||||
|
||||
## Disconnect
|
||||
|
||||
```bash
|
||||
rig network disconnect <binding-or-seat> [--reason "<why>"]
|
||||
```
|
||||
|
||||
The binding ends here, so no new work comes in through it, and its credential is removed. Local items and results
|
||||
stay. The hub still lists the participant until it's removed there too.
|
||||
|
||||
If the credential can't be removed, the binding still ends, and the error names the
|
||||
`rig network disconnect <binding id>` to run again. The daemon also removes credentials whose binding isn't active
|
||||
each time it starts.
|
||||
|
||||
## Limits
|
||||
|
||||
- The connector polls, so new work normally waits up to one interval before it reaches the seat, and longer when the
|
||||
hub fails or a backlog builds.
|
||||
- If someone else closes a hub item between the connector's check and its close, the hub may keep either outcome, or
|
||||
combine the later state with the earlier resolution. The connector counts its result as delivered only when the
|
||||
readback matches its state, closure reason and resolution; otherwise a closed item is reported as closed elsewhere.
|
||||
An atomic close-if-open needs the hub to offer a conditional close.
|
||||
- After a disconnect or a revocation, work already taken keeps running here, but its pickup and result no longer reach
|
||||
the hub item, which stays at its last reported stage.
|
||||
- **Plain http needs your choice.** `rig network connect` refuses an `http:` hub URL until you add `--allow-http`, because
|
||||
the participant's token and its work then cross the network unencrypted, where they can be read. Use the hub's
|
||||
`https:` URL where it has one. The choice is recorded on that binding, so the connector doesn't ask again, and
|
||||
`rig network status` (and its `--json`) marks the binding unencrypted. OpenRig doesn't judge the address: a private or
|
||||
tailnet address doesn't prove the connection is protected.
|
||||
- **Redirects:** the connector follows a hub's redirect only within its own origin, or to its `https:` form on the same
|
||||
host. Any other redirect stops the connection with its target named, and the token isn't sent there. Connect the
|
||||
binding to the hub's own URL.
|
||||
@@ -110,8 +110,8 @@ To move a team that runs on an older server:
|
||||
on it, yours included, has ended.
|
||||
3. Stop each rig deliberately with `rig down <rig>`. Its seats' sessions end, and a
|
||||
turn in progress stops. Queue rows, rigs and seats stay as they are. A session
|
||||
OpenRig didn't start, such as a seat joined with `rig attach --self` from a tmux
|
||||
pane, is left running and `rig down` exits 2; stop it with the commands it prints, or it stays on
|
||||
OpenRig didn't start, such as a seat joined with `rig attach --self`, is left
|
||||
running and `rig down` exits 2; stop it with the commands it prints, or it stays on
|
||||
the old server.
|
||||
4. Restart the daemon with `OPENRIG_TMUX_CONTROL=1` in its environment.
|
||||
`rig terminal status --json` should show `migrationPending: false`. If it's still
|
||||
|
||||
Generated
+1
@@ -7341,6 +7341,7 @@
|
||||
"dependencies": {
|
||||
"@hono/node-server": "^1.13.0",
|
||||
"@hono/node-ws": "^1.3.1",
|
||||
"@modelcontextprotocol/sdk": "^1.32.2",
|
||||
"better-sqlite3": "^13.0.3",
|
||||
"hono": "^4.7.0",
|
||||
"smol-toml": "^1.8.0",
|
||||
|
||||
@@ -24,9 +24,8 @@ import { existsSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { hostname as osHostname, userInfo } from "node:os";
|
||||
import { connect } from "node:net";
|
||||
import { execFile } from "node:child_process";
|
||||
import { getOpenRigHome } from "../openrig-compat.js";
|
||||
import { addHostEntry, updateHostEntry, removeHostEntry, defaultHostRegistryPath, loadHostRegistry, validateHostRegistry, hostDisplayTarget, resolveHost, resolveRemoteBearer, bearerAuthHeaders, type HostEntry, type HttpHostEntry, type SshHostEntry } from "../host-registry.js";
|
||||
import { addHostEntry, updateHostEntry, removeHostEntry, defaultHostRegistryPath, loadHostRegistry, validateHostRegistry, hostDisplayTarget, resolveHost, resolveRemoteBearer, bearerAuthHeaders, type HostEntry, type HttpHostEntry } from "../host-registry.js";
|
||||
import { runCrossHostCommand, type CrossHostResult } from "../cross-host-executor.js";
|
||||
import { recordHostObservation, loadHostBindings, describeBindingConflict, defaultHostBindingsPath, type HostObservationOutcome } from "../host-bindings.js";
|
||||
import { DaemonClient, operatorAuthHeaders } from "../client.js";
|
||||
@@ -168,34 +167,13 @@ function defaultDoctorDeps(): DoctorDeps {
|
||||
};
|
||||
}
|
||||
|
||||
/** Where ssh would connect for this host, from `ssh -G`, which reads ssh config without connecting.
|
||||
* Null when ssh can't resolve the target or a ProxyJump/ProxyCommand sits in between, since no
|
||||
* direct dial stands for that transport. */
|
||||
export function resolveSshEndpoint(host: SshHostEntry, timeoutMs = 1500): Promise<{ hostname: string; port: number } | null> {
|
||||
// `--` before the target: a destination that begins with `-` is fed to ssh as the host, not
|
||||
// read as an option. ssh answers an invalid hostname on its own.
|
||||
const args = [...(host.user ? ["-l", host.user] : []), "-G", "--", host.target];
|
||||
return new Promise((resolve) => {
|
||||
execFile("ssh", args, { timeout: timeoutMs }, (error, stdout) => {
|
||||
if (error) return resolve(null);
|
||||
const conf = new Map(stdout.split("\n").map((line) => {
|
||||
const i = line.indexOf(" ");
|
||||
return [line.slice(0, i).toLowerCase(), line.slice(i + 1).trim()] as const;
|
||||
}));
|
||||
const proxied = ["proxyjump", "proxycommand"].some((k) => (conf.get(k) ?? "none") !== "none");
|
||||
const hostname = conf.get("hostname");
|
||||
const port = Number(conf.get("port"));
|
||||
resolve(hostname && Number.isInteger(port) && !proxied ? { hostname, port } : null);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/** FR-1 doctor legs — each failing step maps to a DISTINCT actionable error:
|
||||
* "SSH works but daemon is down" ≠ "daemon works but registry is wrong" ≠
|
||||
* "remote rig binary missing/old". */
|
||||
/** OPR.0.4.6.MH1 FR-3 — the coarse ls status word: a BOUNDED tcp dial
|
||||
* against the host's transport endpoint (ssh: the host and port ssh itself
|
||||
* resolves, see resolveSshEndpoint; http: the URL's host:port). Coarse by
|
||||
* against an HTTP host's URL host:port; passive SSH entries report
|
||||
* `unknown` without running `ssh` or opening a raw TCP socket (explicit SSH
|
||||
* verification lives in `rig host doctor`). Coarse by
|
||||
* design (the kubectl STATUS / docker context ls precedent): reachable |
|
||||
* unreachable | unknown — never a hang (hard timeout), never a guess (dial
|
||||
* failure = unreachable, probe error/unparseable endpoint = unknown). */
|
||||
@@ -205,17 +183,12 @@ export async function probeHostStatus(
|
||||
timeoutMs = 1500,
|
||||
): Promise<"reachable" | "unreachable" | "unknown"> {
|
||||
try {
|
||||
let target: string;
|
||||
let port: number;
|
||||
if (host.transport === "ssh") {
|
||||
const endpoint = await resolveSshEndpoint(host, timeoutMs);
|
||||
if (!endpoint) return "unknown";
|
||||
({ hostname: target, port } = endpoint);
|
||||
} else {
|
||||
const u = new URL(host.url);
|
||||
target = u.hostname.replace(/^\[|\]$/g, "");
|
||||
port = u.port ? Number(u.port) : u.protocol === "https:" ? 443 : 80;
|
||||
return "unknown";
|
||||
}
|
||||
const u = new URL(host.url);
|
||||
const target = u.hostname.replace(/^\[|\]$/g, "");
|
||||
const port = u.port ? Number(u.port) : u.protocol === "https:" ? 443 : 80;
|
||||
const r = await tcpProbe(target, port, timeoutMs);
|
||||
return r === "open" ? "reachable" : r === "closed" ? "unreachable" : "unknown";
|
||||
} catch {
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
import { Command } from "commander";
|
||||
import { lstatSync, readFileSync } from "node:fs";
|
||||
import { DaemonClient, operatorAuthHeaders } from "../client.js";
|
||||
import { getDaemonStatus, getDaemonUrl, daemonStatusGuard, type LifecycleDeps } from "../daemon-lifecycle.js";
|
||||
import { realDeps } from "./daemon.js";
|
||||
|
||||
export interface NetworkDeps {
|
||||
lifecycleDeps: LifecycleDeps;
|
||||
clientFactory: (url: string) => Pick<DaemonClient, "get" | "post">;
|
||||
readTokenFile?: (path: string) => string;
|
||||
}
|
||||
|
||||
interface Binding {
|
||||
id: string; localSession: string; hubUrl: string; hubSeat: string; state: string; stateReason: string | null;
|
||||
lastContactAt: string | null; lastError: string | null; unencrypted?: boolean;
|
||||
/** Hub items someone else closed before this binding's result reached them. */
|
||||
closedElsewhere?: Array<{ hubItem: string; localItem: string }>;
|
||||
}
|
||||
|
||||
/** The participant's token, read from the file the account gave the person. A file others can read is used, with a note. */
|
||||
function defaultReadTokenFile(path: string): string {
|
||||
const stat = lstatSync(path);
|
||||
if (!stat.isFile()) throw new Error(`${path} isn't a regular file.`);
|
||||
if ((stat.mode & 0o077) !== 0) console.error(`Note: ${path} can be read by other users of this machine; OpenRig keeps its own private copy.`);
|
||||
return readFileSync(path, "utf8").trim();
|
||||
}
|
||||
|
||||
function line(b: Binding): string {
|
||||
const contact = b.lastContactAt ? `last contact ${b.lastContactAt}` : "no contact yet";
|
||||
const state = b.state === "active" ? contact : `${b.state}${b.stateReason ? ` (${b.stateReason})` : ""}`;
|
||||
const plain = b.unencrypted ? " UNENCRYPTED (http, allowed for this binding)" : "";
|
||||
const conflicts = (b.closedElsewhere ?? []).map((c) => `\n hub item ${c.hubItem} was closed elsewhere before its result arrived; the result stays on ${c.localItem}`).join("");
|
||||
return `${b.id} ${b.localSession} -> ${b.hubSeat} at ${b.hubUrl}${plain} ${state}${b.lastError && b.state === "active" ? ` last error: ${b.lastError}` : ""}${conflicts}`;
|
||||
}
|
||||
|
||||
/** `rig network`: connect a selected local seat to a participant of a remote hub (an account's network rig, or
|
||||
* another OpenRig hub), see where each connection stands, and disconnect one. */
|
||||
export function networkCommand(depsOverride?: NetworkDeps): Command {
|
||||
const cmd = new Command("network")
|
||||
.description("Connect selected local seats to a remote hub, see their status, and disconnect them");
|
||||
const getDeps = (): NetworkDeps => depsOverride ?? { lifecycleDeps: realDeps(), clientFactory: (url: string) => new DaemonClient(url) };
|
||||
|
||||
const client = async () => {
|
||||
const deps = getDeps();
|
||||
const daemon = await getDaemonStatus(deps.lifecycleDeps);
|
||||
if (!daemonStatusGuard(daemon)) return null;
|
||||
return deps.clientFactory(getDaemonUrl(daemon));
|
||||
};
|
||||
const finish = (status: number, data: unknown, json: boolean | undefined, ok: (data: Record<string, unknown>) => string) => {
|
||||
if (status >= 400) {
|
||||
const error = data as { error?: string; message?: string };
|
||||
console.error(json ? JSON.stringify(data) : `${error.message ?? "The daemon refused this"} (${error.error ?? status})`);
|
||||
process.exitCode = status >= 500 ? 2 : 1;
|
||||
return;
|
||||
}
|
||||
console.log(json ? JSON.stringify(data, null, 2) : ok(data as Record<string, unknown>));
|
||||
};
|
||||
|
||||
cmd.command("connect")
|
||||
.description("Connect a local seat to a hub participant, using that participant's token")
|
||||
.argument("<seat>", "The local seat to connect (its address, member@rig)")
|
||||
.requiredOption("--hub <url>", "The hub's MCP URL")
|
||||
.requiredOption("--as <participant>", "The participant's address in the hub (member@rig)")
|
||||
.requiredOption("--token-file <path>", "A file holding the participant's token")
|
||||
.option("--allow-http", "Accept a plain-http hub for this binding: its token and work cross the network unencrypted")
|
||||
.option("--json", "Print JSON")
|
||||
.action(async (seat: string, opts: { hub: string; as: string; tokenFile: string; allowHttp?: boolean; json?: boolean }) => {
|
||||
let token: string;
|
||||
try { token = (getDeps().readTokenFile ?? defaultReadTokenFile)(opts.tokenFile); } catch (error) {
|
||||
console.error(`Can't read the token file: ${(error as Error).message}`);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const daemon = await client();
|
||||
if (!daemon) return;
|
||||
const res = await daemon.post<Record<string, unknown>>("/api/network/bindings", { session: seat, hubUrl: opts.hub, hubSeat: opts.as, token, allowHttp: opts.allowHttp === true }, { headers: operatorAuthHeaders() });
|
||||
finish(res.status, res.data, opts.json, (b) => `Connected ${line(b as unknown as Binding)}\nWork for ${opts.as} reaches ${seat} once the connector runs; rig network status shows its last contact.`);
|
||||
});
|
||||
|
||||
cmd.command("status")
|
||||
.description("List the connections and where each stands")
|
||||
.option("--json", "Print JSON")
|
||||
.action(async (opts: { json?: boolean }) => {
|
||||
const daemon = await client();
|
||||
if (!daemon) return;
|
||||
const res = await daemon.get<{ bindings: Binding[] }>("/api/network/bindings");
|
||||
finish(res.status, res.data, opts.json, (d) => {
|
||||
const bindings = (d.bindings ?? []) as Binding[];
|
||||
return bindings.length ? bindings.map(line).join("\n") : "No seats are connected to a hub.";
|
||||
});
|
||||
});
|
||||
|
||||
cmd.command("disconnect")
|
||||
.description("Disconnect a seat: no new work comes in through it; its local work and results stay")
|
||||
.argument("<binding-or-seat>", "The binding id, or the connected seat's address")
|
||||
.option("--reason <text>", "Why, for the record")
|
||||
.option("--json", "Print JSON")
|
||||
.action(async (ref: string, opts: { reason?: string; json?: boolean }) => {
|
||||
const daemon = await client();
|
||||
if (!daemon) return;
|
||||
const res = await daemon.post<Record<string, unknown>>(`/api/network/bindings/${encodeURIComponent(ref)}/disconnect`, { reason: opts.reason }, { headers: operatorAuthHeaders() });
|
||||
finish(res.status, res.data, opts.json, (b) => `Disconnected ${line(b as unknown as Binding)}\nRemove the participant on the hub too, so the account stops listing it.`);
|
||||
});
|
||||
|
||||
return cmd;
|
||||
}
|
||||
@@ -1493,9 +1493,10 @@ Examples:
|
||||
.addHelpText("after", `
|
||||
Siblings are untouched. A dead seat is refused (use rig seat clean). An adopted
|
||||
session is refused: to end only it, run rig unclaim <session> (it releases the
|
||||
session and leaves its process running), then end that process in its own
|
||||
terminal. rig down <rig> ends every seat in its rig, so it is the route only
|
||||
for a one-seat rig.
|
||||
session and leaves its process running), then end that process and the
|
||||
underlying tmux session, including any remaining shell. Closing only its
|
||||
viewing window leaves the tmux session running. rig down <rig> leaves a
|
||||
session OpenRig didn't start running.
|
||||
A seat handed over (fresh, fork or rebuild) before OpenRig 0.7.9 still reads
|
||||
adopted and is refused too, until its next launched handover or managed launch;
|
||||
the refusal says so. Stopping ends that agent's session and any work in progress:
|
||||
|
||||
@@ -85,7 +85,7 @@ export function watchdogCommand(depsOverride?: WatchdogDeps): Command {
|
||||
"Context transcripts measured 113K–153K tokens/MB. The margin is the protection " +
|
||||
"because prompt-bound consumers act only at turn boundaries",
|
||||
)
|
||||
.option("--spec <path>", "Path to YAML spec file (optional for context-usage-threshold)")
|
||||
.option("--spec <path>", "YAML spec file (required except for context-usage-threshold)")
|
||||
.requiredOption("--policy <policy>", "Policy name (one of: periodic-reminder, artifact-pool-ready, edge-artifact-required, workflow-keepalive, idle-gate-qitem, context-usage-threshold)")
|
||||
.requiredOption("--target-session <session>", "Canonical <member>@<rig> target")
|
||||
.requiredOption("--interval-seconds <n>", "Evaluation interval (positive integer)")
|
||||
@@ -100,7 +100,27 @@ export function watchdogCommand(depsOverride?: WatchdogDeps): Command {
|
||||
.option("--active-wake-interval-seconds <n>", "(pool-ready-specific) wake-up cadence when actionable artifacts exist")
|
||||
.option("--scan-interval-seconds <n>", "(pool-ready-specific) artifact pool scan cadence")
|
||||
.option("--json", "JSON output for agents")
|
||||
.addHelpText("after", "\nTranscript density varies by session shape; do not tune a context threshold near the context wall.\n\nThe returned job id can be attached to a deliberate park with: rig queue block <qitemId> --on <blocker> --continuation <what-resumes> --wake-watchdog <jobId>")
|
||||
.addHelpText("after", `
|
||||
Recurring reminder for an agent (not a person's one-shot queue watch):
|
||||
Save periodic-reminder.yaml:
|
||||
policy: periodic-reminder
|
||||
target:
|
||||
session: worker@my-team
|
||||
message: "Finish the current action, then check the pending work."
|
||||
|
||||
rig watchdog register --spec periodic-reminder.yaml --policy periodic-reminder \\
|
||||
--target-session worker@my-team --interval-seconds 600 \\
|
||||
--registered-by coordinator@my-team
|
||||
|
||||
All five flags above are required for periodic-reminder. At registration,
|
||||
--policy selects the policy; a YAML policy field is not the selector.
|
||||
YAML target.session takes precedence over --target-session; keep them aligned.
|
||||
Put the reminder message in YAML. --message is only a context-usage-threshold shortcut.
|
||||
For a person's one-shot watch, use rig queue block --wake-after instead.
|
||||
|
||||
Transcript density varies by session shape; do not tune a context threshold near the context wall.
|
||||
|
||||
The returned job id can be attached to a deliberate park with: rig queue block <qitemId> --on <blocker> --continuation <what-resumes> --wake-watchdog <jobId>`)
|
||||
.action(async (opts: {
|
||||
spec?: string;
|
||||
policy: string;
|
||||
|
||||
@@ -91,6 +91,7 @@ import { compactPlanCommand, type CompactPlanDeps } from "./commands/compact-pla
|
||||
import { compactCommand, type CompactDeps } from "./commands/compact.js";
|
||||
import { heartbeatCommand, type HeartbeatDeps } from "./commands/heartbeat.js";
|
||||
import { seatCommand, handoverCommand, type SeatDeps } from "./commands/seat.js";
|
||||
import { networkCommand, type NetworkDeps } from "./commands/network.js";
|
||||
import { rigModeCommand, type RigModeDeps } from "./commands/rig-mode.js";
|
||||
import { policyCommand } from "./commands/policy.js";
|
||||
import { startupProofCommand, type StartupProofDeps } from "./commands/startup-proof.js";
|
||||
@@ -168,6 +169,7 @@ export interface ProgramDeps {
|
||||
compactDeps?: CompactDeps;
|
||||
heartbeatDeps?: HeartbeatDeps;
|
||||
seatDeps?: SeatDeps;
|
||||
networkDeps?: NetworkDeps;
|
||||
rigModeDeps?: RigModeDeps;
|
||||
startupProofDeps?: StartupProofDeps;
|
||||
healthDeps?: HealthDeps;
|
||||
@@ -269,6 +271,7 @@ export function createProgram(depsOverride?: ProgramDeps): Command {
|
||||
program.addCommand(heartbeatCommand(depsOverride?.heartbeatDeps));
|
||||
program.addCommand(seatCommand(depsOverride?.seatDeps));
|
||||
program.addCommand(handoverCommand(depsOverride?.seatDeps));
|
||||
program.addCommand(networkCommand(depsOverride?.networkDeps));
|
||||
program.addCommand(startupProofCommand(depsOverride?.startupProofDeps));
|
||||
// release-0.3.2 slice 12 — rig scope CLI primitive.
|
||||
program.addCommand(scopeCommand());
|
||||
|
||||
@@ -102,6 +102,7 @@ export const COMMAND_LOADERS: ReadonlyMap<string, Load> = new Map<string, Load>(
|
||||
["heartbeat", async () => (await import("./commands/heartbeat.js")).heartbeatCommand()],
|
||||
["seat", async () => (await import("./commands/seat.js")).seatCommand()],
|
||||
["handover", async () => (await import("./commands/seat.js")).handoverCommand()],
|
||||
["network", async () => (await import("./commands/network.js")).networkCommand()],
|
||||
["startup-proof", async () => (await import("./commands/startup-proof.js")).startupProofCommand()],
|
||||
["scope", async () => (await import("./commands/scope.js")).scopeCommand()],
|
||||
["proof", async () => (await import("./commands/proof.js")).proofCommand()],
|
||||
|
||||
@@ -1,51 +1,144 @@
|
||||
// `rig host ls` status for an ssh host dials where ssh itself would connect (`ssh -G`), so an
|
||||
// ssh-config alias or a non-22 port isn't reported unreachable. A stand-in `ssh` on PATH plays
|
||||
// `ssh -G`; no real ssh runs.
|
||||
import { describe, it, expect, beforeEach, afterEach } from "vitest";
|
||||
import { mkdtempSync, writeFileSync, chmodSync, rmSync } from "node:fs";
|
||||
// `rig host list` / `rig host ls` status is tested at process and probe boundaries:
|
||||
// passive SSH rows must not invoke ssh -G or trigger a TCP probe.
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { probeHostStatus } from "../src/commands/host.js";
|
||||
import { Command } from "commander";
|
||||
import { hostCommand, type DoctorDeps } from "../src/commands/host.js";
|
||||
|
||||
const { execFileMock, connectMock } = vi.hoisted(() => ({
|
||||
execFileMock: vi.fn(),
|
||||
connectMock: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("node:child_process", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("node:child_process")>();
|
||||
return { ...actual, execFile: execFileMock };
|
||||
});
|
||||
vi.mock("node:net", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("node:net")>();
|
||||
return { ...actual, connect: connectMock };
|
||||
});
|
||||
|
||||
interface ProbeCall {
|
||||
target: string;
|
||||
port: number;
|
||||
timeoutMs: number;
|
||||
}
|
||||
interface ExecFileCall {
|
||||
command: string;
|
||||
args: string[];
|
||||
timeout: number | undefined;
|
||||
}
|
||||
|
||||
let home: string;
|
||||
let savedHome: string | undefined;
|
||||
let execFileCalls: ExecFileCall[];
|
||||
let log: ReturnType<typeof vi.spyOn>;
|
||||
|
||||
function writeRegistry(hosts: unknown[]): void {
|
||||
writeFileSync(join(home, "hosts.yaml"), JSON.stringify({ hosts }));
|
||||
}
|
||||
|
||||
function deps(tcpProbe: DoctorDeps["tcpProbe"]): DoctorDeps {
|
||||
return {
|
||||
run: async () => { throw new Error("unexpected remote SSH command"); },
|
||||
httpGet: async () => { throw new Error("unexpected HTTP request"); },
|
||||
tcpProbe,
|
||||
};
|
||||
}
|
||||
|
||||
async function listHosts(hosts: unknown[], tcpProbe: DoctorDeps["tcpProbe"], verb: "list" | "ls" = "ls") {
|
||||
writeRegistry(hosts);
|
||||
const program = new Command().exitOverride();
|
||||
program.addCommand(hostCommand(deps(tcpProbe)));
|
||||
await program.parseAsync(["node", "rig", "host", verb, "--json"]);
|
||||
const output = log.mock.calls.map((call) => String(call[0])).find((line) => line.trim().startsWith("["));
|
||||
if (output === undefined) throw new Error("host list did not print its JSON array");
|
||||
return JSON.parse(output) as Array<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
let dir: string;
|
||||
let savedPath: string | undefined;
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(join(tmpdir(), "host-status-ssh-"));
|
||||
writeFileSync(join(dir, "ssh"), `#!/bin/sh
|
||||
case "$*" in
|
||||
*-G*devbox*) printf 'user ops\\nhostname 10.1.2.3\\nport 2222\\n' ;;
|
||||
*-G*jumpy*) printf 'hostname jumpy\\nport 22\\nproxyjump bastion\\n' ;;
|
||||
*) exit 255 ;;
|
||||
esac
|
||||
`);
|
||||
chmodSync(join(dir, "ssh"), 0o755);
|
||||
savedPath = process.env["PATH"];
|
||||
process.env["PATH"] = `${dir}:${savedPath}`;
|
||||
});
|
||||
afterEach(() => {
|
||||
process.env["PATH"] = savedPath;
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
home = mkdtempSync(join(tmpdir(), "host-list-ssh-inert-"));
|
||||
savedHome = process.env["OPENRIG_HOME"];
|
||||
process.env["OPENRIG_HOME"] = home;
|
||||
execFileCalls = [];
|
||||
execFileMock.mockReset();
|
||||
execFileMock.mockImplementation((
|
||||
command: string,
|
||||
args: string[],
|
||||
options: { timeout?: number },
|
||||
callback: (error: Error | null, stdout: string) => void,
|
||||
) => {
|
||||
execFileCalls.push({ command, args: [...args], timeout: options.timeout });
|
||||
queueMicrotask(() => callback(null, ""));
|
||||
});
|
||||
connectMock.mockReset();
|
||||
connectMock.mockImplementation(() => { throw new Error("unexpected direct TCP socket"); });
|
||||
log = vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
});
|
||||
|
||||
describe("rig host ls status for an ssh host", () => {
|
||||
it("dials the host and port ssh resolves for an ssh-config alias", async () => {
|
||||
const dials: string[] = [];
|
||||
const status = await probeHostStatus({ id: "devbox", transport: "ssh", target: "devbox" }, async (target, port) => {
|
||||
dials.push(`${target}:${port}`);
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
if (savedHome === undefined) delete process.env["OPENRIG_HOME"];
|
||||
else process.env["OPENRIG_HOME"] = savedHome;
|
||||
});
|
||||
|
||||
describe("rig host list/ls passive status", () => {
|
||||
it.each(["list", "ls"] as const)("reports an SSH host unknown without launching a process or probing TCP through host %s", async (verb) => {
|
||||
const probes: ProbeCall[] = [];
|
||||
const tcpProbe: DoctorDeps["tcpProbe"] = async (target, port, timeoutMs) => {
|
||||
probes.push({ target, port, timeoutMs });
|
||||
return "open";
|
||||
};
|
||||
|
||||
const rows = await listHosts([{ id: "devbox", transport: "ssh", target: "devbox", user: "ops" }], tcpProbe, verb);
|
||||
|
||||
expect({ status: rows[0]?.status, execFileCalls, probes, directSockets: connectMock.mock.calls }).toEqual({
|
||||
status: "unknown",
|
||||
execFileCalls: [],
|
||||
probes: [],
|
||||
directSockets: [],
|
||||
});
|
||||
expect(status).toBe("reachable");
|
||||
expect(dials).toEqual(["10.1.2.3:2222"]);
|
||||
});
|
||||
|
||||
it.each([["a ProxyJump host", "jumpy"], ["a target ssh can't resolve", "broken"]])(
|
||||
"reads unknown for %s, since a direct dial can't stand for the transport", async (_label, target) => {
|
||||
const dials: string[] = [];
|
||||
const status = await probeHostStatus({ id: target, transport: "ssh", target }, async (t, port) => {
|
||||
dials.push(`${t}:${port}`);
|
||||
return "closed";
|
||||
});
|
||||
expect(status).toBe("unknown");
|
||||
expect(dials).toEqual([]);
|
||||
it.each([
|
||||
{ label: "HTTP default port", url: "http://example.test", target: "example.test", port: 80 },
|
||||
{ label: "HTTP explicit port", url: "http://example.test:7433", target: "example.test", port: 7433 },
|
||||
{ label: "HTTPS default port", url: "https://example.test", target: "example.test", port: 443 },
|
||||
{ label: "bracketed IPv6 HTTPS", url: "https://[2001:db8::7]:9443", target: "2001:db8::7", port: 9443 },
|
||||
])("preserves CLI $label status probing through its injected seam", async ({ url, target, port }) => {
|
||||
const probes: ProbeCall[] = [];
|
||||
const tcpProbe: DoctorDeps["tcpProbe"] = async (host, servicePort, timeoutMs) => {
|
||||
probes.push({ target: host, port: servicePort, timeoutMs });
|
||||
return "open";
|
||||
};
|
||||
|
||||
const rows = await listHosts([{ id: "web", transport: "http", url }], tcpProbe);
|
||||
|
||||
expect({ status: rows[0]?.status, probes, execFileCalls, directSockets: connectMock.mock.calls }).toEqual({
|
||||
status: "reachable",
|
||||
probes: [{ target, port, timeoutMs: 1500 }],
|
||||
execFileCalls: [],
|
||||
directSockets: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("maps an HTTP closed-probe control to unreachable", async () => {
|
||||
const probes: ProbeCall[] = [];
|
||||
const tcpProbe: DoctorDeps["tcpProbe"] = async (target, port, timeoutMs) => {
|
||||
probes.push({ target, port, timeoutMs });
|
||||
return "closed";
|
||||
};
|
||||
|
||||
const rows = await listHosts([{ id: "web", transport: "http", url: "http://example.test:7433" }], tcpProbe);
|
||||
|
||||
expect({ status: rows[0]?.status, probes, directSockets: connectMock.mock.calls }).toEqual({
|
||||
status: "unreachable",
|
||||
probes: [{ target: "example.test", port: 7433, timeoutMs: 1500 }],
|
||||
directSockets: [],
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -14,11 +14,13 @@ import { configCommand } from "../src/commands/config.js";
|
||||
describe("rig host add/list", () => {
|
||||
let dir: string;
|
||||
let savedHome: string | undefined;
|
||||
let tcpProbeCalls: Array<{ target: string; port: number; timeoutMs: number }> = [];
|
||||
|
||||
beforeEach(() => {
|
||||
dir = fs.mkdtempSync(path.join(os.tmpdir(), "hostcmd-"));
|
||||
savedHome = process.env.OPENRIG_HOME;
|
||||
process.env.OPENRIG_HOME = dir;
|
||||
tcpProbeCalls = [];
|
||||
});
|
||||
afterEach(() => {
|
||||
if (savedHome === undefined) delete process.env.OPENRIG_HOME;
|
||||
@@ -43,7 +45,15 @@ describe("rig host add/list", () => {
|
||||
function run(argv: string[]) {
|
||||
const prog = new Command();
|
||||
prog.exitOverride();
|
||||
prog.addCommand(hostCommand());
|
||||
prog.addCommand(hostCommand({
|
||||
run: async () => { throw new Error("unexpected cross-host command in add/list test"); },
|
||||
httpGet: async () => { throw new Error("unexpected HTTP GET in add/list test"); },
|
||||
httpPost: async () => { throw new Error("unexpected HTTP POST in add/list test"); },
|
||||
tcpProbe: async (target, port, timeoutMs) => {
|
||||
tcpProbeCalls.push({ target, port, timeoutMs });
|
||||
return "unknown";
|
||||
},
|
||||
}));
|
||||
return prog.parseAsync(["node", "rig", "host", ...argv]);
|
||||
}
|
||||
|
||||
@@ -97,11 +107,17 @@ describe("rig host add/list", () => {
|
||||
const added = await capture(() => run(["add", "--id", "f1", "--transport", "http", "--url", "http://100.64.0.9:7433", "--bearer-env", "FACTORY_TOKEN", "--json"]));
|
||||
expect(JSON.parse(added.out.join("\n"))).toMatchObject({ ok: true, entry: { id: "f1" } });
|
||||
expect(added.err).toEqual([]);
|
||||
const beforeTextList = tcpProbeCalls.length;
|
||||
const { out } = await capture(() => run(["list"]));
|
||||
expect(tcpProbeCalls.length - beforeTextList).toBe(1);
|
||||
const text = out.join("\n");
|
||||
expect(text).toContain("env:FACTORY_TOKEN");
|
||||
expect(text).toContain("unknown");
|
||||
expect(text).not.toContain("super-secret-bearer-value");
|
||||
const beforeJsonList = tcpProbeCalls.length;
|
||||
const { out: jsonOut } = await capture(() => run(["list", "--json"]));
|
||||
expect(tcpProbeCalls.length - beforeJsonList).toBe(1);
|
||||
expect(JSON.parse(jsonOut.join(""))).toMatchObject([{ id: "f1", status: "unknown" }]);
|
||||
expect(jsonOut.join("")).toContain("FACTORY_TOKEN");
|
||||
expect(jsonOut.join("")).not.toContain("super-secret-bearer-value");
|
||||
});
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
// `rig network connect | status | disconnect` against the daemon's real network routes.
|
||||
import { afterEach, beforeEach, expect, it, vi } from "vitest";
|
||||
import { Command } from "commander";
|
||||
import { Hono } from "hono";
|
||||
import Database from "better-sqlite3";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { networkCommand, type NetworkDeps } from "../src/commands/network.js";
|
||||
import { STATE_FILE, type LifecycleDeps } from "../src/daemon-lifecycle.js";
|
||||
import { ALL_MIGRATIONS } from "../../daemon/src/db/all-migrations.js";
|
||||
import { migrate } from "../../daemon/src/db/migrate.js";
|
||||
import { RigRepository } from "../../daemon/src/domain/rig-repository.js";
|
||||
import { networkRoutes } from "../../daemon/src/routes/network.js";
|
||||
|
||||
let db: Database.Database;
|
||||
let home: string;
|
||||
let app: Hono;
|
||||
let rigs: RigRepository;
|
||||
const routesWith = (bearerToken: string | null) => {
|
||||
const next = new Hono();
|
||||
next.use("*", async (c, n) => { c.set("rigRepo" as never, rigs as never); await n(); });
|
||||
next.route("/api/network", networkRoutes({ home, bearerToken }));
|
||||
return next;
|
||||
};
|
||||
beforeEach(() => {
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
rigs = new RigRepository(db);
|
||||
const node = rigs.addNode(rigs.createRig("lab").id, "impl", { runtime: "claude-code" });
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES ('s1', ?, 'dev-impl@lab', 'running')").run(node.id);
|
||||
home = mkdtempSync(join(tmpdir(), "openrig-network-cli-"));
|
||||
app = routesWith(null);
|
||||
});
|
||||
afterEach(() => { db.close(); rmSync(home, { recursive: true, force: true }); vi.restoreAllMocks(); vi.unstubAllEnvs(); process.exitCode = undefined; });
|
||||
|
||||
function cli(tokenFile: Record<string, string> = {}) {
|
||||
const lifecycleDeps: LifecycleDeps = {
|
||||
spawn: vi.fn(), fetch: vi.fn(async () => ({ ok: true })), kill: vi.fn(),
|
||||
readFile: (p) => p === STATE_FILE ? JSON.stringify({ pid: 123, port: 19731, db: "/fixture/test.sqlite", startedAt: new Date().toISOString() }) : null,
|
||||
writeFile: vi.fn(), removeFile: vi.fn(), exists: (p) => p === STATE_FILE, mkdirp: vi.fn(), openForAppend: () => 3, isProcessAlive: () => true,
|
||||
} as LifecycleDeps;
|
||||
const call = async (method: "GET" | "POST", url: string, body?: unknown, headers: Record<string, string> = {}) => {
|
||||
const res = await app.request(url, { method, headers: { "Content-Type": "application/json", ...headers }, ...(body === undefined ? {} : { body: JSON.stringify(body) }) });
|
||||
return { status: res.status, data: await res.json() };
|
||||
};
|
||||
const deps: NetworkDeps = {
|
||||
lifecycleDeps,
|
||||
clientFactory: () => ({ get: (url: string) => call("GET", url), post: (url: string, body: unknown, options?: { headers?: Record<string, string> }) => call("POST", url, body, options?.headers) }) as never,
|
||||
readTokenFile: (path) => { if (!(path in tokenFile)) throw new Error("no such file"); return tokenFile[path]!; },
|
||||
};
|
||||
return new Command().exitOverride().addCommand(networkCommand(deps));
|
||||
}
|
||||
|
||||
it("connects a seat with its token file, shows it in status without the token, and disconnects it", async () => {
|
||||
const out = vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
await cli({ "/tok": "tok-abc" }).parseAsync(["node", "rig", "network", "connect", "dev-impl@lab", "--hub", "https://hub.example/mcp", "--as", "alex@team", "--token-file", "/tok"]);
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Connected netb-[0-9a-f]{20} dev-impl@lab -> alex@team at https:\/\/hub\.example\/mcp no contact yet/);
|
||||
|
||||
await cli().parseAsync(["node", "rig", "network", "status", "--json"]);
|
||||
const status = String(out.mock.calls.at(-1)?.[0]);
|
||||
expect(JSON.parse(status).bindings).toEqual([expect.objectContaining({ localSession: "dev-impl@lab", hubSeat: "alex@team", state: "active" })]);
|
||||
expect(status).not.toContain("tok-abc");
|
||||
|
||||
await cli().parseAsync(["node", "rig", "network", "disconnect", "dev-impl@lab", "--reason", "done"]);
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Disconnected netb-[0-9a-f]{20} dev-impl@lab -> alex@team .*disconnected \(done\)/);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
});
|
||||
|
||||
it("names the daemon's refusal and exits 1, and an unreadable token file sends nothing", async () => {
|
||||
const err = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
await cli({ "/tok": "tok-abc" }).parseAsync(["node", "rig", "network", "connect", "nobody@lab", "--hub", "https://hub.example/mcp", "--as", "alex@team", "--token-file", "/tok"]);
|
||||
expect(String(err.mock.calls.at(-1)?.[0])).toBe("No local seat is named 'nobody@lab'. (seat_not_found)");
|
||||
expect(process.exitCode).toBe(1);
|
||||
process.exitCode = undefined;
|
||||
await cli().parseAsync(["node", "rig", "network", "connect", "dev-impl@lab", "--hub", "https://hub.example/mcp", "--as", "alex@team", "--token-file", "/missing"]);
|
||||
expect(String(err.mock.calls.at(-1)?.[0])).toBe("Can't read the token file: no such file");
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect((await app.request("/api/network/bindings").then((r) => r.json()) as { bindings: unknown[] }).bindings).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("a plain-http hub needs --allow-http: refused with the remedy, then connected and shown unencrypted in status and JSON", async () => {
|
||||
const out = vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
const err = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
const args = ["node", "rig", "network", "connect", "dev-impl@lab", "--hub", "http://192.0.2.7:7433/mcp", "--as", "alex@team", "--token-file", "/tok"];
|
||||
await cli({ "/tok": "tok-abc" }).parseAsync(args);
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(String(err.mock.calls.at(-1)?.[0])).toMatch(/plain http, .*unencrypted.*--allow-http to accept that for this binding\. \(http_not_allowed\)$/);
|
||||
process.exitCode = undefined;
|
||||
|
||||
await cli({ "/tok": "tok-abc" }).parseAsync([...args, "--allow-http"]);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Connected netb-[0-9a-f]{20} dev-impl@lab -> alex@team at http:\/\/192\.0\.2\.7:7433\/mcp UNENCRYPTED \(http, allowed for this binding\)/);
|
||||
await cli().parseAsync(["node", "rig", "network", "status"]);
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toContain("UNENCRYPTED (http, allowed for this binding)");
|
||||
await cli().parseAsync(["node", "rig", "network", "status", "--json"]);
|
||||
const json = String(out.mock.calls.at(-1)?.[0]);
|
||||
expect(JSON.parse(json).bindings).toEqual([expect.objectContaining({ hubUrl: "http://192.0.2.7:7433/mcp", allowHttp: true, unencrypted: true })]);
|
||||
expect(json).not.toContain("tok-abc");
|
||||
});
|
||||
|
||||
it("status shows a hub item closed elsewhere under its binding, in text and JSON", async () => {
|
||||
const out = vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
await cli({ "/tok": "tok-abc" }).parseAsync(["node", "rig", "network", "connect", "dev-impl@lab", "--hub", "https://hub.example/mcp", "--as", "alex@team", "--token-file", "/tok"]);
|
||||
const bindingId = String(out.mock.calls.at(-1)?.[0]).match(/netb-[0-9a-f]{20}/)![0];
|
||||
db.prepare(`INSERT INTO network_receipts (local_qitem_id, binding_id, hub_qitem_id, created_at, settled_at, outcome, last_error)
|
||||
VALUES ('qitem-net-0001', ?, 'hub-9', '2026-11-11T19:00:00Z', '2026-11-11T19:01:00Z', 'closed_on_hub', 'closed elsewhere (done, canceled)')`).run(bindingId);
|
||||
await cli().parseAsync(["node", "rig", "network", "status"]);
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toContain("\n hub item hub-9 was closed elsewhere before its result arrived; the result stays on qitem-net-0001");
|
||||
await cli().parseAsync(["node", "rig", "network", "status", "--json"]);
|
||||
expect(JSON.parse(String(out.mock.calls.at(-1)?.[0])).bindings[0].closedElsewhere).toEqual([{ hubItem: "hub-9", localItem: "qitem-net-0001", note: "closed elsewhere (done, canceled)" }]);
|
||||
});
|
||||
|
||||
it("sends the operator bearer on its writes, so a daemon with one configured takes them", async () => {
|
||||
app = routesWith("op-secret");
|
||||
const out = vi.spyOn(console, "log").mockImplementation(() => {});
|
||||
const err = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||
const connectArgs = ["node", "rig", "network", "connect", "dev-impl@lab", "--hub", "https://hub.example/mcp", "--as", "alex@team", "--token-file", "/tok"];
|
||||
vi.stubEnv("OPENRIG_AUTH_BEARER_TOKEN", "wrong");
|
||||
await cli({ "/tok": "tok-abc" }).parseAsync(connectArgs);
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(String(err.mock.calls.at(-1)?.[0])).toContain("bearer token does not match");
|
||||
process.exitCode = undefined;
|
||||
vi.stubEnv("OPENRIG_AUTH_BEARER_TOKEN", "op-secret");
|
||||
await cli({ "/tok": "tok-abc" }).parseAsync(connectArgs);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Connected netb-/);
|
||||
await cli().parseAsync(["node", "rig", "network", "disconnect", "dev-impl@lab"]);
|
||||
expect(process.exitCode).toBeUndefined();
|
||||
expect(String(out.mock.calls.at(-1)?.[0])).toMatch(/^Disconnected netb-/);
|
||||
});
|
||||
@@ -0,0 +1,159 @@
|
||||
// The outbound connector against a real hub: the daemon's routes behind `rig mcp serve --http` with a grant per
|
||||
// participant, reached by the connector's own MCP client. A second, local queue takes the work, and what happens there
|
||||
// comes back to the hub item: queued, picked up, the result, and a revoked grant ending the binding.
|
||||
import { afterEach, beforeEach, expect, it } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import { serve, type ServerType } from "@hono/node-server";
|
||||
import type Database from "better-sqlite3";
|
||||
import { createTestApp } from "../../daemon/test/helpers/test-app.js";
|
||||
import { createDb } from "../../daemon/src/db/connection.js";
|
||||
import { migrate } from "../../daemon/src/db/migrate.js";
|
||||
import { ALL_MIGRATIONS } from "../../daemon/src/db/all-migrations.js";
|
||||
import { EventBus } from "../../daemon/src/domain/event-bus.js";
|
||||
import { QueueRepository } from "../../daemon/src/domain/queue-repository.js";
|
||||
import { OutboxHandler } from "../../daemon/src/domain/outbox-handler.js";
|
||||
import { RigRepository } from "../../daemon/src/domain/rig-repository.js";
|
||||
import { NetworkBindingStore } from "../../daemon/src/domain/network/network-bindings.js";
|
||||
import { networkClosedElsewhere, runNetworkBindingPass, type NetworkConnectorDeps } from "../../daemon/src/domain/network/network-connector.js";
|
||||
import { addGrant, revokeGrant, startMcpHttpServer } from "../src/mcp-http.js";
|
||||
|
||||
let hubDb: Database.Database;
|
||||
let localDb: Database.Database;
|
||||
let hubHome: string;
|
||||
let localHome: string;
|
||||
let hubRepo: QueueRepository;
|
||||
let daemon: ServerType;
|
||||
let hubUrl: string;
|
||||
let tokenFile: string;
|
||||
const closers: Array<() => Promise<void>> = [];
|
||||
const saved = { home: process.env.OPENRIG_HOME, session: process.env.OPENRIG_SESSION_NAME };
|
||||
|
||||
const queue = (db: Database.Database) => {
|
||||
const repo = new QueueRepository(db, new EventBus(db), { validateRig: () => true, loadHumanRegistry: () => ({ ok: true as const, entities: [] }) });
|
||||
repo.attachOutbox(new OutboxHandler(db));
|
||||
return repo;
|
||||
};
|
||||
|
||||
beforeEach(async () => {
|
||||
hubDb = createDb(":memory:");
|
||||
migrate(hubDb, ALL_MIGRATIONS);
|
||||
hubHome = fs.mkdtempSync(path.join(os.tmpdir(), "network-hub-"));
|
||||
localHome = fs.mkdtempSync(path.join(os.tmpdir(), "network-local-"));
|
||||
process.env.OPENRIG_HOME = hubHome;
|
||||
process.env.OPENRIG_SESSION_NAME = "env-seat@elsewhere";
|
||||
tokenFile = path.join(hubHome, "mcp-http-tokens.json");
|
||||
hubRepo = queue(hubDb);
|
||||
const inert = async () => ({ ok: false, error: "inert fixture" });
|
||||
const { app } = createTestApp(hubDb, { appDeps: {
|
||||
sessionTransport: { resolveSessions: async () => ({ ok: false, code: "not_found", error: "inert fixture" }), send: inert, capture: inert, broadcast: inert } as never,
|
||||
queueRepo: hubRepo,
|
||||
} });
|
||||
daemon = await new Promise<ServerType>((resolve) => { const s = serve({ fetch: app.fetch, port: 0, hostname: "127.0.0.1" }, () => resolve(s)); });
|
||||
const served = await startMcpHttpServer({ daemonUrl: `http://127.0.0.1:${(daemon.address() as AddressInfo).port}`, host: "127.0.0.1", port: 0, tokenFile });
|
||||
closers.push(() => served.close());
|
||||
hubUrl = served.url;
|
||||
|
||||
localDb = createDb(":memory:");
|
||||
migrate(localDb, ALL_MIGRATIONS);
|
||||
const rigs = new RigRepository(localDb);
|
||||
const node = rigs.addNode(rigs.createRig("lab").id, "impl", { runtime: "claude-code" });
|
||||
localDb.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES ('s1', ?, 'dev-impl@lab', 'running')").run(node.id);
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
for (const close of closers.splice(0).reverse()) await close();
|
||||
await new Promise<void>((resolve) => daemon.close(() => resolve()));
|
||||
hubDb.close();
|
||||
localDb.close();
|
||||
fs.rmSync(hubHome, { recursive: true, force: true });
|
||||
fs.rmSync(localHome, { recursive: true, force: true });
|
||||
if (saved.home === undefined) delete process.env.OPENRIG_HOME; else process.env.OPENRIG_HOME = saved.home;
|
||||
if (saved.session === undefined) delete process.env.OPENRIG_SESSION_NAME; else process.env.OPENRIG_SESSION_NAME = saved.session;
|
||||
});
|
||||
|
||||
it("takes the participant's item over the wire, carries pickup and the result back, and a revoked grant ends the binding", async () => {
|
||||
const token = await addGrant(tokenFile, { name: "alex", seat: "alex@team", rigs: ["team"], roots: [], workspace: false, owner: false });
|
||||
// Longer than a list preview, with its end marked, so a shortened body would show.
|
||||
const body = `Summarize the release notes. ${"Context. ".repeat(400)}END-OF-BODY`;
|
||||
const task = await hubRepo.create({ sourceSession: "lead@team", destinationSession: "alex@team", summary: "Summarize the notes", body, nudge: false } as never);
|
||||
|
||||
const localRepo = queue(localDb);
|
||||
const bindings = new NetworkBindingStore(localDb, localHome);
|
||||
// The test hub listens on loopback http, so the binding records the person's --allow-http choice.
|
||||
const binding = bindings.create({ session: "dev-impl@lab", hubUrl, hubSeat: "alex@team", token, allowHttp: true });
|
||||
const deps: NetworkConnectorDeps = { db: localDb, queueRepo: localRepo, bindings, home: localHome };
|
||||
const pass = () => runNetworkBindingPass(deps, bindings.get(binding.id)!);
|
||||
const notes = () => hubRepo.listTransitions(task.qitemId).map((t) => t.transitionNote).filter(Boolean);
|
||||
|
||||
const first = await pass();
|
||||
expect(first.error).toBeUndefined();
|
||||
expect(first).toMatchObject({ taken: 1 });
|
||||
const [local] = localRepo.list({ limit: 50 });
|
||||
expect(local).toMatchObject({ sourceSession: "daemon@kernel", destinationSession: "dev-impl@lab", summary: "From lead@team via network hub: Summarize the notes", state: "pending" });
|
||||
expect(local!.body!.split("\n")[0]).toBe(`From lead@team through the network hub ${hubUrl}, as alex@team (hub item ${task.qitemId}).`);
|
||||
expect(local!.body).toContain(body);
|
||||
expect(hubRepo.getById(task.qitemId)!.state).toBe("in-progress");
|
||||
expect(notes().at(-1)).toBe(`Queued for the connected local seat as ${local!.qitemId}. Not picked up yet.`);
|
||||
expect(hubRepo.listTransitions(task.qitemId).find((t) => t.state === "in-progress")?.actorSession).toMatch(/^alex@team/);
|
||||
|
||||
expect(await pass()).toMatchObject({ taken: 0, pickupsSent: 0 });
|
||||
expect(localRepo.list({ limit: 50 })).toHaveLength(1);
|
||||
|
||||
localRepo.claim({ qitemId: local!.qitemId, destinationSession: "dev-impl@lab" });
|
||||
expect(await pass()).toMatchObject({ pickupsSent: 1 });
|
||||
expect(notes().at(-1)).toMatch(/^Picked up by the local seat at /);
|
||||
expect(hubRepo.getById(task.qitemId)!.state).toBe("in-progress");
|
||||
|
||||
await localRepo.update({ qitemId: local!.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "Three changes; none breaking." });
|
||||
expect(await pass()).toMatchObject({ resultsSent: 1 });
|
||||
expect(hubRepo.getById(task.qitemId)).toMatchObject({ state: "done", resolution: "Three changes; none breaking." });
|
||||
expect(bindings.get(binding.id)!.lastError).toBeNull();
|
||||
|
||||
// A failure and a cancel close the hub item the way its queue takes them: failed, and done with closure reason canceled.
|
||||
const failing = await hubRepo.create({ sourceSession: "lead@team", destinationSession: "alex@team", summary: "Build it", body: "Build the release.", nudge: false } as never);
|
||||
const dropped = await hubRepo.create({ sourceSession: "lead@team", destinationSession: "alex@team", summary: "Skip it", body: "Maybe not.", nudge: false } as never);
|
||||
expect(await pass()).toMatchObject({ taken: 2 });
|
||||
const byHub = (id: string) => localRepo.list({ limit: 50 }).find((i) => (i.tags ?? []).includes(`network-hub-item:${id}`))!;
|
||||
await localRepo.update({ qitemId: byHub(failing.qitemId).qitemId, actorSession: "dev-impl@lab", state: "failed", transitionNote: "The build broke." });
|
||||
await localRepo.update({ qitemId: byHub(dropped.qitemId).qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "canceled", resolution: "Not needed." });
|
||||
expect(await pass()).toMatchObject({ resultsSent: 2 });
|
||||
expect(hubRepo.getById(failing.qitemId)).toMatchObject({ state: "failed", resolution: "The build broke." });
|
||||
expect(hubRepo.getById(dropped.qitemId)).toMatchObject({ state: "done", closureReason: "canceled", resolution: "Not needed." });
|
||||
|
||||
// Two items the requester fails on the hub while they're queued here. Our failed and done results then aren't ours to
|
||||
// deliver: the hub items keep their failure, and the receipts say closed elsewhere.
|
||||
const theirs = [
|
||||
await hubRepo.create({ sourceSession: "lead@team", destinationSession: "alex@team", summary: "One", body: "First.", nudge: false } as never),
|
||||
await hubRepo.create({ sourceSession: "lead@team", destinationSession: "alex@team", summary: "Two", body: "Second.", nudge: false } as never),
|
||||
];
|
||||
expect(await pass()).toMatchObject({ taken: 2 });
|
||||
for (const item of theirs) await hubRepo.update({ qitemId: item.qitemId, actorSession: "lead@team", state: "failed", resolution: "Dropped by the requester." } as never);
|
||||
await localRepo.update({ qitemId: byHub(theirs[0]!.qitemId).qitemId, actorSession: "dev-impl@lab", state: "failed", transitionNote: "Could not finish." });
|
||||
await localRepo.update({ qitemId: byHub(theirs[1]!.qitemId).qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "Finished it." });
|
||||
expect(await pass()).toMatchObject({ resultsSent: 0 });
|
||||
for (const item of theirs) expect(hubRepo.getById(item.qitemId)).toMatchObject({ state: "failed", resolution: "Dropped by the requester." });
|
||||
expect(localDb.prepare("SELECT DISTINCT outcome FROM network_receipts WHERE hub_qitem_id IN (?, ?)").all(theirs[0]!.qitemId, theirs[1]!.qitemId)).toEqual([{ outcome: "closed_on_hub" }]);
|
||||
|
||||
// An operator on the hub hands the original on to another seat while it's queued here. It's closed to the connector:
|
||||
// nothing is written to it, the original and its successor are unchanged, status shows it, and it isn't retried.
|
||||
const handed = await hubRepo.create({ sourceSession: "lead@team", destinationSession: "alex@team", summary: "Three", body: "Third.", nudge: false } as never);
|
||||
expect(await pass()).toMatchObject({ taken: 1 });
|
||||
const { created: successor } = await hubRepo.handoff({ qitemId: handed.qitemId, fromSession: "lead@team", toSession: "other@team", nudge: false });
|
||||
const before = hubRepo.listTransitions(handed.qitemId).length;
|
||||
await localRepo.update({ qitemId: byHub(handed.qitemId).qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "Done here." });
|
||||
expect(await pass()).toMatchObject({ resultsSent: 0 });
|
||||
expect(hubRepo.getById(handed.qitemId)!.state).toBe("handed-off");
|
||||
expect(hubRepo.listTransitions(handed.qitemId)).toHaveLength(before);
|
||||
expect(hubRepo.getById(successor.qitemId)).toMatchObject({ state: "pending", destinationSession: "other@team" });
|
||||
expect(networkClosedElsewhere(localDb, binding.id)).toContainEqual(expect.objectContaining({ hubItem: handed.qitemId, localItem: byHub(handed.qitemId).qitemId,
|
||||
note: expect.stringContaining("closed elsewhere (handed-off") }));
|
||||
await pass();
|
||||
expect(hubRepo.listTransitions(handed.qitemId)).toHaveLength(before);
|
||||
|
||||
await revokeGrant(tokenFile, "alex");
|
||||
expect(await pass()).toMatchObject({ error: "revoked" });
|
||||
expect(bindings.get(binding.id)!.state).toBe("revoked");
|
||||
});
|
||||
@@ -7,7 +7,7 @@
|
||||
"url": "https://github.com/mvschwarz"
|
||||
},
|
||||
"homepage": "https://github.com/mvschwarz/openrig-plugins",
|
||||
"repository": "github:mvschwarz/openrig-plugins",
|
||||
"repository": "https://github.com/mvschwarz/openrig-plugins",
|
||||
"license": "Apache-2.0",
|
||||
"keywords": ["openrig", "agent", "topology", "rig", "continuity", "coordination", "compaction", "handoff"],
|
||||
"skills": "./skills",
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
"url": "https://github.com/mvschwarz"
|
||||
},
|
||||
"homepage": "https://github.com/mvschwarz/openrig-plugins",
|
||||
"repository": "github:mvschwarz/openrig-plugins",
|
||||
"repository": "https://github.com/mvschwarz/openrig-plugins",
|
||||
"license": "Apache-2.0",
|
||||
"keywords": ["openrig", "agent", "topology", "rig", "continuity", "coordination", "compaction", "handoff"],
|
||||
"skills": "./skills",
|
||||
|
||||
@@ -50,6 +50,12 @@ This plugin ships dual-manifest packaging — both `.claude-plugin/` and `.codex
|
||||
no-ops. The plugin supplies refocus and skills. Disabling an inline activity hook is not bypassed by the plugin.
|
||||
- **Seat scope:** the commands require both `OPENRIG_SEAT_PROJECT_DIR` and the executable `OPENRIG_NODE`. Personal and
|
||||
unselected sessions are no-ops. A nested Codex inheriting those seat variables still acts as that seat.
|
||||
- **Refocus output:** the Codex prompt handler keeps its unchanged declaration and default 2,500-token budget
|
||||
(10,000 UTF-8 bytes), preserving its native trust identity. Larger packets
|
||||
send a prefix and an explicit shortening notice for the current attempt. After a claimed, successful render/write
|
||||
and state save, the refocus record lists that attempt's omitted tail byte count and full packet's `.context.txt`
|
||||
path, or says the full text could not be saved. A trace/content failure or unavailable final state guard/write can
|
||||
leave older metadata or no record. The metadata describes the last successfully recorded attempt, not model receipt.
|
||||
|
||||
Generated Claude settings commands for the activity relay and context collector use the daemon's absolute Node
|
||||
executable when its name is recognized (`node` or `node.exe`); other names keep the legacy bare `node` command and
|
||||
|
||||
@@ -144,10 +144,134 @@ function buildOpenRigPayload(providerPayload, env = process.env, now = () => new
|
||||
...promptBoundaryFields(hookEvent, providerPayload),
|
||||
occurredAt: now().toISOString(),
|
||||
relayRoute: argv.includes("--from-plugin") ? "plugin" : "managed",
|
||||
...limitResetField(runtime, hookEvent, subtype, providerPayload, now),
|
||||
...conversationFields(providerPayload, env),
|
||||
};
|
||||
}
|
||||
|
||||
// #99: a Claude usage limit says when it resets only in its rendered text ("You've hit your weekly limit ·
|
||||
// resets Nov 10, 4pm (America/Sao_Paulo)"). The typed error still decides THAT the seat is limited; the text
|
||||
// is read here, on the seat's machine and in its time zone, only for WHEN, and only the instant leaves the
|
||||
// relay: never the text itself.
|
||||
function limitResetField(runtime, hookEvent, subtype, providerPayload, now) {
|
||||
if (runtime !== "claude-code" || hookEvent !== "StopFailure" || subtype !== "rate_limit") return {};
|
||||
for (const value of [providerPayload.last_assistant_message, providerPayload.error_details]) {
|
||||
if (typeof value !== "string") continue;
|
||||
const resetsAt = claudeLimitResetAt(value.slice(0, 2000), now());
|
||||
if (resetsAt) return { resetsAt };
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
const RESET_MAX_MS = 8 * 24 * 3600 * 1000;
|
||||
const MONTHS = ["jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec"];
|
||||
const RESET_EPOCH_RE = /usage limit reached\|(\d{10})\b/i;
|
||||
// Each unit must be a whole word ("2 months" is not "2 m"), and a "resets in" fragment is read only this way.
|
||||
const RESET_IN_LEAD_RE = /\bresets?\s+in\b/i;
|
||||
const RESET_IN_RE = /\bresets?\s+in\s+(?:(\d+)\s*(?:d|days?)\b\s*)?(?:(\d+)\s*(?:h|hrs?|hours?)\b\s*)?(?:(\d+)\s*(?:m|mins?|minutes?)\b)?/i;
|
||||
// A zone, when the text opens one, must be closed: "4pm (Not/AZone" is truncated, not local time.
|
||||
const RESET_AT_RE = /\bresets?\s+(?:at\s+|on\s+)?(?:([a-z]{3})[a-z]*\.?\s+(\d{1,2}),?\s+(?:at\s+)?)?(\d{1,2})(?::(\d{2}))?\s*(am|pm)\b(?:\s*(\()([^)\n]{0,64})(\))?)?/i;
|
||||
|
||||
/** The reset instant a Claude usage-limit message states, as an ISO string, or null when it states none
|
||||
* this can read. A wall-clock time without a zone is the seat's local time, which is what Claude renders. */
|
||||
function claudeLimitResetAt(text, nowDate) {
|
||||
const nowMs = nowDate.getTime();
|
||||
if (!Number.isFinite(nowMs) || typeof text !== "string") return null;
|
||||
let resetMs = null;
|
||||
const epoch = RESET_EPOCH_RE.exec(text);
|
||||
if (epoch) {
|
||||
resetMs = Number(epoch[1]) * 1000;
|
||||
} else if (RESET_IN_LEAD_RE.test(text)) {
|
||||
const relative = RESET_IN_RE.exec(text);
|
||||
// The whole fragment must be read: nothing numeric may follow what was, across a separator either
|
||||
// ("in 2h 30x", "in 2h, 30x" and "in 2h and 30x" are not two hours).
|
||||
if (!relative || !(relative[1] || relative[2] || relative[3])
|
||||
|| /^(?:\s|[,;&+]|and\b)*\d/i.test(text.slice(relative.index + relative[0].length))) return null;
|
||||
resetMs = nowMs + ((Number(relative[1] || 0) * 24 + Number(relative[2] || 0)) * 60 + Number(relative[3] || 0)) * 60_000;
|
||||
} else {
|
||||
const wall = RESET_AT_RE.exec(text);
|
||||
if (wall) resetMs = wallClockReset(wall, nowMs);
|
||||
}
|
||||
if (resetMs === null || !Number.isFinite(resetMs) || resetMs <= nowMs || resetMs > nowMs + RESET_MAX_MS) return null;
|
||||
return new Date(resetMs).toISOString();
|
||||
}
|
||||
|
||||
function wallClockReset(match, nowMs) {
|
||||
const [, monthName, day, hourText, minuteText, meridiem, zoneOpen, zoneText, zoneClose] = match;
|
||||
if (zoneOpen && (!zoneClose || !zoneText.trim())) return null; // a truncated or empty zone: no reset
|
||||
const hour12 = Number(hourText);
|
||||
const minute = Number(minuteText || 0);
|
||||
if (hour12 < 1 || hour12 > 12 || minute > 59) return null;
|
||||
const hour = (hour12 % 12) + (meridiem.toLowerCase() === "pm" ? 12 : 0);
|
||||
const zone = zoneText ? zoneText.trim() : null;
|
||||
let toUtc;
|
||||
try {
|
||||
toUtc = zone ? zonedClock(zone) : localClock();
|
||||
} catch {
|
||||
return null; // a zone this runtime doesn't know: no reset rather than a guessed one
|
||||
}
|
||||
const today = toUtc.partsAt(nowMs);
|
||||
if (monthName) {
|
||||
const month = MONTHS.indexOf(monthName.toLowerCase());
|
||||
if (month < 0) return null;
|
||||
for (const year of [today.year, today.year + 1]) {
|
||||
if (!isCalendarDay(year, month, Number(day))) continue; // "Feb 30" names no day; never roll it into March
|
||||
const at = toUtc.instant(year, month, Number(day), hour, minute);
|
||||
if (Number.isNaN(at)) return null;
|
||||
if (at > nowMs) return at;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
const at = toUtc.instant(today.year, today.month, today.day, hour, minute);
|
||||
if (Number.isNaN(at)) return null; // ambiguous today: never advance a day on a guess
|
||||
return at > nowMs ? at : toUtc.instant(today.year, today.month, today.day + 1, hour, minute);
|
||||
}
|
||||
|
||||
function isCalendarDay(year, month, day) {
|
||||
return Number.isInteger(day) && day >= 1 && day <= new Date(Date.UTC(year, month + 1, 0)).getUTCDate();
|
||||
}
|
||||
|
||||
function localClock() {
|
||||
return wallClock(
|
||||
(ms) => { const d = new Date(ms); return { year: d.getFullYear(), month: d.getMonth(), day: d.getDate(), hour: d.getHours(), minute: d.getMinutes() }; },
|
||||
(ms) => -new Date(ms).getTimezoneOffset() * 60_000,
|
||||
);
|
||||
}
|
||||
|
||||
function zonedClock(timeZone) {
|
||||
const format = new Intl.DateTimeFormat("en-US", { timeZone, hourCycle: "h23", year: "numeric", month: "numeric",
|
||||
day: "numeric", hour: "numeric", minute: "numeric", second: "numeric" });
|
||||
const partsAt = (ms) => {
|
||||
const parts = Object.fromEntries(format.formatToParts(new Date(ms)).map((part) => [part.type, Number(part.value)]));
|
||||
return { year: parts.year, month: parts.month - 1, day: parts.day, hour: parts.hour, minute: parts.minute, second: parts.second };
|
||||
};
|
||||
const offsetAt = (ms) => {
|
||||
const p = partsAt(ms);
|
||||
return Date.UTC(p.year, p.month, p.day, p.hour, p.minute, p.second) - Math.floor(ms / 1000) * 1000;
|
||||
};
|
||||
return wallClock(partsAt, offsetAt);
|
||||
}
|
||||
|
||||
/** A clock whose instant() is the one moment a wall time names, or NaN when it names none (skipped by a DST jump) or
|
||||
* two (repeated by a DST fall-back): the message doesn't say which, so no reset rather than a guess a day off. */
|
||||
function wallClock(partsAt, offsetAt) {
|
||||
return {
|
||||
partsAt,
|
||||
instant: (year, month, day, hour, minute) => {
|
||||
const asUtc = Date.UTC(year, month, day, hour, minute);
|
||||
const want = new Date(asUtc); // normalizes a day past the month's end (tomorrow)
|
||||
const matches = new Set();
|
||||
for (const probe of [asUtc - 14 * 3600_000, asUtc, asUtc + 14 * 3600_000]) {
|
||||
const candidate = asUtc - offsetAt(probe);
|
||||
const p = partsAt(candidate);
|
||||
if (p.year === want.getUTCFullYear() && p.month === want.getUTCMonth() && p.day === want.getUTCDate()
|
||||
&& p.hour === want.getUTCHours() && p.minute === want.getUTCMinutes()) matches.add(candidate);
|
||||
}
|
||||
return matches.size === 1 ? [...matches][0] : NaN;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// OPR.0.4.3.28 B1+B3 — resolve the ingest base URL + token without depending on
|
||||
// the operator seeding OPENRIG_URL/OPENRIG_ACTIVITY_HOOK_TOKEN into the shell:
|
||||
// 1. env OPENRIG_URL / token (unchanged fast path).
|
||||
@@ -390,6 +514,7 @@ if (require.main === module) {
|
||||
|
||||
module.exports = {
|
||||
buildOpenRigPayload,
|
||||
claudeLimitResetAt,
|
||||
claimFirstLaunchHook,
|
||||
buildSessionIdentityPayload,
|
||||
parseJson,
|
||||
|
||||
@@ -630,10 +630,30 @@ function runsForThisSeatsProject(env = process.env) {
|
||||
"",
|
||||
...body,
|
||||
]).join("\n");
|
||||
let additionalContext = payload;
|
||||
let contextOutput;
|
||||
if (runtime() === "codex") {
|
||||
// Keep the handler declaration unchanged; Codex's default is 2,500 tokens, ceil(UTF-8 bytes / 4).
|
||||
const limitTokens = 2_500;
|
||||
const bytes = Buffer.from(payload);
|
||||
contextOutput = { status: "full", limitTokens, originalBytes: bytes.length, outputBytes: bytes.length, omittedBytes: 0 };
|
||||
if (bytes.length > limitTokens * 4) {
|
||||
let fullPath = `${stateFile}.${crypto.randomUUID()}.context.txt`;
|
||||
try { fs.writeFileSync(fullPath, payload, { flag: "wx", mode: 0o600 }); }
|
||||
catch { fullPath = null; }
|
||||
const location = fullPath ? `Full text saved to: ${fullPath}` : "Full text could not be saved.";
|
||||
// Reserve the notice before taking the prefix; streaming decode drops an incomplete final UTF-8 character.
|
||||
const prefix = new TextDecoder().decode(bytes.subarray(0, Math.max(0, limitTokens * 4 - Buffer.byteLength(location) - 256)), { stream: true });
|
||||
const prefixBytes = Buffer.byteLength(prefix);
|
||||
const omittedBytes = bytes.length - prefixBytes;
|
||||
additionalContext = `${prefix}\n\n[REFOCUS SHORTENED: omitted ${omittedBytes} trailing UTF-8 bytes from the assembled trace, role and standing instructions. ${location}]`;
|
||||
contextOutput = { status: "shortened", limitTokens, originalBytes: bytes.length, outputBytes: Buffer.byteLength(additionalContext), prefixBytes, omittedBytes, fullPath };
|
||||
}
|
||||
}
|
||||
const output = JSON.stringify({
|
||||
hookSpecificOutput: {
|
||||
hookEventName: "UserPromptSubmit",
|
||||
additionalContext: payload,
|
||||
additionalContext,
|
||||
},
|
||||
});
|
||||
process.stdout.write(output, (error) => {
|
||||
@@ -661,6 +681,9 @@ function runsForThisSeatsProject(env = process.env) {
|
||||
if (size > 0 && latest.lastReset?.at === state.lastReset?.at) latest.lastBytes = size;
|
||||
latest.firedAt = new Date().toISOString();
|
||||
latest.firedOn = event;
|
||||
// Writer completion, not a model-context receipt. A shortened output names its omitted tail and full-text file.
|
||||
if (contextOutput) latest.contextOutput = contextOutput;
|
||||
else delete latest.contextOutput;
|
||||
if (samePending) {
|
||||
delete latest.pendingOn;
|
||||
delete latest.pendingAt;
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 53 KiB |
@@ -7,7 +7,7 @@
|
||||
"url": "https://github.com/mvschwarz"
|
||||
},
|
||||
"homepage": "https://github.com/mvschwarz/openrig",
|
||||
"repository": "github:mvschwarz/openrig",
|
||||
"repository": "https://github.com/mvschwarz/openrig",
|
||||
"license": "Apache-2.0",
|
||||
"keywords": ["openrig", "agents", "team", "delegation", "mcp"],
|
||||
"skills": "./skills",
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
"url": "https://github.com/mvschwarz"
|
||||
},
|
||||
"homepage": "https://github.com/mvschwarz/openrig",
|
||||
"repository": "github:mvschwarz/openrig",
|
||||
"repository": "https://github.com/mvschwarz/openrig",
|
||||
"license": "Apache-2.0",
|
||||
"keywords": [
|
||||
"openrig",
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
"url": "https://github.com/mvschwarz"
|
||||
},
|
||||
"homepage": "https://github.com/mvschwarz/openrig",
|
||||
"repository": "github:mvschwarz/openrig",
|
||||
"repository": "https://github.com/mvschwarz/openrig",
|
||||
"license": "Apache-2.0",
|
||||
"keywords": ["openrig", "agents", "team", "delegation", "mcp"]
|
||||
}
|
||||
@@ -148,10 +148,9 @@ unseen — which makes messages the one delivery channel that never gets skipped
|
||||
A seat's typing guard and `--wait-for-idle` retain their checks. Managed compaction follows the
|
||||
ordinary draft default and explicit operator protections; it separately reconciles its own
|
||||
uncertain `/compact` write before sending restore input. Draft detection isn't complete.
|
||||
**Each send records one `transport.delivery_evidence` event**, best effort: what was admitted, what
|
||||
reached the input, what the screen check showed, and for which occupant. Read it as what was known
|
||||
about the delivery, not proof the agent read it; where the input landed can read unknown, and a
|
||||
missing record doesn't show there was no send.
|
||||
**Each send records one `transport.delivery_evidence` event**: what was admitted, what reached
|
||||
the input, what the screen check showed, and for which occupant. Read it as what was known about
|
||||
the delivery, not proof the agent read it; where the input landed can read unknown.
|
||||
- **`rig send --raw`** — send exact text or keystrokes without the From/To envelope;
|
||||
the interactive-prompt guard still applies. To deliberately drive an interactive prompt,
|
||||
**`--dangerously-interact --reason "<why>"`** is the explicit override and implies raw text.
|
||||
@@ -685,7 +684,7 @@ not have to tear one down to change it, and you rarely have to start from nothin
|
||||
someone edited it.
|
||||
**Since 0.7.12 it stops only a tmux session that carries the seat's own `OPENRIG_NODE_ID`**, on the
|
||||
server and session it read. A terminal OpenRig didn't create, a mismarked or replaced one, or a seat
|
||||
joined with `rig attach --self` from a tmux pane is left running, and `rig down` exits 2 naming it and the manual way
|
||||
joined with `rig attach --self` is left running, and `rig down` exits 2 naming it and the manual way
|
||||
to stop it. Teardown is incomplete for those terminals: if one is yours, look at it and stop it
|
||||
yourself, then run `rig down` again.
|
||||
- **`rig launch <rig> [seat]`** — one seat is down; start just that one, without disturbing the
|
||||
@@ -791,9 +790,6 @@ version, from checking the environment to a useful report to the OpenRig team.
|
||||
`Daemon starting: launch reserved by pid N since T; not listening yet` in `rig daemon status`, not stopped.
|
||||
A start reservation left from an earlier boot is archived by the next start; don't clear it by
|
||||
hand.
|
||||
- **`rig config set daemon.seat_poll_interval_ms <1000-2000>`** slows how often the daemon reads each seat's
|
||||
activity (default 1000), trading activity freshness for daemon CPU on a large fleet. It's read at daemon start,
|
||||
so restart the daemon to apply it. Near 2000, a slow tmux read can let a working Codex seat briefly read idle.
|
||||
- **A busy daemon isn't a down one.** The CLI refuses only when the daemon is confirmed down
|
||||
(stopped, stale, a refused probe), its probe is blocked, or it's a sibling home; a busy daemon
|
||||
gets the real request. A write that then times out says its outcome is **unknown**: check its
|
||||
@@ -1274,11 +1270,6 @@ reaching them is ordinary work rather than an escalation.
|
||||
a person can decide directly in the TUI's pairing settings. A relay connector that can't accept
|
||||
incoming connections can pull instead (`OPENRIG_RELAY_MODE=pull` in the relay secrets file); push
|
||||
stays the default, and a person's connector binding can be unset.
|
||||
- **The `relay` gateway connector** carries a registered person's questions, updates and notices to a
|
||||
bridge process you run, over loopback HTTP, and takes their replies back with the sender resolved
|
||||
against their bindings. It's off until a person has a relay binding (`rig gateway human add <id> …
|
||||
--binding "relay:<connectorRef>:file:<absolute env file>:primary"`), and the bridge, with its reach to
|
||||
any messaging platform, is yours (`docs/reference/gateway-relay.md`).
|
||||
- **`rig gateway channel add <id> --display-name <name> --binding
|
||||
slack:<connectorRef>:<secretsRef>:<channelId>`** — a Slack shared channel as one participant,
|
||||
`<id>@external` (Slack only). Anyone writing there reaches the team, registered or not. Each row
|
||||
@@ -1448,12 +1439,6 @@ your circumstances is configuration, and the ones that are not, another agent ca
|
||||
- **Move to an existing account home** with `rig provider switch --seat <seat> --to-account <name>
|
||||
--account-home <dir> --reason "<why>"` (preview with `--dry-run`). It starts a fresh conversation with a handover
|
||||
and retains the seat's address and work. `--clear-account-home --reason "<why>"` returns to configured defaults.
|
||||
- **A Pi seat switches accounts** with `rig provider switch --seat <pi seat> --to-account <name> --account-home <dir>
|
||||
--reason "<why>"` (`--dry-run` shows the plan), through a confirmed successor that keeps its model, effort and
|
||||
wrapper. A Pi seat that declares an absolute `PI_CODING_AGENT_DIR` in its `env:` uses that folder as its account
|
||||
home on launch, restart, resume and restore, and needs a readable `auth.json` there, or the launch stops; remove
|
||||
the declaration to keep the managed seat login. A seat that has its own Pi login as well stops at launch even with
|
||||
an `auth.json` there, and the error says why (for example, that the existing login isn't a managed account link).
|
||||
- **Claude subscription/API-key switching resumes the same conversation and interrupts work:**
|
||||
`rig provider switch --seat <seat> --to-account api --api-key-file <abs>`; return with
|
||||
`--to-account plan --token-file <abs>`. The API-key path remains unqualified; verify the resulting
|
||||
@@ -1595,9 +1580,6 @@ your circumstances is configuration, and the ones that are not, another agent ca
|
||||
selected plugin's skills there (into `.claude/skills/` or `.agents/skills/` only for a seat
|
||||
without that storage), so don't copy them by hand; read the `plugin_skill_*` warnings if one
|
||||
is missing. TDD is no longer a default skill.
|
||||
- **`on_restore: follow_source`** on a spec's `resources.skills[]` entry refreshes that skill from its source when
|
||||
a Claude or Codex seat is restored, keeping its conversation; `keep_saved`, the default, keeps the saved copy, but a
|
||||
skill whose source changed after `rig down` comes back missing, named in the restore's warnings.
|
||||
- **`rig startup-proof submit`** — answer an explicitly selected authenticated startup challenge.
|
||||
Startup adds no proof exercise by default. Declare `startup_proof` with `authenticated` or
|
||||
`none` in the applicable startup layers; see `docs/reference/rig-spec.md#startup-block/startup-proof-selection`.
|
||||
|
||||
@@ -107,6 +107,7 @@
|
||||
"dependencies": {
|
||||
"@hono/node-server": "^1.13.0",
|
||||
"@hono/node-ws": "^1.3.1",
|
||||
"@modelcontextprotocol/sdk": "^1.32.2",
|
||||
"better-sqlite3": "^13.0.3",
|
||||
"hono": "^4.7.0",
|
||||
"smol-toml": "^1.8.0",
|
||||
|
||||
@@ -94,6 +94,11 @@ pools and missing edge artifacts.
|
||||
|
||||
## Active surface
|
||||
|
||||
For a worked `periodic-reminder` YAML spec, full registration command and required
|
||||
flags, read `rig context get reference/keeping-a-watch.md#recurring-reminders-for-agents`.
|
||||
This is an agent's recurring reminder. A person's watch uses the one-shot queue
|
||||
pattern in `rig context get reference/keeping-a-watch.md#setting-a-watch`.
|
||||
|
||||
Daemon `rig watchdog` is the active host coordination surface for new
|
||||
work. History records only loud evaluations (`sent` / `terminal`);
|
||||
quiet skip reasons are NOT recorded — POC parity so agents are not
|
||||
|
||||
@@ -21,7 +21,8 @@
|
||||
import fs from "node:fs";
|
||||
import nodePath from "node:path";
|
||||
import readline from "node:readline";
|
||||
import { PassThrough } from "node:stream";
|
||||
import { PassThrough, Writable } from "node:stream";
|
||||
import stringWidth from "string-width";
|
||||
import { StringDecoder } from "node:string_decoder";
|
||||
import { execFileSync, spawn, spawnSync } from "node:child_process";
|
||||
import { pathToFileURL } from "node:url";
|
||||
@@ -71,14 +72,306 @@ export function preparePiAccount(agentDir: string, accountHome: string | undefin
|
||||
// Use Node's line editor in raw mode, with only paste framing handled here.
|
||||
export const MAX_PI_INPUT_BYTES = 1024 * 1024;
|
||||
|
||||
// ── The compose region's physical layout ──────────────────────────────────────────
|
||||
// One layout model for painting, erasing and the caret: cells fill terminal rows
|
||||
// greedily, a wide character that no longer fits wraps whole to the next row, and
|
||||
// an explicit newline starts one. Every row the compositor paints or erases, and
|
||||
// the caret it places, derives from here — measured with the daemon's string-width.
|
||||
|
||||
/** Where a compose region sits on screen, and where its caret lands. */
|
||||
export interface ComposeLayout { rows: number; caretRow: number; caretColumn: number; }
|
||||
|
||||
const SEGMENTER = (() => {
|
||||
try {
|
||||
return new Intl.Segmenter(undefined, { granularity: "grapheme" });
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
})();
|
||||
|
||||
/** Grapheme clusters: what a terminal lays out. A ZWJ family emoji is one cluster
|
||||
* taking two cells where its code points would sum to six; a combining mark adds none. */
|
||||
export function* graphemesOf(text: string): Generator<string> {
|
||||
if (SEGMENTER) {
|
||||
for (const part of SEGMENTER.segment(text)) yield part.segment;
|
||||
} else {
|
||||
for (const character of text) yield character;
|
||||
}
|
||||
}
|
||||
|
||||
const TAB_WIDTH = 8;
|
||||
|
||||
export function composeLayout(text: string, width: number, caret = text.length): ComposeLayout {
|
||||
if (!text) return { rows: 0, caretRow: 0, caretColumn: 0 };
|
||||
if (width <= 0) return { rows: text.split("\n").length, caretRow: 0, caretColumn: 0 };
|
||||
let row = 0;
|
||||
let column = 0;
|
||||
let caretRow = 0;
|
||||
let caretColumn = 0;
|
||||
let index = 0;
|
||||
const place = () => { caretRow = row; caretColumn = column; };
|
||||
for (const segment of graphemesOf(text)) {
|
||||
if (index === caret) place();
|
||||
if (segment === "\n") {
|
||||
row += 1;
|
||||
column = 0;
|
||||
} else if (segment === "\t") {
|
||||
// A tab advances to the next ordinary tab stop, clamped to the last column.
|
||||
column = Math.min((Math.floor(column / TAB_WIDTH) + 1) * TAB_WIDTH, width - 1);
|
||||
} else {
|
||||
const cells = Math.max(0, stringWidth(segment));
|
||||
if (column + cells > width) {
|
||||
row += 1;
|
||||
column = 0;
|
||||
}
|
||||
column += cells;
|
||||
}
|
||||
index += segment.length;
|
||||
}
|
||||
if (index === caret) place();
|
||||
return { rows: row + 1, caretRow, caretColumn };
|
||||
}
|
||||
|
||||
// ── Pane live region (compose + status) ────────────────────────────────────────
|
||||
// A seat pane is a plain terminal: one bottom live region holds the operator's draft
|
||||
// and the runner's status line, and is erased and repainted around every transcript
|
||||
// write. Nothing else may write at the cursor, or a draft freezes into scrollback and
|
||||
// streamed text fuses with it mid-word. When the terminal's width is unknown the
|
||||
// compositor places nothing — the pane falls back to plain output, and the draft,
|
||||
// though still editable, is not drawn.
|
||||
|
||||
/** The live region a pane host repaints around transcript writes. */
|
||||
export interface PaneCompositor {
|
||||
/** Bind the line editor the compose line renders from, and where its cursor sits. */
|
||||
attach(read: () => string, redraw: () => void, caretUp?: () => number, readCursor?: () => number): void;
|
||||
/** Unbind the editor; any held streamed tail is flushed and the live region is dropped. */
|
||||
detach(): void;
|
||||
/** A keystroke changed the draft: repaint the live region from readline's state. A
|
||||
* draft that just went away flushes a held streamed tail first, as plain output would. */
|
||||
refresh(): void;
|
||||
/** Streamed model text: verbatim while no live region exists; once a draft or a
|
||||
* status line is on the bottom, complete lines go above the region and an
|
||||
* unfinished line is held, so no live line is split or frozen into scrollback. */
|
||||
transcript(text: string): void;
|
||||
/** One complete line of transcript. */
|
||||
line(text: string): void;
|
||||
/** A keypress is about to echo: close an open transcript line first, or the echo
|
||||
* and the next streamed deltas would land on top of it. */
|
||||
beforeInput(): void;
|
||||
/** Pin the latest diagnostic on the status line. The transcript copy is the caller's;
|
||||
* this pin is never the only copy. */
|
||||
diagnostic(text: string): void;
|
||||
/** Drop a pinned status line. */
|
||||
clearStatus(): void;
|
||||
/** Before the submit commits: drop the draft line so no partial copy scrolls, and
|
||||
* finish any held streamed line; the durable copy is the commit line below. */
|
||||
beforeSubmit(): void;
|
||||
/** Repaint the live region in place. */
|
||||
repaint(): void;
|
||||
}
|
||||
|
||||
export function createPaneCompositor(out: NodeJS.WriteStream): PaneCompositor {
|
||||
let status: string | null = null;
|
||||
/** The status text actually on screen: clipped when it was painted. A resize must
|
||||
* size the region from this, not from the longer text it was pinned with. */
|
||||
let paintedStatus: string | null = null;
|
||||
let readInput: (() => string) | null = null;
|
||||
let redrawLine: (() => void) | null = null;
|
||||
let held = "";
|
||||
let openLine = false;
|
||||
let screenRows = 0;
|
||||
let readCaretUp: (() => number) | null = null;
|
||||
let readCaretIndex: (() => number) | null = null;
|
||||
const widthKnown = () => typeof out.columns === "number" && out.columns > 0;
|
||||
const composeText = () => (readInput && redrawLine ? readInput() : "");
|
||||
const composeRows = () => {
|
||||
const text = composeText();
|
||||
return text && widthKnown() ? composeLayout(text, out.columns).rows : 0;
|
||||
};
|
||||
const clip = (text: string) => {
|
||||
const flat = text.replace(/[\r\n\t]+/g, " ").trimEnd();
|
||||
if (!widthKnown() || stringWidth(flat) < out.columns) return flat;
|
||||
let used = 0;
|
||||
let clipped = "";
|
||||
for (const segment of graphemesOf(flat)) {
|
||||
const cells = Math.max(0, stringWidth(segment));
|
||||
if (used + cells > out.columns - 1) break;
|
||||
clipped += segment;
|
||||
used += cells;
|
||||
}
|
||||
return `${clipped}\u2026`;
|
||||
};
|
||||
const eraseLines = (lines: number) => {
|
||||
let sequence = "";
|
||||
for (let index = 0; index < lines; index += 1) sequence += index === 0 ? "\r\u001b[2K" : "\u001b[1A\r\u001b[2K";
|
||||
if (sequence) out.write(sequence);
|
||||
};
|
||||
/** Rows this compositor has on screen, recorded when it painted them. Readline never writes
|
||||
* to the terminal, so nothing else can change these rows behind the compositor's back —
|
||||
* except a reflow, which the resize handler owns. */
|
||||
const paint = () => {
|
||||
paintedStatus = status !== null ? clip(status) : null;
|
||||
if (paintedStatus !== null) {
|
||||
out.write(paintedStatus);
|
||||
out.write("\n");
|
||||
}
|
||||
const rows = composeRows();
|
||||
if (rows) redrawLine?.();
|
||||
screenRows = (status !== null ? 1 : 0) + (rows || (status !== null ? 1 : 0));
|
||||
// Only a painted region closes an open transcript line. With nothing painted the
|
||||
// cursor still sits on the line a flush left open, and the next keystroke belongs
|
||||
// to that line's context, not to a fresh one.
|
||||
if (screenRows > 0) openLine = false;
|
||||
};
|
||||
const eraseScreen = () => {
|
||||
// Nothing painted: there is no region to erase, and stepping down from a stale
|
||||
// caret count would move the cursor off the transcript line it is still on.
|
||||
if (screenRows === 0) return;
|
||||
const down = readCaretUp ? readCaretUp() : 0;
|
||||
if (down > 0) out.write(`\u001b[${down}B`);
|
||||
eraseLines(screenRows);
|
||||
screenRows = 0;
|
||||
};
|
||||
const flushHeld = () => {
|
||||
if (!held) return;
|
||||
out.write(held.endsWith("\n") ? held : `${held}\n`);
|
||||
held = "";
|
||||
openLine = false;
|
||||
};
|
||||
const writeVerbatim = (text: string) => {
|
||||
out.write(text);
|
||||
openLine = !text.endsWith("\n");
|
||||
};
|
||||
const writeAbove = (text: string) => {
|
||||
eraseScreen();
|
||||
out.write(text.endsWith("\n") ? text : `${text}\n`);
|
||||
paint();
|
||||
};
|
||||
// A tile attaching or detaching reflows the pane, and the terminal keeps the cursor on
|
||||
// its character. So the old paint now occupies exactly what the new width says it does:
|
||||
// the pinned status wraps to ceil(cells/columns) rows, the draft to its layout rows.
|
||||
// Move down to the region's last row — past the lines below the caret — erase exactly
|
||||
// that many rows, and repaint at the new width. No max(): under-erasing leaves a stale
|
||||
// copy of a wrapped status, over-erasing drops a transcript line off the screen.
|
||||
const onResize = () => {
|
||||
if (!widthKnown()) {
|
||||
screenRows = 0;
|
||||
return;
|
||||
}
|
||||
const draft = composeText();
|
||||
const layout = draft ? composeLayout(draft, out.columns, readCaretIndex ? readCaretIndex() : draft.length) : null;
|
||||
const draftRows = layout ? layout.rows : 0;
|
||||
const statusLines = paintedStatus !== null ? Math.max(1, Math.ceil(stringWidth(paintedStatus) / out.columns)) : 0;
|
||||
const down = layout ? layout.rows - 1 - layout.caretRow : 0;
|
||||
if (down > 0) out.write(`\u001b[${down}B`);
|
||||
// A pinned status without a draft also owns the blank row its trailing newline
|
||||
// left the cursor on — the same row paint() counts into screenRows.
|
||||
eraseLines(statusLines + (draftRows || (status !== null ? 1 : 0)));
|
||||
screenRows = 0;
|
||||
paint();
|
||||
};
|
||||
out.on("resize", onResize);
|
||||
const pane: PaneCompositor = {
|
||||
attach(read, redraw, caretUp, readCursor) {
|
||||
readInput = read;
|
||||
redrawLine = redraw;
|
||||
readCaretUp = caretUp ?? null;
|
||||
readCaretIndex = readCursor ?? null;
|
||||
},
|
||||
detach() {
|
||||
eraseScreen();
|
||||
flushHeld();
|
||||
readInput = null;
|
||||
redrawLine = null;
|
||||
readCaretUp = null;
|
||||
readCaretIndex = null;
|
||||
openLine = false;
|
||||
screenRows = 0;
|
||||
},
|
||||
refresh() {
|
||||
eraseScreen();
|
||||
// With no region left, an unfinished streamed line goes out verbatim so the
|
||||
// next delta continues it on the same row instead of splitting mid-word.
|
||||
if (held && status === null && !composeText()) {
|
||||
writeVerbatim(held);
|
||||
held = "";
|
||||
}
|
||||
paint();
|
||||
},
|
||||
transcript(text) {
|
||||
if (!text) return;
|
||||
if (status === null && composeRows() === 0 && screenRows === 0) {
|
||||
if (held) {
|
||||
writeVerbatim(held);
|
||||
held = "";
|
||||
}
|
||||
writeVerbatim(text);
|
||||
return;
|
||||
}
|
||||
held += text;
|
||||
let newline = held.indexOf("\n");
|
||||
while (newline >= 0) {
|
||||
const line = held.slice(0, newline + 1);
|
||||
held = held.slice(newline + 1);
|
||||
writeAbove(line);
|
||||
newline = held.indexOf("\n");
|
||||
}
|
||||
},
|
||||
line(text) {
|
||||
pane.transcript(`${text}\n`);
|
||||
},
|
||||
beforeInput() {
|
||||
if (!openLine) return;
|
||||
out.write("\n");
|
||||
openLine = false;
|
||||
},
|
||||
diagnostic(text) {
|
||||
// Without a known width there is no region to pin into. Callers already wrote
|
||||
// the transcript copy; printing here would duplicate it.
|
||||
if (!widthKnown()) return;
|
||||
if (openLine) {
|
||||
out.write("\n");
|
||||
openLine = false;
|
||||
}
|
||||
eraseScreen();
|
||||
status = clip(text);
|
||||
paint();
|
||||
},
|
||||
clearStatus() {
|
||||
if (status === null) return;
|
||||
eraseScreen();
|
||||
status = null;
|
||||
paint();
|
||||
},
|
||||
beforeSubmit() {
|
||||
eraseScreen();
|
||||
openLine = false;
|
||||
flushHeld();
|
||||
},
|
||||
repaint() {
|
||||
paint();
|
||||
},
|
||||
};
|
||||
return pane;
|
||||
}
|
||||
|
||||
export function createRunnerInput(
|
||||
input: NodeJS.ReadStream,
|
||||
output: NodeJS.WriteStream,
|
||||
onSubmit: (block: string) => void,
|
||||
pane?: PaneCompositor,
|
||||
): readline.Interface {
|
||||
const reject = () => output.write(
|
||||
"[pi-runner] input rejected: maximum 1048576 UTF-8 bytes; send a smaller message. Ctrl-C clears unfinished input.\n",
|
||||
);
|
||||
const say = (text: string) => {
|
||||
if (pane) {
|
||||
pane.line(text);
|
||||
pane.diagnostic(text);
|
||||
} else {
|
||||
output.write(`${text}\n`);
|
||||
}
|
||||
};
|
||||
const reject = () => {
|
||||
say("[pi-runner] input rejected: maximum 1048576 UTF-8 bytes; send a smaller message. Ctrl-C clears unfinished input.");
|
||||
};
|
||||
const submit = (block: string) => {
|
||||
if (Buffer.byteLength(block) > MAX_PI_INPUT_BYTES) reject();
|
||||
else if (block.trim()) onSubmit(block);
|
||||
@@ -94,14 +387,37 @@ export function createRunnerInput(
|
||||
|| (major === 25 && (minor === 0 || (minor === 1 && patch === 0)));
|
||||
const keys = new PassThrough();
|
||||
const editorInput = batchText ? new PassThrough() : keys;
|
||||
// Readline edits against a discard sink: it must never draw on the terminal, because
|
||||
// _refreshLine does moveCursor(0, -prevRows) + clearScreenDown(), which erases whatever the
|
||||
// compositor put above the draft. The compositor owns every terminal byte instead.
|
||||
const sink = new Writable({ write(_chunk, _encoding, callback) { callback(); } });
|
||||
// Readline wraps its edit against output.columns; carry the terminal's geometry onto the sink.
|
||||
Object.assign(sink, { columns: output.columns, rows: output.rows });
|
||||
let caretUp = 0;
|
||||
const renderCompose = () => {
|
||||
const text = `${editor.line ?? ""}`;
|
||||
if (!text) return;
|
||||
const width = typeof output.columns === "number" && output.columns > 0 ? output.columns : 0;
|
||||
output.write(text);
|
||||
if (!width) return;
|
||||
const cursor = Math.max(0, Math.min(Number(editor.cursor) || 0, text.length));
|
||||
const layout = composeLayout(text, width, cursor);
|
||||
caretUp = layout.rows - 1 - layout.caretRow;
|
||||
if (caretUp > 0) output.write(`\u001b[${caretUp}A`);
|
||||
output.write("\r");
|
||||
if (layout.caretColumn > 0) output.write(`\u001b[${layout.caretColumn}C`);
|
||||
};
|
||||
pane?.attach(() => `${editor.line ?? ""}`, renderCompose, () => caretUp, () => Math.max(0, Math.min(Number(editor.cursor) || 0, `${editor.line ?? ""}`.length)));
|
||||
|
||||
const editor = readline.createInterface({
|
||||
input: editorInput, output, terminal: true, prompt: "", historySize: 0, crlfDelay: Infinity,
|
||||
input: editorInput, output: sink, terminal: true, prompt: "", historySize: 0, crlfDelay: Infinity,
|
||||
});
|
||||
// Node forces dumb-terminal mode without editing if TERM=dumb. When terminal: true
|
||||
// was requested, remove the dumb-mode override so VT100 line editing is preserved.
|
||||
if (Object.prototype.hasOwnProperty.call(editor, "_ttyWrite")) {
|
||||
delete (editor as any)._ttyWrite;
|
||||
}
|
||||
|
||||
const decoder = new StringDecoder("utf8");
|
||||
let pending = "";
|
||||
let paste: string | null = null;
|
||||
@@ -111,7 +427,7 @@ export function createRunnerInput(
|
||||
// Node documents changing rl.line together with rl.cursor. The installed
|
||||
// typings mark them readonly, so assign this pair through one explicit seam.
|
||||
Object.assign(editor, { line, cursor });
|
||||
editor.prompt(true);
|
||||
pane?.refresh();
|
||||
};
|
||||
const clear = () => setLine("", 0);
|
||||
// Node 22.21–22.22.0 (also some 24/25 releases) appends batched text after
|
||||
@@ -137,6 +453,7 @@ export function createRunnerInput(
|
||||
}
|
||||
const consume = (text: string) => {
|
||||
if (paste === null) {
|
||||
pane?.beforeInput();
|
||||
keys.write(text);
|
||||
if (batchText) flushTyped();
|
||||
} else if (!discarded) {
|
||||
@@ -170,7 +487,7 @@ export function createRunnerInput(
|
||||
paste = null;
|
||||
discarded = false;
|
||||
clear();
|
||||
output.write("\n[pi-runner] input cleared\n");
|
||||
say("[pi-runner] input cleared");
|
||||
onSubmit("/abort");
|
||||
} else if (paste === null) {
|
||||
paste = "";
|
||||
@@ -189,6 +506,7 @@ export function createRunnerInput(
|
||||
paste = null;
|
||||
}
|
||||
}
|
||||
pane?.refresh();
|
||||
};
|
||||
const wasRaw = input.isRaw;
|
||||
const end = () => editor.close();
|
||||
@@ -196,7 +514,10 @@ export function createRunnerInput(
|
||||
output.write("\u001b[?2004h");
|
||||
input.on("data", receive);
|
||||
input.once("end", end);
|
||||
editor.on("line", submit);
|
||||
editor.on("line", (line) => {
|
||||
pane?.beforeSubmit();
|
||||
submit(line);
|
||||
});
|
||||
editor.once("close", () => {
|
||||
input.removeListener("data", receive);
|
||||
input.removeListener("end", end);
|
||||
@@ -204,6 +525,7 @@ export function createRunnerInput(
|
||||
input.pause();
|
||||
keys.removeListener("keypress", keypress);
|
||||
keys.destroy();
|
||||
pane?.detach();
|
||||
if (editorInput !== keys) editorInput.destroy();
|
||||
output.write("\u001b[?2004l");
|
||||
});
|
||||
@@ -359,6 +681,11 @@ export interface RunnerIo {
|
||||
mirrorLine(line: string): void;
|
||||
/** Append raw text to the current pane line (streamed deltas). */
|
||||
mirrorAppend(text: string): void;
|
||||
/** Pin a copy of a diagnostic on the pane's status line, when the pane provides one.
|
||||
* The transcript copy is written by the caller; this pin is never the only copy. */
|
||||
diagnostic?(text: string): void;
|
||||
/** Drop a pinned status line at the start of a new agent turn. */
|
||||
clearStatus?(): void;
|
||||
/** POST to the daemon activity endpoint. May resolve to the parsed JSON
|
||||
* response (null on transport failure) so identity delivery can retry. */
|
||||
postActivity(payload: Record<string, unknown>): void | Promise<Record<string, unknown> | null>;
|
||||
@@ -592,7 +919,7 @@ export class RunnerCore {
|
||||
|
||||
/** A frame that can't be reassembled is lost, and said so. If it was a get_entries reply, the cursor stays where it was. */
|
||||
private reportDroppedFrame(reason: string | null | undefined): void {
|
||||
if (reason) this.io.mirrorLine(`[omp-runner] ERROR rpc frame: ${reason}; the chunked frame was dropped`);
|
||||
if (reason) this.diag(`[omp-runner] ERROR rpc frame: ${reason}; the chunked frame was dropped`);
|
||||
}
|
||||
|
||||
private get runtime(): string { return this.opts.runtime ?? "pi"; }
|
||||
@@ -601,7 +928,7 @@ export class RunnerCore {
|
||||
handleUserBlock(block: string): void {
|
||||
if (block === "/abort") {
|
||||
this.io.sendRpc({ type: "abort" });
|
||||
this.io.mirrorLine("[pi-runner] abort sent");
|
||||
this.diag("[pi-runner] abort sent");
|
||||
return;
|
||||
}
|
||||
if (block.startsWith("/followup ")) {
|
||||
@@ -646,7 +973,7 @@ export class RunnerCore {
|
||||
} else {
|
||||
// Advertised but refused: v1 as before, so a reply over 1 MiB still fails, visibly.
|
||||
const detail = typeof record.error === "string" ? record.error : "refused";
|
||||
this.io.mirrorLine(`[omp-runner] RPC protocol v2 not negotiated (${detail}); replies over 1 MiB will fail`);
|
||||
this.diag(`[omp-runner] RPC protocol v2 not negotiated (${detail}); replies over 1 MiB will fail`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
@@ -656,7 +983,7 @@ export class RunnerCore {
|
||||
if (record.id === GET_STATE_ID) {
|
||||
this.ready = false;
|
||||
this.writeSidecar({});
|
||||
this.io.mirrorLine(this.runtime === "omp" ? `[omp-runner] ERROR rpc get_state: ${message}` : `${PI_RUNNER_ERROR_MARKER} rpc: ${message}`);
|
||||
this.diag(this.runtime === "omp" ? `[omp-runner] ERROR rpc get_state: ${message}` : `${PI_RUNNER_ERROR_MARKER} rpc: ${message}`);
|
||||
// OMP cannot provide a resume token without get_state; stop it so the
|
||||
// launch fails now instead of waiting out readiness.
|
||||
if (this.runtime === "omp") this.io.stopChild?.();
|
||||
@@ -665,11 +992,11 @@ export class RunnerCore {
|
||||
if (record.id === CATCH_UP_ID && this.runtime === "pi") {
|
||||
// A refused catch-up (a saved cursor this session's history doesn't hold, such as an older runner's UI-event
|
||||
// id) is history sync, not the seat's resume: say so, and take a confirmed cursor from the history's tail.
|
||||
this.io.mirrorLine(`[pi-runner] History catch-up didn't run (${message}). That's history sync, separate from the seat's resume; reading the history cursor again.`);
|
||||
this.diag(`[pi-runner] History catch-up didn't run (${message}). That's history sync, separate from the seat's resume; reading the history cursor again.`);
|
||||
this.io.sendRpc({ type: "get_entries", id: CURSOR_REFRESH_ID });
|
||||
return;
|
||||
}
|
||||
this.io.mirrorLine(`[${this.runtime}-runner] ERROR rpc: ${message}`);
|
||||
this.diag(`[${this.runtime}-runner] ERROR rpc: ${message}`);
|
||||
// Only a rejected input means the seat cannot do the requested work.
|
||||
// A mistimed /abort (or any control command) leaves the seat idle.
|
||||
if (this.runtime === "omp" && typeof record.command === "string" && Object.hasOwn(INPUT_COMMANDS, record.command)) {
|
||||
@@ -687,13 +1014,13 @@ export class RunnerCore {
|
||||
this.ready = this.runtime === "pi" || !!this.sessionFile;
|
||||
if (!this.ready) {
|
||||
if (this.runtime === "omp") {
|
||||
this.io.mirrorLine("[omp-runner] ERROR rpc get_state returned no session file; cannot provide a resume token");
|
||||
this.diag("[omp-runner] ERROR rpc get_state returned no session file; cannot provide a resume token");
|
||||
this.io.stopChild?.();
|
||||
return;
|
||||
}
|
||||
// For custom protocol: pi-rpc community harnesses, reporting no session file
|
||||
// reads as needing attention rather than stopping the child process.
|
||||
this.io.mirrorLine(`[${this.runtime}-runner] ERROR rpc get_state returned no session file; cannot provide a resume token`);
|
||||
this.diag(`[${this.runtime}-runner] ERROR rpc get_state returned no session file; cannot provide a resume token`);
|
||||
this.raiseAttention("runtime_error");
|
||||
return;
|
||||
}
|
||||
@@ -856,6 +1183,13 @@ export class RunnerCore {
|
||||
this.attentionIsLatest = false;
|
||||
}
|
||||
|
||||
/** A diagnostic stays in the transcript — the readiness probe and the operator both read
|
||||
* it there; a pane with a status line pins the latest one as a convenience copy. */
|
||||
private diag(text: string): void {
|
||||
this.io.mirrorLine(text);
|
||||
this.io.diagnostic?.(text);
|
||||
}
|
||||
|
||||
private handleEvent(event: Record<string, unknown>): void {
|
||||
if (event.type === "extension_ui_request") {
|
||||
const method = event.method;
|
||||
@@ -863,7 +1197,7 @@ export class RunnerCore {
|
||||
const detail = [event.title, event.message].filter((text): text is string => typeof text === "string")
|
||||
.map(stripVTControlCharacters).join(" — ").replace(/[\p{Cc}\p{Cf}\p{Zl}\p{Zp}]/gu, " ").slice(0, 400);
|
||||
if (this.runtime !== "omp" && !dialog) {
|
||||
if (method === "notify") this.io.mirrorLine(`[${this.runtime}-runner] ${detail}`);
|
||||
if (method === "notify") this.diag(`[${this.runtime}-runner] ${detail}`);
|
||||
return; // Pi notifications and other fire-and-forget UI requests expect no answer.
|
||||
}
|
||||
// RPC does not mount an interactive UI. Reply at once; otherwise the
|
||||
@@ -872,7 +1206,7 @@ export class RunnerCore {
|
||||
this.io.sendRpc({ type: "extension_ui_response", id: event.id, cancelled: true });
|
||||
}
|
||||
if (dialog) {
|
||||
this.io.mirrorLine(this.runtime !== "omp"
|
||||
this.diag(this.runtime !== "omp"
|
||||
? `[${this.runtime}-runner] Extension ${method}: ${detail}; cancelled because the managed RPC pane cannot answer it. Operator action required: review the cancelled request and re-send an instruction the extension can complete without a dialog.`
|
||||
: "[omp-runner] Approval/input requested; cancelled because the managed RPC pane cannot answer it. Operator action required: review the denied action and re-send a safe instruction, or change the seat permission policy explicitly.");
|
||||
this.raiseAttention("permission_prompt");
|
||||
@@ -892,7 +1226,10 @@ export class RunnerCore {
|
||||
this.writeSidecar({});
|
||||
}
|
||||
|
||||
if (event.type === "agent_start") this.clearAttention();
|
||||
if (event.type === "agent_start") {
|
||||
this.clearAttention();
|
||||
this.io.clearStatus?.();
|
||||
}
|
||||
if (this.runtime === "omp") {
|
||||
if (event.type === "agent_start" || event.type === "turn_start"
|
||||
|| (event.type === "message_start" && message?.role === "assistant")
|
||||
@@ -936,7 +1273,7 @@ export class RunnerCore {
|
||||
// the repeat lacks it, then reset at the next assistant message/agent
|
||||
// turn, not at agent_end.
|
||||
if (event.type === "message_end" || !this.assistantErrorShown) {
|
||||
this.io.mirrorLine(mapped.errorNotice);
|
||||
this.diag(mapped.errorNotice);
|
||||
}
|
||||
this.assistantErrorShown = true;
|
||||
// Only typed, explicitly transient HTTP failures may release the send
|
||||
@@ -1445,6 +1782,14 @@ export async function main(argv = process.argv.slice(2)): Promise<void> {
|
||||
// The child may close its pipe before the first RPC write completes.
|
||||
child.stdin.on("error", () => { /* spawn/exit handlers own the diagnosis */ });
|
||||
|
||||
const pane = createPaneCompositor(process.stdout);
|
||||
// Diagnostics stay in the transcript — the pane's readiness probe greps it, and people
|
||||
// and agents read the history there; the status line only pins the latest copy.
|
||||
const paneDiag = (text: string) => {
|
||||
pane.line(text);
|
||||
pane.diagnostic(text);
|
||||
};
|
||||
|
||||
const chargeLookup = new OpenRouterChargeLookup({
|
||||
fetchGeneration: openRouterGenerationFetcher(() => childEnv["OPENROUTER_API_KEY"]),
|
||||
postCharges: async (piSessionId, charges) => {
|
||||
@@ -1463,7 +1808,7 @@ export async function main(argv = process.argv.slice(2)): Promise<void> {
|
||||
}, () => false as const).finally(() => clearTimeout(timeout));
|
||||
},
|
||||
schedule: (fn, ms) => { setTimeout(fn, ms).unref(); },
|
||||
notice: (line) => { process.stdout.write(`${line}\n`); },
|
||||
notice: (line) => { paneDiag(line); },
|
||||
});
|
||||
|
||||
// One activity post at a time: the daemon reads its newest record by arrival, so a beat still in flight when a turn
|
||||
@@ -1486,8 +1831,10 @@ export async function main(argv = process.argv.slice(2)): Promise<void> {
|
||||
sendRpc: (cmd) => {
|
||||
try { child.stdin.write(`${JSON.stringify(cmd)}\n`); } catch { /* exit handler reports */ }
|
||||
},
|
||||
mirrorLine: (line) => process.stdout.write(`${line}\n`),
|
||||
mirrorAppend: (text) => process.stdout.write(text),
|
||||
mirrorLine: (line) => pane.line(line),
|
||||
mirrorAppend: (text) => pane.transcript(text),
|
||||
diagnostic: (text) => pane.diagnostic(text),
|
||||
clearStatus: () => pane.clearStatus(),
|
||||
postActivity: (payload) => {
|
||||
if (!endpoint || typeof fetch !== "function") return;
|
||||
const result = postInOrder(payload).catch(() => null);
|
||||
@@ -1568,7 +1915,7 @@ export async function main(argv = process.argv.slice(2)): Promise<void> {
|
||||
let reportedPiCapability = false;
|
||||
readline.createInterface({ input: child.stderr }).on("line", (line) => {
|
||||
if (!line.trim()) return;
|
||||
process.stdout.write(`[${runtime}:err] ${line}\n`);
|
||||
paneDiag(`[${runtime}:err] ${line}`);
|
||||
stderrTail.push(line.length > OMP_STDERR_TAIL_CHARS ? `${line.slice(0, OMP_STDERR_TAIL_CHARS)}...` : line);
|
||||
if (stderrTail.length > OMP_STDERR_TAIL_LINES) stderrTail.shift();
|
||||
if (runtime === "pi" && !core.isReady() && !reportedPiCapability) {
|
||||
@@ -1582,15 +1929,15 @@ export async function main(argv = process.argv.slice(2)): Promise<void> {
|
||||
});
|
||||
if (detail) {
|
||||
reportedPiCapability = true;
|
||||
process.stdout.write(`[pi-runner] ${detail}\n`);
|
||||
paneDiag(`[pi-runner] ${detail}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
const input = createRunnerInput(process.stdin, process.stdout, (block) => core.handleUserBlock(block));
|
||||
const input = createRunnerInput(process.stdin, process.stdout, (block) => core.handleUserBlock(block), pane);
|
||||
|
||||
if (runtime !== "omp") {
|
||||
child.on("error", (err) => {
|
||||
console.error(`${PI_RUNNER_ERROR_MARKER} failed to spawn ${args.launchWrapper?.length ? `launch_wrapper (${spawnFile})` : (args.command ?? "pi")}: ${err.message}`);
|
||||
paneDiag(`${PI_RUNNER_ERROR_MARKER} failed to spawn ${args.launchWrapper?.length ? `launch_wrapper (${spawnFile})` : (args.command ?? "pi")}: ${err.message}`);
|
||||
const code = (err as NodeJS.ErrnoException).code === "ENOENT" ? 127 : 1;
|
||||
core.handlePiExit(code);
|
||||
input.close();
|
||||
@@ -1617,7 +1964,7 @@ export async function main(argv = process.argv.slice(2)): Promise<void> {
|
||||
if (!core.isReady()) {
|
||||
const how = signal ? `signal ${signal}` : `exit code ${code ?? "unknown"}`;
|
||||
const output = stderrTail.length > 0 ? ` Its last output: ${stderrTail.join(" | ")}` : " It printed nothing on stderr.";
|
||||
console.error(transportUp
|
||||
paneDiag(transportUp
|
||||
? "[omp-runner] ERROR OMP did not establish a resumable RPC session. Authenticate this isolated seat using HOME=<seat-root> PI_CODING_AGENT_DIR=<seat-root>/agent omp and /login, or provide its declared model provider key in the OpenRig daemon environment. Default OMP credentials are not shared."
|
||||
: `[omp-runner] ERROR OMP exited during startup, before its RPC transport started (${command}, ${how}).${output}`);
|
||||
}
|
||||
@@ -1626,7 +1973,7 @@ export async function main(argv = process.argv.slice(2)): Promise<void> {
|
||||
process.exitCode = code ?? 1;
|
||||
};
|
||||
child.on("error", (err) => {
|
||||
console.error(`[omp-runner] ERROR failed to spawn omp: ${err.message}`);
|
||||
paneDiag(`[omp-runner] ERROR failed to spawn omp: ${err.message}`);
|
||||
recordExit(null);
|
||||
});
|
||||
// "close" fires once OMP's stdio is drained, so its stderr is in the tail
|
||||
|
||||
@@ -7,6 +7,8 @@ import { queueErasureLinkIndexesSchema } from "./migrations/156_queue_erasure_li
|
||||
import { queueErasureBatchesSchema } from "./migrations/157_queue_erasure_batches.js";
|
||||
import { queueInboxCheckinsSchema } from "./migrations/158_queue_inbox_checkins.js";
|
||||
import { queueItemsHandedOffFromIndexSchema } from "./migrations/159_queue_items_handed_off_from_index.js";
|
||||
import { networkBindingsSchema } from "./migrations/160_network_bindings.js";
|
||||
import { networkReceiptsSchema } from "./migrations/161_network_receipts.js";
|
||||
import { activityGrantsSchema } from "./migrations/143_activity_grants.js";
|
||||
import { environmentBindingsSchema } from "./migrations/144_environment_bindings.js";
|
||||
import { nodePermissionsToSchema } from "./migrations/145_node_permissions_to.js";
|
||||
@@ -321,4 +323,6 @@ export const ALL_MIGRATIONS: Migration[] = [
|
||||
queueErasureBatchesSchema,
|
||||
queueInboxCheckinsSchema,
|
||||
queueItemsHandedOffFromIndexSchema,
|
||||
networkBindingsSchema,
|
||||
networkReceiptsSchema,
|
||||
];
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { Migration } from "../migrate.js";
|
||||
|
||||
/** A selected local seat joined to one participant of a remote hub (an account's network rig, reached over MCP). The
|
||||
* participant's credential lives in a private file under the OpenRig home, never in this table. One active binding per
|
||||
* local seat and per hub participant; a disconnected or revoked binding stays as history. `allow_http` records the
|
||||
* person's choice to send the participant's token to a plain-http hub. */
|
||||
export const networkBindingsSchema: Migration = {
|
||||
name: "160_network_bindings.sql",
|
||||
sql: `
|
||||
CREATE TABLE IF NOT EXISTS network_bindings (
|
||||
id TEXT PRIMARY KEY,
|
||||
node_id TEXT NOT NULL,
|
||||
local_session TEXT NOT NULL,
|
||||
hub_url TEXT NOT NULL,
|
||||
hub_seat TEXT NOT NULL,
|
||||
allow_http INTEGER NOT NULL DEFAULT 0 CHECK (allow_http IN (0, 1)),
|
||||
state TEXT NOT NULL CHECK (state IN ('active', 'revoked', 'disconnected')),
|
||||
state_reason TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL,
|
||||
last_contact_at TEXT,
|
||||
last_error TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_network_bindings_active_node ON network_bindings(node_id) WHERE state = 'active';
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_network_bindings_active_participant ON network_bindings(hub_url, hub_seat) WHERE state = 'active';
|
||||
`,
|
||||
};
|
||||
@@ -0,0 +1,26 @@
|
||||
import type { Migration } from "../migrate.js";
|
||||
|
||||
/** One hub item taken in, and the local item that carries it: what was reported back, and when. It's a receipt, not a
|
||||
* task: the work itself is the local queue item, found by its id, which is derived from the seat's node, the hub, the
|
||||
* participant and the hub item. `binding_id` is the binding that reports it now, and a new binding for the same seat,
|
||||
* hub and participant takes it over. `settled_at` is when nothing more is reported (the result was sent, the outcome is
|
||||
* unknown, or the hub item was closed elsewhere: `outcome` says which); `checked_at` gives every open receipt a turn. */
|
||||
export const networkReceiptsSchema: Migration = {
|
||||
name: "161_network_receipts.sql",
|
||||
sql: `
|
||||
CREATE TABLE IF NOT EXISTS network_receipts (
|
||||
local_qitem_id TEXT PRIMARY KEY,
|
||||
binding_id TEXT NOT NULL,
|
||||
hub_qitem_id TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
queued_sent_at TEXT,
|
||||
pickup_sent_at TEXT,
|
||||
settled_at TEXT,
|
||||
checked_at TEXT,
|
||||
outcome TEXT,
|
||||
last_error TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_network_receipts_hub ON network_receipts(binding_id, hub_qitem_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_network_receipts_open ON network_receipts(binding_id) WHERE settled_at IS NULL;
|
||||
`,
|
||||
};
|
||||
@@ -26,6 +26,11 @@ export interface HookActivityInput {
|
||||
sourceSequence?: number | null;
|
||||
/** How often the emitter promises to report while it stays idle (OpenRig's pi runner). */
|
||||
livenessSeconds?: number | null;
|
||||
/** #99: the reset instant a Claude usage limit stated; kept only on that typed failure. */
|
||||
resetsAt?: string | null;
|
||||
/** The provider's own session id the hook reports (Claude's session_id). A stated reset is kept only when this is
|
||||
* the seat's own Claude session (its resume token), so a helper on another login can't park the seat's pool. */
|
||||
sessionId?: string | null;
|
||||
}
|
||||
|
||||
export type RecordHookActivityResult =
|
||||
@@ -113,6 +118,16 @@ export class AgentActivityStore {
|
||||
} catch { return unavailablePermissionStop(); }
|
||||
}
|
||||
|
||||
/** #99: whether a hook's Claude session id is the seat's own Claude session (its current resume token). A helper
|
||||
* Claude started from the seat's shell posts as the seat but runs its own session, possibly on another login. */
|
||||
private isSeatOwnClaudeSession(nodeId: string, hookSessionId: string | null | undefined): boolean {
|
||||
const token = typeof hookSessionId === "string" ? hookSessionId.trim() : "";
|
||||
if (!token) return false;
|
||||
const row = this.db.prepare("SELECT resume_type, resume_token FROM sessions WHERE node_id = ? ORDER BY id DESC LIMIT 1")
|
||||
.get(nodeId) as { resume_type: string | null; resume_token: string | null } | undefined;
|
||||
return row?.resume_type === "claude_id" && row.resume_token?.trim() === token;
|
||||
}
|
||||
|
||||
recordHookEvent(input: HookActivityInput): RecordHookActivityResult {
|
||||
if (!input.sessionName && !input.nodeId) {
|
||||
return {
|
||||
@@ -159,6 +174,10 @@ export class AgentActivityStore {
|
||||
if (input.hookAgentId !== undefined) activity.hookAgentId = input.hookAgentId;
|
||||
if (input.sourceSequence) activity.sourceSequence = input.sourceSequence;
|
||||
if (input.livenessSeconds) activity.livenessSeconds = input.livenessSeconds;
|
||||
if (input.resetsAt && isClaudeRateLimitHook(activity.runtime, input.hookEvent, input.subtype)
|
||||
&& this.isSeatOwnClaudeSession(session.node_id, input.sessionId)) {
|
||||
activity.resetsAt = input.resetsAt;
|
||||
}
|
||||
|
||||
const event = this.eventBus.emit({
|
||||
type: "agent.activity",
|
||||
|
||||
@@ -2,7 +2,7 @@ import { physicalSessionName } from "./session-registry.js";
|
||||
import { resolveNodeFile } from "./scope/node-file.js";
|
||||
import { belongsToProject } from "./workspace/project-catalog.js";
|
||||
import { inspectGraph } from "./workflow-reconciliation.js";
|
||||
import { createProofPolicyRead, readMissionReadiness, readProjectReadiness } from "./proof/judgments.js";
|
||||
import { createProofPolicyRead, readMissionReadiness, readProjectReadinessAsync } from "./proof/judgments.js";
|
||||
import { lifecycleObligations, requiredLifecycleSteps } from "./lifecycle-obligations.js";
|
||||
import { QueueWakeRepository } from "./queue-wake-repository.js";
|
||||
import { isTypedGateBlocker } from "./queue-repository.js";
|
||||
@@ -1361,7 +1361,7 @@ export async function buildExecutionView(deps: ExecutionViewDeps, opts?: { missi
|
||||
...(opts?.project ? { project: opts.project, membership: "exact project:<id> queue tags and lifecycle identity; unscoped rows excluded" } : {}),
|
||||
view: "execution",
|
||||
readiness,
|
||||
project_readiness: missionsRoot ? readProjectReadiness(missionsRoot, readPolicy) : null,
|
||||
project_readiness: missionsRoot ? await readProjectReadinessAsync(missionsRoot, readPolicy) : null,
|
||||
// Authored guidance is carried verbatim in meaning, never parsed into edges or acceptance.
|
||||
planning_guidance: arrangement?.state === "valid" ? arrangement.guidance : [],
|
||||
mission,
|
||||
|
||||
@@ -967,7 +967,7 @@ function verbToClosure(input: MissionControlActionInput): ClosureMapping {
|
||||
* person's own receipt says why). Undefined for a request the answer closed (no agent wired here). */
|
||||
function runnerAnswerNotice(ask: QueueItem): string | undefined {
|
||||
if (ask.state === "handed-off" && ask.handedOffTo) {
|
||||
return `Your answer on ${ask.qitemId} went to ${ask.handedOffTo}, the receiving agent, which decides. The runner waits until then; nothing has run.`;
|
||||
return `Your answer on ${ask.qitemId} went to ${ask.handedOffTo}, the receiving agent, which decides. The runner waits until then; nothing has run. A note to you says the agent's recorded state.`;
|
||||
}
|
||||
if (ask.state === "pending" || ask.state === "in-progress" || ask.state === "blocked") {
|
||||
return `Your answer is kept on ${ask.qitemId}, and the request is still waiting: the tool hasn't run, and nothing was permitted or refused. A note to you says why it wasn't passed to a receiving agent. To decide directly: rig seat decide-permission ${ask.qitemId} --option <id>.`;
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
// The connector's view of a remote hub: the queue tools an external grant offers its participant (list its own items,
|
||||
// claim, show, update), reached over MCP's Streamable HTTP with that participant's bearer. Nothing here decides what to
|
||||
// do with an item; it only says what the hub answered, and sorts a failure into what the caller must do about it.
|
||||
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
||||
import { StreamableHTTPClientTransport, StreamableHTTPError } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
|
||||
|
||||
export interface HubItem {
|
||||
qitemId: string;
|
||||
sourceSession: string;
|
||||
destinationSession: string;
|
||||
state: string;
|
||||
body?: string | null;
|
||||
summary?: string | null;
|
||||
resolution?: string | null;
|
||||
closureReason?: string | null;
|
||||
}
|
||||
|
||||
/** `revoked`: the hub refused the credential (401), so the binding ends. `unavailable`: the hub or its grant authority
|
||||
* couldn't answer, or the connection failed, so try later. `refused`: the hub answered this one call with an error. */
|
||||
export type HubErrorKind = "revoked" | "unavailable" | "refused";
|
||||
|
||||
export class HubError extends Error {
|
||||
constructor(readonly kind: HubErrorKind, message: string) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
export interface HubSession {
|
||||
listOwned(state: "pending" | "in-progress", limit: number): Promise<HubItem[]>;
|
||||
claim(qitemId: string): Promise<void>;
|
||||
show(qitemId: string): Promise<HubItem>;
|
||||
note(qitemId: string, note: string): Promise<void>;
|
||||
/** Close the way the queue closes an item: done with a closure reason (no-follow-on, canceled or denied), or failed. */
|
||||
close(qitemId: string, input: { state: "done" | "failed"; closureReason: string | null; resolution: string }): Promise<void>;
|
||||
end(): Promise<void>;
|
||||
}
|
||||
|
||||
/** `signal` closes the connection when it aborts, which ends any call still waiting on the hub. */
|
||||
export type HubConnect = (hubUrl: string, token: string, signal?: AbortSignal) => Promise<HubSession>;
|
||||
|
||||
/** A failure, sorted. The hub's own MCP endpoint answers 401 for an unknown or revoked credential and 503 when its grant
|
||||
* authority can't answer (`packages/cli/src/mcp-http.ts`). */
|
||||
export function hubFailure(error: unknown): HubError {
|
||||
if (error instanceof HubError) return error;
|
||||
if (error instanceof StreamableHTTPError && error.code === 401) {
|
||||
return new HubError("revoked", "The hub refused this participant's credential: it's unknown there or was revoked.");
|
||||
}
|
||||
if (error instanceof StreamableHTTPError && typeof error.code === "number" && error.code >= 300 && error.code < 400) {
|
||||
return new HubError("unavailable", `The hub redirected the connection somewhere else, and the participant's token isn't sent there (${error.message.slice(0, 300)}). Connect the binding to the hub's own URL.`);
|
||||
}
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
return new HubError("unavailable", `The hub couldn't be reached: ${message.slice(0, 300)}`);
|
||||
}
|
||||
|
||||
function parsed(result: { isError?: boolean; content?: unknown }): unknown {
|
||||
const content = Array.isArray(result.content) ? result.content as Array<{ type?: string; text?: string }> : [];
|
||||
const text = content.filter((part) => part.type === "text").map((part) => part.text ?? "").join("\n");
|
||||
if (result.isError) throw new HubError("refused", text.slice(0, 500) || "The hub refused this call.");
|
||||
try { return JSON.parse(text); } catch { throw new HubError("refused", "The hub answered with something that isn't JSON."); }
|
||||
}
|
||||
|
||||
const item = (value: unknown): HubItem => {
|
||||
const row = (value ?? {}) as Record<string, unknown>;
|
||||
if (typeof row.qitemId !== "string") throw new HubError("refused", "The hub answered without an item id.");
|
||||
return {
|
||||
qitemId: row.qitemId,
|
||||
sourceSession: String(row.sourceSession ?? ""),
|
||||
destinationSession: String(row.destinationSession ?? ""),
|
||||
state: String(row.state ?? ""),
|
||||
body: typeof row.body === "string" ? row.body : null,
|
||||
summary: typeof row.summary === "string" ? row.summary : null,
|
||||
resolution: typeof row.resolution === "string" ? row.resolution : null,
|
||||
closureReason: typeof row.closureReason === "string" ? row.closureReason : null,
|
||||
};
|
||||
};
|
||||
|
||||
/** Connect to a hub as one participant. The bearer goes only in this transport's Authorization header. The SDK's
|
||||
* same-origin policy, set explicitly, follows a redirect only within the hub's origin, or to its https form on the same
|
||||
* host with default ports, so the bearer follows only those; any other redirect fails with its target named. */
|
||||
export const connectMcpHub: HubConnect = async (hubUrl, token, signal) => {
|
||||
const transport = new StreamableHTTPClientTransport(new URL(hubUrl), {
|
||||
requestInit: { headers: { Authorization: `Bearer ${token}` } },
|
||||
redirectPolicy: "same-origin",
|
||||
});
|
||||
const client = new Client({ name: "openrig-network-connector", version: "1" });
|
||||
const close = () => { void client.close().catch(() => {}); };
|
||||
if (signal?.aborted) throw new HubError("unavailable", "The pass ended before it reached the hub.");
|
||||
signal?.addEventListener("abort", close, { once: true });
|
||||
try {
|
||||
await client.connect(transport);
|
||||
} catch (error) {
|
||||
signal?.removeEventListener("abort", close);
|
||||
await client.close().catch(() => {});
|
||||
throw hubFailure(error);
|
||||
}
|
||||
const call = async (name: string, args: Record<string, unknown>): Promise<unknown> => {
|
||||
try {
|
||||
return parsed(await client.callTool({ name, arguments: args }) as { isError?: boolean; content?: unknown });
|
||||
} catch (error) {
|
||||
throw hubFailure(error);
|
||||
}
|
||||
};
|
||||
return {
|
||||
listOwned: async (state, limit) => {
|
||||
const rows = await call("rig_queue_list", { owned: true, state, limit });
|
||||
if (!Array.isArray(rows)) throw new HubError("refused", "The hub's list wasn't a list.");
|
||||
return rows.map(item);
|
||||
},
|
||||
claim: async (qitemId) => { await call("rig_queue_claim", { qitemId }); },
|
||||
show: async (qitemId) => {
|
||||
const shown = await call("rig_queue_show", { qitemId }) as Record<string, unknown>;
|
||||
return item(shown?.item ?? shown);
|
||||
},
|
||||
note: async (qitemId, note) => { await call("rig_queue_update", { qitemId, note }); },
|
||||
close: async (qitemId, input) => {
|
||||
await call("rig_queue_update", { qitemId, state: input.state, ...(input.closureReason ? { closureReason: input.closureReason } : {}), resolution: input.resolution });
|
||||
},
|
||||
end: async () => { signal?.removeEventListener("abort", close); await client.close().catch(() => {}); },
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,238 @@
|
||||
// A selected local seat joined to one participant of a remote hub: the account's network rig, which this daemon reaches
|
||||
// as an MCP client with that participant's own credential (an external grant). The binding names the local seat by its
|
||||
// durable node id, so a renamed or relaunched seat keeps it; the hub address represents the seat and never renames it.
|
||||
// The credential is a private file under the OpenRig home; this table only says where the binding stands.
|
||||
import type Database from "better-sqlite3";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { closeSync, existsSync, lstatSync, mkdirSync, openSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { resolveSessionNodeId } from "../queue-owner.js";
|
||||
|
||||
export type NetworkBindingState = "active" | "revoked" | "disconnected";
|
||||
|
||||
export interface NetworkBinding {
|
||||
id: string;
|
||||
nodeId: string;
|
||||
/** The seat's address when it was bound, for display; the node id is what the binding follows. */
|
||||
localSession: string;
|
||||
hubUrl: string;
|
||||
/** The participant's address in the hub (`member@rig`). */
|
||||
hubSeat: string;
|
||||
/** The person chose, for this binding, to send the participant's token to a plain-http hub. */
|
||||
allowHttp: boolean;
|
||||
/** The hub URL is plain http, so the token and the work cross the network unencrypted. */
|
||||
unencrypted: boolean;
|
||||
state: NetworkBindingState;
|
||||
stateReason: string | null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
lastContactAt: string | null;
|
||||
lastError: string | null;
|
||||
}
|
||||
|
||||
export class NetworkBindingError extends Error {
|
||||
constructor(readonly code: string, message: string, readonly status: 400 | 404 | 409 | 500 = 400) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
interface Row {
|
||||
id: string; node_id: string; local_session: string; hub_url: string; hub_seat: string; allow_http: number; state: NetworkBindingState;
|
||||
state_reason: string | null; created_at: string; updated_at: string; last_contact_at: string | null; last_error: string | null;
|
||||
}
|
||||
|
||||
const project = (row: Row): NetworkBinding => ({
|
||||
id: row.id, nodeId: row.node_id, localSession: row.local_session, hubUrl: row.hub_url, hubSeat: row.hub_seat,
|
||||
allowHttp: row.allow_http === 1, unencrypted: row.hub_url.startsWith("http:"), state: row.state, stateReason: row.state_reason, createdAt: row.created_at, updatedAt: row.updated_at,
|
||||
lastContactAt: row.last_contact_at, lastError: row.last_error,
|
||||
});
|
||||
|
||||
const MAX_CREDENTIAL_BYTES = 16384;
|
||||
|
||||
/** Where a binding's credential lives: `<home>/network-secrets/<binding id>/credential.json`, private to this user. */
|
||||
export function networkCredentialFile(home: string, bindingId: string): string {
|
||||
if (!/^netb-[0-9a-f]{20}$/.test(bindingId)) throw new NetworkBindingError("invalid_binding_id", `'${bindingId}' isn't a network binding id.`);
|
||||
return join(home, "network-secrets", bindingId, "credential.json");
|
||||
}
|
||||
|
||||
/** The participant's bearer, read from its private file. A file anyone else can read, a link or an oversized file is
|
||||
* refused, with the path named and the value never shown. */
|
||||
export function readNetworkCredential(home: string, bindingId: string): string {
|
||||
const file = networkCredentialFile(home, bindingId);
|
||||
let stat;
|
||||
try { stat = lstatSync(file); } catch {
|
||||
throw new NetworkBindingError("credential_missing", `The credential for ${bindingId} isn't at ${file}. Connect the seat again with its token file.`, 404);
|
||||
}
|
||||
if (!stat.isFile() || stat.isSymbolicLink() || stat.size > MAX_CREDENTIAL_BYTES || (stat.mode & 0o077) !== 0) {
|
||||
throw new NetworkBindingError("credential_unsafe", `${file} must be a regular file readable only by this user.`);
|
||||
}
|
||||
// A parse error can quote the file's text, which holds the token: say only which file.
|
||||
let value: { version?: unknown; token?: unknown };
|
||||
try { value = JSON.parse(readFileSync(file, "utf8")) as typeof value; } catch {
|
||||
throw new NetworkBindingError("credential_invalid", `${file} isn't a network credential.`);
|
||||
}
|
||||
if (value === null || typeof value !== "object" || value.version !== 1 || typeof value.token !== "string" || !value.token) {
|
||||
throw new NetworkBindingError("credential_invalid", `${file} isn't a network credential.`);
|
||||
}
|
||||
return value.token;
|
||||
}
|
||||
|
||||
function validHubUrl(raw: unknown, allowHttp: boolean): string {
|
||||
if (typeof raw !== "string" || !raw.trim()) throw new NetworkBindingError("invalid_hub_url", "A hub URL is required.");
|
||||
let url: URL;
|
||||
try { url = new URL(raw.trim()); } catch { throw new NetworkBindingError("invalid_hub_url", `'${raw}' isn't a URL.`); }
|
||||
if (url.protocol !== "https:" && url.protocol !== "http:") throw new NetworkBindingError("invalid_hub_url", "The hub URL must use https (or http).");
|
||||
// A credential in the URL would be stored in plain text in this table and shown in status.
|
||||
if (url.username || url.password) throw new NetworkBindingError("invalid_hub_url", "Put the participant's token in its token file, not in the hub URL.");
|
||||
// Plain http is the person's choice per binding. The scheme is kept as given, and the address isn't judged: a private
|
||||
// or tailnet address doesn't prove the connection is protected.
|
||||
if (url.protocol === "http:" && !allowHttp) {
|
||||
throw new NetworkBindingError("http_not_allowed", `${url.toString()} is plain http, so the participant's token and its work would cross the network unencrypted, where they can be read. Use the hub's https URL, or connect again with --allow-http to accept that for this binding.`);
|
||||
}
|
||||
return url.toString();
|
||||
}
|
||||
|
||||
function validHubSeat(raw: unknown): string {
|
||||
if (typeof raw !== "string" || !/^[^@\s]+@[^@\s]+$/.test(raw.trim())) {
|
||||
throw new NetworkBindingError("invalid_hub_seat", "The hub participant is its address in the hub, member@rig.");
|
||||
}
|
||||
return raw.trim();
|
||||
}
|
||||
|
||||
function validToken(raw: unknown): string {
|
||||
const token = typeof raw === "string" ? raw.trim() : "";
|
||||
if (!token || /\s/.test(token) || Buffer.byteLength(token) > 8192) {
|
||||
throw new NetworkBindingError("invalid_token", "The participant's token must be one non-empty line with no spaces.");
|
||||
}
|
||||
return token;
|
||||
}
|
||||
|
||||
export class NetworkBindingStore {
|
||||
constructor(
|
||||
private readonly db: Database.Database,
|
||||
private readonly home: string,
|
||||
private readonly now: () => Date = () => new Date(),
|
||||
private readonly newId: () => string = () => `netb-${randomBytes(10).toString("hex")}`,
|
||||
) {}
|
||||
|
||||
list(): NetworkBinding[] {
|
||||
return (this.db.prepare("SELECT * FROM network_bindings ORDER BY created_at, id").all() as Row[]).map(project);
|
||||
}
|
||||
|
||||
active(): NetworkBinding[] {
|
||||
return (this.db.prepare("SELECT * FROM network_bindings WHERE state = 'active' ORDER BY created_at, id").all() as Row[]).map(project);
|
||||
}
|
||||
|
||||
get(id: string): NetworkBinding | null {
|
||||
const row = this.db.prepare("SELECT * FROM network_bindings WHERE id = ?").get(id) as Row | undefined;
|
||||
return row ? project(row) : null;
|
||||
}
|
||||
|
||||
/** A binding by its id, or the active binding of a local seat named by its address. */
|
||||
find(ref: string): NetworkBinding | null {
|
||||
const byId = this.get(ref);
|
||||
if (byId) return byId;
|
||||
const nodeId = resolveSessionNodeId(this.db, ref);
|
||||
const row = nodeId ? this.db.prepare("SELECT * FROM network_bindings WHERE node_id = ? AND state = 'active'").get(nodeId) as Row | undefined : undefined;
|
||||
return row ? project(row) : null;
|
||||
}
|
||||
|
||||
/** Bind a local seat to a hub participant with that participant's token. The credential is written first, exclusively
|
||||
* and private, then the row. If either step fails, the credential this call created is removed, so a refused bind
|
||||
* leaves none behind; a file that was already at that path is never touched. */
|
||||
create(input: { session: unknown; hubUrl: unknown; hubSeat: unknown; token: unknown; allowHttp?: unknown }): NetworkBinding {
|
||||
const session = typeof input.session === "string" ? input.session.trim() : "";
|
||||
if (!session) throw new NetworkBindingError("invalid_seat", "Name the local seat to connect.");
|
||||
const nodeId = resolveSessionNodeId(this.db, session);
|
||||
if (!nodeId) throw new NetworkBindingError("seat_not_found", `No local seat is named '${session}'.`, 404);
|
||||
const allowHttp = input.allowHttp === true;
|
||||
const hubUrl = validHubUrl(input.hubUrl, allowHttp);
|
||||
const hubSeat = validHubSeat(input.hubSeat);
|
||||
const token = validToken(input.token);
|
||||
const bound = this.db.prepare("SELECT id FROM network_bindings WHERE node_id = ? AND state = 'active'").get(nodeId) as { id: string } | undefined;
|
||||
if (bound) throw new NetworkBindingError("seat_already_connected", `${session} is already connected (${bound.id}). Disconnect it first.`, 409);
|
||||
const taken = this.db.prepare("SELECT id FROM network_bindings WHERE hub_url = ? AND hub_seat = ? AND state = 'active'").get(hubUrl, hubSeat) as { id: string } | undefined;
|
||||
if (taken) throw new NetworkBindingError("participant_already_connected", `${hubSeat} at ${hubUrl} is already connected (${taken.id}).`, 409);
|
||||
|
||||
const id = this.newId();
|
||||
const file = networkCredentialFile(this.home, id);
|
||||
let created = false;
|
||||
let madeFolder: string | undefined;
|
||||
try {
|
||||
madeFolder = mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
|
||||
const fd = openSync(file, "wx", 0o600);
|
||||
created = true;
|
||||
try { writeFileSync(fd, `${JSON.stringify({ version: 1, token })}\n`); } finally { closeSync(fd); }
|
||||
const at = this.now().toISOString();
|
||||
this.db.prepare(`INSERT INTO network_bindings (id, node_id, local_session, hub_url, hub_seat, allow_http, state, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active', ?, ?)`).run(id, nodeId, session, hubUrl, hubSeat, allowHttp ? 1 : 0, at, at);
|
||||
} catch (error) {
|
||||
// Only what this call made: the file it opened, and this binding's folder if it made it (`mkdirSync` returns the
|
||||
// first folder it made, so any value means this one is new). A folder that was already there keeps its other files.
|
||||
if (created) rmSync(file, { force: true });
|
||||
if (madeFolder !== undefined) rmSync(dirname(file), { recursive: true, force: true });
|
||||
throw error;
|
||||
}
|
||||
return this.get(id)!;
|
||||
}
|
||||
|
||||
/** The person ends this binding here. No new work comes in through it; local rows, results and other bindings stay. The
|
||||
* credential is removed, and the hub still lists the participant until it's removed there too. */
|
||||
disconnect(ref: string, reason?: string): NetworkBinding {
|
||||
return this.end(ref, "disconnected", reason?.trim() || "disconnected here");
|
||||
}
|
||||
|
||||
/** The hub refused the credential: it was revoked there. Same effect as a local disconnect. */
|
||||
markRevoked(id: string, reason: string): NetworkBinding {
|
||||
return this.end(id, "revoked", reason);
|
||||
}
|
||||
|
||||
/** The connector reached the hub as this binding. */
|
||||
recordContact(id: string, at: Date = this.now()): void {
|
||||
this.db.prepare("UPDATE network_bindings SET last_contact_at = ?, last_error = NULL, updated_at = ? WHERE id = ?")
|
||||
.run(at.toISOString(), at.toISOString(), id);
|
||||
}
|
||||
|
||||
/** The connector couldn't reach the hub as this binding, or the hub answered with an error; the binding stays active.
|
||||
* Control characters are replaced, since the text can come from the hub and `rig network status` prints it. */
|
||||
recordError(id: string, message: string): void {
|
||||
this.db.prepare("UPDATE network_bindings SET last_error = ?, updated_at = ? WHERE id = ?")
|
||||
.run(message.replace(/[\u0000-\u001f\u007f-\u009f]/g, " ").slice(0, 500), this.now().toISOString(), id);
|
||||
}
|
||||
|
||||
/** The binding ends first, so no new work comes in even if the credential can't be removed; removal is tried on every
|
||||
* call, including again on an already ended binding, until its credential is gone. */
|
||||
private end(ref: string, state: "disconnected" | "revoked", reason: string): NetworkBinding {
|
||||
const binding = this.find(ref);
|
||||
if (!binding) throw new NetworkBindingError("binding_not_found", `No network binding or connected seat is named '${ref}'.`, 404);
|
||||
if (binding.state === "active") {
|
||||
this.db.prepare("UPDATE network_bindings SET state = ?, state_reason = ?, updated_at = ? WHERE id = ? AND state = 'active'")
|
||||
.run(state, reason, this.now().toISOString(), binding.id);
|
||||
}
|
||||
try {
|
||||
this.removeCredential(binding.id);
|
||||
} catch (error) {
|
||||
throw new NetworkBindingError("credential_cleanup_failed",
|
||||
`${binding.id} is ${this.get(binding.id)!.state}, but its credential at ${networkCredentialFile(this.home, binding.id)} couldn't be removed (${(error as Error).message}). Run \`rig network disconnect ${binding.id}\` again to retry.`, 500);
|
||||
}
|
||||
return this.get(binding.id)!;
|
||||
}
|
||||
|
||||
/** Removes the credential folders whose binding isn't active: one a stop left between the credential write and its
|
||||
* row, or one whose removal failed when its binding ended. Other names under the folder are left alone. */
|
||||
sweepCredentials(): { removed: string[]; failed: string[] } {
|
||||
const swept = { removed: [] as string[], failed: [] as string[] };
|
||||
let names: string[];
|
||||
try { names = readdirSync(join(this.home, "network-secrets")); } catch { return swept; }
|
||||
for (const name of names.sort()) {
|
||||
if (!/^netb-[0-9a-f]{20}$/.test(name) || this.get(name)?.state === "active") continue;
|
||||
try { this.removeCredential(name); swept.removed.push(name); } catch { swept.failed.push(name); }
|
||||
}
|
||||
return swept;
|
||||
}
|
||||
|
||||
private removeCredential(id: string): void {
|
||||
const file = networkCredentialFile(this.home, id);
|
||||
if (existsSync(dirname(file))) rmSync(dirname(file), { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,405 @@
|
||||
// The outbound connector: for each active network binding, reach the hub as that participant, take the items addressed
|
||||
// to it into the bound local seat's queue, and carry what happens locally back to the hub item.
|
||||
//
|
||||
// Honest stages, kept apart on the hub item:
|
||||
// - a hub claim is the connector taking custody of the item, not the agent picking it up;
|
||||
// - "picked up" is reported only once the local item is claimed;
|
||||
// - the result is the local item's own closing outcome.
|
||||
//
|
||||
// Each hub item maps to one local item, under an id derived from what it is: the seat's node, the hub, the participant
|
||||
// and the hub item. The queue's same-id absorption makes every retry, restart or new binding for the same seat, hub and
|
||||
// participant land on that one item, so a lost response never starts a second one. An item whose local record is gone
|
||||
// reads unknown and is never run again.
|
||||
import type Database from "better-sqlite3";
|
||||
import { createHash } from "node:crypto";
|
||||
import { QueueRepositoryError, type QueueItem, type QueueRepository } from "../queue-repository.js";
|
||||
import { NetworkBindingError, readNetworkCredential, type NetworkBinding, type NetworkBindingStore } from "./network-bindings.js";
|
||||
import { connectMcpHub, HubError, type HubConnect, type HubItem, type HubSession } from "./hub-client.js";
|
||||
|
||||
/** OpenRig's own sender: the connector creates the local item for the hub's requester, who is named first in it. */
|
||||
const LOCAL_SOURCE = "daemon@kernel";
|
||||
const PAGE = 25;
|
||||
const MAX_PAGES = 8;
|
||||
/** Receipts looked at per pass, the least recently looked at first, so a long backlog still gives every item a turn. */
|
||||
const REPORT_LIMIT = 50;
|
||||
/** How long one binding's pass may take before its connection is closed, so one slow hub can't hold the others. */
|
||||
const PASS_TIMEOUT_MS = 60_000;
|
||||
|
||||
/** The local item for a hub item, by what it is. A retry, a restart or a new binding for the same seat, hub and
|
||||
* participant lands on the same item. */
|
||||
export function networkLocalItemId(binding: Pick<NetworkBinding, "nodeId" | "hubUrl" | "hubSeat">, hubQitemId: string): string {
|
||||
return `qitem-net-${createHash("sha256").update([binding.nodeId, binding.hubUrl, binding.hubSeat, hubQitemId].join("\n")).digest("hex").slice(0, 20)}`;
|
||||
}
|
||||
|
||||
export interface NetworkConnectorDeps {
|
||||
db: Database.Database;
|
||||
queueRepo: QueueRepository;
|
||||
bindings: NetworkBindingStore;
|
||||
home: string;
|
||||
connect?: HubConnect;
|
||||
now?: () => Date;
|
||||
log?: (line: string) => void;
|
||||
passTimeoutMs?: number;
|
||||
reportLimit?: number;
|
||||
}
|
||||
|
||||
export interface NetworkPassResult {
|
||||
bindingId: string;
|
||||
taken: number;
|
||||
pickupsSent: number;
|
||||
resultsSent: number;
|
||||
unknown: number;
|
||||
/** Why the pass stopped early, if it did. `revoked` ended the binding; `no_session` left the hub's items pending;
|
||||
* `local` is a failure here, not the hub's. */
|
||||
error?: HubError["kind"] | "credential" | "no_session" | "http_not_allowed" | "local";
|
||||
}
|
||||
|
||||
/** The seat's current address: the node's latest session, as the queue's other node lookups read it. */
|
||||
function currentSession(db: Database.Database, nodeId: string): string | null {
|
||||
const row = db.prepare("SELECT session_name FROM sessions WHERE node_id = ? ORDER BY created_at DESC, id DESC LIMIT 1")
|
||||
.get(nodeId) as { session_name: string } | undefined;
|
||||
return row?.session_name ?? null;
|
||||
}
|
||||
|
||||
interface Receipt {
|
||||
local_qitem_id: string; binding_id: string; hub_qitem_id: string; queued_sent_at: string | null; pickup_sent_at: string | null;
|
||||
settled_at: string | null; outcome: string | null;
|
||||
}
|
||||
|
||||
/** The states a local item ends in. A handed-off local item ends at its successor, which is followed. */
|
||||
const FINAL = new Set(["done", "canceled", "denied", "failed"]);
|
||||
/** The states in which the hub item is closed to this connector. A hub item handed on by someone there is closed here:
|
||||
* its successor isn't followed, and it's never written to again. */
|
||||
const HUB_CLOSED = new Set([...FINAL, "handed-off"]);
|
||||
|
||||
/** One pass for one binding, bounded by its deadline and the connector's stop. Never throws: a failure is recorded on
|
||||
* the binding and returned. */
|
||||
export async function runNetworkBindingPass(deps: NetworkConnectorDeps, binding: NetworkBinding,
|
||||
opts: { signal?: AbortSignal } = {}): Promise<NetworkPassResult> {
|
||||
const now = deps.now ?? (() => new Date());
|
||||
const result: NetworkPassResult = { bindingId: binding.id, taken: 0, pickupsSent: 0, resultsSent: 0, unknown: 0 };
|
||||
// The token goes to a plain-http hub only on the person's recorded choice for this binding.
|
||||
if (binding.unencrypted && !binding.allowHttp) {
|
||||
deps.bindings.recordError(binding.id, `${binding.hubUrl} is plain http and this binding has no --allow-http choice, so its token isn't sent. Connect again with the hub's https URL, or with --allow-http.`);
|
||||
return { ...result, error: "http_not_allowed" };
|
||||
}
|
||||
let token: string;
|
||||
try {
|
||||
token = readNetworkCredential(deps.home, binding.id);
|
||||
} catch (error) {
|
||||
deps.bindings.recordError(binding.id, error instanceof NetworkBindingError ? error.message : String(error));
|
||||
return { ...result, error: "credential" };
|
||||
}
|
||||
|
||||
const timeoutMs = deps.passTimeoutMs ?? PASS_TIMEOUT_MS;
|
||||
const deadline = new AbortController();
|
||||
const timer = setTimeout(() => deadline.abort(), timeoutMs);
|
||||
timer.unref?.();
|
||||
const onStop = () => deadline.abort();
|
||||
if (opts.signal?.aborted) deadline.abort();
|
||||
opts.signal?.addEventListener("abort", onStop, { once: true });
|
||||
// Checked before every take, and again just before a local item is made: an ended binding, the pass's deadline or
|
||||
// the connector's stop starts no local work, even when a hub answer arrives late.
|
||||
const live = () => !deadline.signal.aborted && deps.bindings.get(binding.id)?.state === "active";
|
||||
|
||||
let hub: HubSession | null = null;
|
||||
try {
|
||||
hub = await (deps.connect ?? connectMcpHub)(binding.hubUrl, token, deadline.signal);
|
||||
// Work goes to the seat at its current address. With no session there's nobody to give it to, so nothing is taken.
|
||||
const destination = currentSession(deps.db, binding.nodeId);
|
||||
if (destination) {
|
||||
// What this participant already holds (claimed before a restart, or through an earlier binding) comes first.
|
||||
for (const held of await hub.listOwned("in-progress", PAGE * MAX_PAGES)) {
|
||||
if (!live()) break;
|
||||
if (held.destinationSession === binding.hubSeat && await ensureLocal(deps, binding, hub, held, destination, now, live)) result.taken++;
|
||||
}
|
||||
// New work, a page at a time: a claim moves an item out of pending, so each page is new items.
|
||||
for (let page = 0; page < MAX_PAGES && live(); page++) {
|
||||
const pending = await hub.listOwned("pending", PAGE);
|
||||
for (const item of pending) {
|
||||
if (!live()) break;
|
||||
if (item.destinationSession !== binding.hubSeat) continue;
|
||||
try {
|
||||
await hub.claim(item.qitemId);
|
||||
} catch (error) {
|
||||
// Another claimant, or the item closed meanwhile: it isn't this binding's to take.
|
||||
if (error instanceof HubError && error.kind === "refused") continue;
|
||||
throw error;
|
||||
}
|
||||
if (await ensureLocal(deps, binding, hub, item, destination, now, live)) result.taken++;
|
||||
}
|
||||
if (pending.length < PAGE) break;
|
||||
}
|
||||
}
|
||||
// Report back what happened locally, for the items this binding holds. A revoked binding never gets here.
|
||||
if (live()) await reportBack(deps, binding, hub, now, result);
|
||||
deps.bindings.recordContact(binding.id, now());
|
||||
if (!destination) {
|
||||
deps.bindings.recordError(binding.id, `${binding.localSession} has no session here, so its hub items stay pending until it has one.`);
|
||||
return { ...result, error: "no_session" };
|
||||
}
|
||||
return result;
|
||||
} catch (error) {
|
||||
if (deadline.signal.aborted) {
|
||||
deps.bindings.recordError(binding.id, opts.signal?.aborted
|
||||
? "The connector stopped during this pass and closed its connection to the hub."
|
||||
: `This pass took longer than ${timeoutMs >= 1000 ? `${Math.round(timeoutMs / 1000)} s` : `${timeoutMs} ms`}, so its connection to the hub was closed. The next pass tries again.`);
|
||||
return { ...result, error: "unavailable" };
|
||||
}
|
||||
// Only the hub session's own failures are the hub's (it reports them as HubError); anything else happened here.
|
||||
if (!(error instanceof HubError)) {
|
||||
deps.bindings.recordError(binding.id, `A local error stopped this pass: ${error instanceof Error ? error.message : String(error)}`);
|
||||
return { ...result, error: "local" };
|
||||
}
|
||||
if (error.kind === "revoked") {
|
||||
try {
|
||||
deps.bindings.markRevoked(binding.id, error.message);
|
||||
} catch (cleanup) {
|
||||
// The binding is revoked either way; only its credential's removal failed, and the error says how to retry.
|
||||
if (!(cleanup instanceof NetworkBindingError) || cleanup.code !== "credential_cleanup_failed") throw cleanup;
|
||||
deps.bindings.recordError(binding.id, cleanup.message);
|
||||
}
|
||||
deps.log?.(`network ${binding.id}: revoked by the hub; no new work comes in through it`);
|
||||
} else {
|
||||
deps.bindings.recordError(binding.id, error.message);
|
||||
}
|
||||
return { ...result, error: error.kind };
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
opts.signal?.removeEventListener("abort", onStop);
|
||||
await hub?.end();
|
||||
}
|
||||
}
|
||||
|
||||
/** The local item for a hub item this participant holds: created once, or found again. Returns true when it was new. */
|
||||
async function ensureLocal(deps: NetworkConnectorDeps, binding: NetworkBinding, hub: HubSession, item: HubItem, destination: string,
|
||||
now: () => Date, live: () => boolean): Promise<boolean> {
|
||||
const localId = networkLocalItemId(binding, item.qitemId);
|
||||
const taken = deps.db.prepare("SELECT binding_id FROM network_receipts WHERE local_qitem_id = ?").get(localId) as { binding_id: string } | undefined;
|
||||
if (taken) {
|
||||
// Taken already, perhaps through an earlier binding for this seat, hub and participant: this binding reports it now,
|
||||
// with what was already sent and its outcome, unknown included, kept.
|
||||
if (taken.binding_id !== binding.id) deps.db.prepare("UPDATE network_receipts SET binding_id = ? WHERE local_qitem_id = ?").run(binding.id, localId);
|
||||
return false;
|
||||
}
|
||||
const full = item.body == null ? await hub.show(item.qitemId) : item;
|
||||
if (!live()) return false;
|
||||
let outcome: string | null = null;
|
||||
try {
|
||||
await deps.queueRepo.create({
|
||||
qitemId: localId,
|
||||
sourceSession: LOCAL_SOURCE,
|
||||
destinationSession: destination,
|
||||
// Where it came from is read first: the requester and the hub, before the requester's own words.
|
||||
summary: full.summary ? `From ${item.sourceSession} via network hub: ${full.summary}` : `From ${item.sourceSession} via network hub`,
|
||||
body: [
|
||||
`From ${item.sourceSession} through the network hub ${binding.hubUrl}, as ${binding.hubSeat} (hub item ${item.qitemId}).`,
|
||||
"",
|
||||
full.body ?? "",
|
||||
"",
|
||||
"---",
|
||||
`Answer by closing this item with your result: rig queue update ${localId} --state done --closure-reason no-follow-on --resolution "<your answer>".`,
|
||||
"OpenRig returns the resolution to the hub item.",
|
||||
].join("\n"),
|
||||
tags: [`network-binding:${binding.id}`, `network-hub-item:${item.qitemId}`],
|
||||
});
|
||||
} catch (error) {
|
||||
const code = error instanceof QueueRepositoryError ? error.code : null;
|
||||
// Its id was used before and its record erased: whether it ran is unknown, and it's never run again. The hub hears
|
||||
// that from reportBack, which retries the note until it lands.
|
||||
if (code === "qitem_erased") outcome = "unknown";
|
||||
// The same id with another destination: the seat's address changed between attempts; the item already exists.
|
||||
else if (code !== "qitem_id_reuse" || !deps.queueRepo.getById(localId)) throw error;
|
||||
}
|
||||
const inserted = deps.db.prepare(`INSERT INTO network_receipts (local_qitem_id, binding_id, hub_qitem_id, created_at, outcome)
|
||||
VALUES (?, ?, ?, ?, ?) ON CONFLICT DO NOTHING`).run(localId, binding.id, item.qitemId, now().toISOString(), outcome).changes > 0;
|
||||
// Every stage note, queued included, goes out from reportBack, which marks it sent only once it lands.
|
||||
return inserted && outcome === null;
|
||||
}
|
||||
|
||||
/** The item a local item ends at, following local handoffs: a successor names the item it came from. Null when a
|
||||
* handed-off item has no one successor here (handed to another host, or the record is gone): its outcome is unknown. */
|
||||
function finalItem(repo: QueueRepository, start: QueueItem): QueueItem | null {
|
||||
let current = start;
|
||||
for (let hop = 0; hop < 20 && current.state === "handed-off"; hop++) {
|
||||
const next = repo.list({ handedOffFrom: current.qitemId, limit: 2 });
|
||||
if (next.length !== 1) return null;
|
||||
current = next[0]!;
|
||||
}
|
||||
return current.state === "handed-off" ? null : current;
|
||||
}
|
||||
|
||||
/** What the hub hears as the result: the resolution, else the closer's own note on the close itself. The close is read
|
||||
* with `closingTransition`, since later notes on a closed item repeat its state. Other notes on the item can be
|
||||
* OpenRig's or another seat's, and name local seats, so they're never sent. */
|
||||
function closingWords(repo: QueueRepository, item: QueueItem): string {
|
||||
if (item.resolution) return item.resolution;
|
||||
const close = repo.closingTransition(item.qitemId, item.state);
|
||||
return close?.transitionNote && close.actorSession === item.destinationSession ? close.transitionNote : "Closed without a stated result.";
|
||||
}
|
||||
|
||||
/** The hub items a binding's result never reached because someone else closed them first, for status. */
|
||||
export function networkClosedElsewhere(db: Database.Database, bindingId: string): Array<{ hubItem: string; localItem: string; note: string | null }> {
|
||||
return (db.prepare("SELECT hub_qitem_id, local_qitem_id, last_error FROM network_receipts WHERE binding_id = ? AND outcome = 'closed_on_hub' ORDER BY settled_at, local_qitem_id")
|
||||
.all(bindingId) as Array<{ hub_qitem_id: string; local_qitem_id: string; last_error: string | null }>)
|
||||
.map((row) => ({ hubItem: row.hub_qitem_id, localItem: row.local_qitem_id, note: row.last_error }));
|
||||
}
|
||||
|
||||
async function reportBack(deps: NetworkConnectorDeps, binding: NetworkBinding, hub: HubSession, now: () => Date, result: NetworkPassResult): Promise<void> {
|
||||
const open = deps.db.prepare(`SELECT * FROM network_receipts WHERE binding_id = ? AND settled_at IS NULL
|
||||
ORDER BY COALESCE(checked_at, ''), created_at LIMIT ?`).all(binding.id, deps.reportLimit ?? REPORT_LIMIT) as Receipt[];
|
||||
const mark = (column: "queued_sent_at" | "pickup_sent_at" | "settled_at", receipt: Receipt, outcome?: string, lastError: string | null = null) =>
|
||||
deps.db.prepare(`UPDATE network_receipts SET ${column} = ?, outcome = COALESCE(?, outcome), last_error = ? WHERE local_qitem_id = ?`)
|
||||
.run(now().toISOString(), outcome ?? null, lastError, receipt.local_qitem_id);
|
||||
for (const receipt of open) {
|
||||
deps.db.prepare("UPDATE network_receipts SET checked_at = ? WHERE local_qitem_id = ?").run(now().toISOString(), receipt.local_qitem_id);
|
||||
try {
|
||||
const local = deps.queueRepo.getById(receipt.local_qitem_id);
|
||||
const unknown = async (why: string) => {
|
||||
await hub.note(receipt.hub_qitem_id, why);
|
||||
mark("settled_at", receipt, "unknown");
|
||||
result.unknown++;
|
||||
};
|
||||
if (!local || receipt.outcome === "unknown") {
|
||||
await unknown(`This item's local record (${receipt.local_qitem_id}) is gone, so whether it ran is unknown. It won't be run again.`);
|
||||
continue;
|
||||
}
|
||||
if (!receipt.queued_sent_at) {
|
||||
await hub.note(receipt.hub_qitem_id, `Queued for the connected local seat as ${local.qitemId}. Not picked up yet.`);
|
||||
mark("queued_sent_at", receipt);
|
||||
}
|
||||
if (!receipt.pickup_sent_at && local.claimedAt) {
|
||||
await hub.note(receipt.hub_qitem_id, `Picked up by the local seat at ${local.claimedAt}.`);
|
||||
mark("pickup_sent_at", receipt);
|
||||
result.pickupsSent++;
|
||||
}
|
||||
const end = local.state === "handed-off" ? finalItem(deps.queueRepo, local) : local;
|
||||
if (!end) {
|
||||
await unknown(`This item was handed on from ${local.qitemId} without one successor here, so its outcome is unknown. It won't be run again.`);
|
||||
continue;
|
||||
}
|
||||
if (!FINAL.has(end.state)) continue;
|
||||
// The hub item closes the way the queue closes one: done with the local closure reason, or failed. A refusal, a
|
||||
// cancel or a failure never reads as success.
|
||||
const reason = end.state === "canceled" || end.closureReason === "canceled" ? "canceled"
|
||||
: end.state === "denied" || end.closureReason === "denied" ? "denied" : "no-follow-on";
|
||||
const state = end.state === "failed" ? "failed" as const : "done" as const;
|
||||
const closureReason = state === "failed" ? null : reason;
|
||||
const words = closingWords(deps.queueRepo, end);
|
||||
// A final check before the close: a hub item already closed, by our own close whose answer was lost or by someone
|
||||
// else, gets no write, so another actor's failure or cancel never becomes our done.
|
||||
let shown = await hub.show(receipt.hub_qitem_id);
|
||||
if (!HUB_CLOSED.has(shown.state)) {
|
||||
try {
|
||||
await hub.close(receipt.hub_qitem_id, { state, closureReason, resolution: words });
|
||||
} catch (error) {
|
||||
// Refused: closed by someone else between the check and the close. The readback says how.
|
||||
if (!(error instanceof HubError) || error.kind !== "refused") throw error;
|
||||
}
|
||||
shown = await hub.show(receipt.hub_qitem_id);
|
||||
if (!HUB_CLOSED.has(shown.state)) {
|
||||
deps.db.prepare("UPDATE network_receipts SET last_error = ? WHERE local_qitem_id = ?").run("The close didn't take: the hub item still reads open. Trying again.", receipt.local_qitem_id);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
// It counts as our result only when the hub item shows our state, closure reason and resolution. Otherwise the
|
||||
// local result stays here, and status shows the hub item as closed elsewhere.
|
||||
if (shown.state !== state || shown.resolution !== words || (shown.closureReason ?? null) !== closureReason) {
|
||||
mark("settled_at", receipt, "closed_on_hub", `The hub item was closed elsewhere (${shown.state}${shown.closureReason ? `, ${shown.closureReason}` : ""}) before this result reached it. The result stays on ${local.qitemId}.`);
|
||||
deps.log?.(`network ${binding.id}: hub item ${receipt.hub_qitem_id} was closed elsewhere; its local result stays on ${local.qitemId}`);
|
||||
continue;
|
||||
}
|
||||
mark("settled_at", receipt, end.state);
|
||||
result.resultsSent++;
|
||||
} catch (error) {
|
||||
if (!(error instanceof HubError) || error.kind !== "refused") throw error;
|
||||
deps.db.prepare("UPDATE network_receipts SET last_error = ? WHERE local_qitem_id = ?").run(error.message.slice(0, 500), receipt.local_qitem_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export interface NetworkConnectorSnapshot {
|
||||
running: boolean;
|
||||
intervalMs: number;
|
||||
bindings: Array<{ id: string; nextPassAt: string | null; backoffMs: number; lastPass: NetworkPassResult | null }>;
|
||||
}
|
||||
|
||||
/** The connector the daemon runs: one pass per active binding when it's due, one round at a time, off the event loop's
|
||||
* hot path (the hub calls are async network I/O). Each pass is bounded by its deadline, and stop closes a pass in
|
||||
* flight. A binding that fails backs off, doubling to a cap, with jitter. With no active binding it makes no network
|
||||
* call at all. */
|
||||
export class NetworkConnectorService {
|
||||
private timer: ReturnType<typeof setTimeout> | null = null;
|
||||
private inFlight: Promise<void> | null = null;
|
||||
private stopped = false;
|
||||
private stopping: AbortController | null = null;
|
||||
private readonly due = new Map<string, { at: number; backoffMs: number; last: NetworkPassResult | null }>();
|
||||
readonly intervalMs: number;
|
||||
private readonly maxBackoffMs: number;
|
||||
|
||||
constructor(private readonly deps: NetworkConnectorDeps, opts: { intervalMs?: number; maxBackoffMs?: number; random?: () => number } = {}) {
|
||||
this.intervalMs = opts.intervalMs ?? 30_000;
|
||||
this.maxBackoffMs = Math.max(this.intervalMs, opts.maxBackoffMs ?? 300_000);
|
||||
this.random = opts.random ?? Math.random;
|
||||
}
|
||||
private readonly random: () => number;
|
||||
|
||||
start(): void {
|
||||
this.stopped = false;
|
||||
this.stopping = new AbortController();
|
||||
// A credential whose binding isn't active has nothing left to do here: one a stop left before its row, or one
|
||||
// whose removal failed when its binding ended.
|
||||
const swept = this.deps.bindings.sweepCredentials();
|
||||
if (swept.removed.length) this.deps.log?.(`network connector: removed the credentials of ended bindings ${swept.removed.join(", ")}`);
|
||||
if (swept.failed.length) this.deps.log?.(`network connector: couldn't remove the credentials of ended bindings ${swept.failed.join(", ")}`);
|
||||
this.schedule(0);
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
this.stopped = true;
|
||||
this.stopping?.abort();
|
||||
if (this.timer) clearTimeout(this.timer);
|
||||
this.timer = null;
|
||||
await this.inFlight;
|
||||
}
|
||||
|
||||
/** One round: a pass for each active binding that's due. Exposed for tests and status. */
|
||||
async round(): Promise<void> {
|
||||
if (this.inFlight) return this.inFlight;
|
||||
this.inFlight = (async () => {
|
||||
const at = (this.deps.now ?? (() => new Date()))().getTime();
|
||||
const active = this.deps.bindings.active();
|
||||
for (const id of [...this.due.keys()]) if (!active.some((b) => b.id === id)) this.due.delete(id);
|
||||
for (const binding of active) {
|
||||
if (this.stopped) break;
|
||||
const due = this.due.get(binding.id);
|
||||
if (due && due.at > at) continue;
|
||||
const pass = await runNetworkBindingPass(this.deps, binding, { signal: this.stopping?.signal });
|
||||
const failing = pass.error === "unavailable" || pass.error === "refused" || pass.error === "credential" || pass.error === "local";
|
||||
const backoffMs = failing ? Math.min(this.maxBackoffMs, (due?.backoffMs ?? this.intervalMs) * 2) : this.intervalMs;
|
||||
const jitter = 1 + (this.random() * 0.4 - 0.2);
|
||||
this.due.set(binding.id, { at: at + Math.round(backoffMs * jitter), backoffMs, last: pass });
|
||||
}
|
||||
})();
|
||||
try { await this.inFlight; } finally { this.inFlight = null; }
|
||||
}
|
||||
|
||||
snapshot(): NetworkConnectorSnapshot {
|
||||
return {
|
||||
running: !this.stopped,
|
||||
intervalMs: this.intervalMs,
|
||||
bindings: [...this.due.entries()].map(([id, d]) => ({ id, nextPassAt: new Date(d.at).toISOString(), backoffMs: d.backoffMs, lastPass: d.last })),
|
||||
};
|
||||
}
|
||||
|
||||
private schedule(ms: number): void {
|
||||
if (this.stopped) return;
|
||||
this.timer = setTimeout(() => {
|
||||
void this.round().catch((error) => this.deps.log?.(`network connector round failed: ${(error as Error).message}`))
|
||||
.finally(() => this.schedule(Math.min(this.intervalMs, 5_000)));
|
||||
}, ms);
|
||||
this.timer.unref?.();
|
||||
}
|
||||
}
|
||||
|
||||
export { HubError };
|
||||
@@ -1508,7 +1508,7 @@ export function hookOrPaneActivity(
|
||||
// beats hook-arrival age) and makes ACTIVITY real on the DEFAULT path for hook-less / Codex /
|
||||
// just-finished-a-turn seats: the exact fleet-blindness this fix targets. Structural discrimination
|
||||
// is STRUCTURAL (spinner shapes / esc-to-interrupt / idle prompt), never a verb allowlist.
|
||||
if (structural && structural.state !== "unknown") {
|
||||
if (structural && (structural.state !== "unknown" || structural.reason === "unrecognized_client_version")) {
|
||||
return {
|
||||
source: "pane",
|
||||
activity: {
|
||||
@@ -1617,8 +1617,9 @@ export async function attachAgentActivity(
|
||||
*
|
||||
* needs_input > running > MOTION > idle / unknown
|
||||
*
|
||||
* A current, target-bound complete idle frame is the narrow exception to motion below.
|
||||
* A generic footer or old idle capture is not that evidence; D2 remains unchanged.
|
||||
* A current, target-bound complete idle frame is the narrow idle exception to motion below.
|
||||
* An explicitly unsupported client version also retains unknown rather than inferring work from redraws.
|
||||
* A generic footer or old idle capture is not completed-idle evidence; D2 remains unchanged.
|
||||
* Live motion otherwise UPGRADES a verdict of `idle` or `unknown`; it never touches `needs_input` or an
|
||||
* already-`running` verdict, and it never MANUFACTURES `idle`. That asymmetry is the whole
|
||||
* safety argument, and it is what each half of the defect needs:
|
||||
@@ -1644,7 +1645,7 @@ export async function attachAgentActivity(
|
||||
* signal improving. Silence therefore leaves the prior verdict exactly as it was.
|
||||
*/
|
||||
const withMotion = (activity: AgentActivity, currentIdleFrame = false): AgentActivity => {
|
||||
if (!motionRunning || (currentIdleFrame && activity.state === "idle")) return activity;
|
||||
if (!motionRunning || activity.reason === "unrecognized_client_version" || (currentIdleFrame && activity.state === "idle")) return activity;
|
||||
if (activity.state !== "idle" && activity.state !== "unknown") return activity;
|
||||
return {
|
||||
state: "running",
|
||||
@@ -1682,7 +1683,8 @@ export async function attachAgentActivity(
|
||||
|
||||
// An idle hook alone still yields to motion (D2); when a current complete idle frame exists, report
|
||||
// that frame's provenance instead.
|
||||
const resolved = hookOrPaneActivity(hookActivity, structural, motionRunning && currentIdleFrame, entry.runtime);
|
||||
const resolved = hookOrPaneActivity(hookActivity, structural,
|
||||
motionRunning && (currentIdleFrame || structural?.reason === "unrecognized_client_version"), entry.runtime);
|
||||
if (resolved?.source === "hook") return { ...entry, agentActivity: withMotion(resolved.activity) };
|
||||
if (resolved?.source === "limit") return { ...entry, agentActivity: resolved.activity };
|
||||
if (resolved?.source === "pane") return { ...entry, agentActivity: withMotion(resolved.activity, currentIdleFrame) };
|
||||
|
||||
@@ -389,6 +389,22 @@ function declaredNodeId(dir: string): string | null {
|
||||
* from them); it defaults to the ordinary filesystem reader and changes nothing about the evaluation. */
|
||||
export function readProjectReadiness(missionsRoot: string, readPolicy: ProofPolicyRead = defaultPolicyRead, root?: string, io: ScopeFsDeps = proofFs, observe?: (file: string) => void) {
|
||||
const missions = fs.readdirSync(missionsRoot).filter(n => fs.statSync(path.join(missionsRoot, n)).isDirectory()).map(name => ({ name, ...readMissionReadiness(contained(missionsRoot, path.join(missionsRoot, name)), readPolicy, root, io, observe) }));
|
||||
return projectReadiness(missions);
|
||||
}
|
||||
|
||||
/** The same semantic read, yielding between missions for daemon requests and source-watch refreshes.
|
||||
* A single mission (including its evidence) is still synchronous; this is not a hard time bound or atomic snapshot. */
|
||||
export async function readProjectReadinessAsync(missionsRoot: string, readPolicy: ProofPolicyRead = defaultPolicyRead, root?: string, io: ScopeFsDeps = proofFs, observe?: (file: string) => void) {
|
||||
const names = fs.readdirSync(missionsRoot).filter(n => fs.statSync(path.join(missionsRoot, n)).isDirectory());
|
||||
const missions: Array<MissionReadiness & { name: string }> = [];
|
||||
for (const name of names) {
|
||||
await new Promise<void>(resolve => setImmediate(resolve));
|
||||
missions.push({ name, ...readMissionReadiness(contained(missionsRoot, path.join(missionsRoot, name)), readPolicy, root, io, observe) });
|
||||
}
|
||||
return projectReadiness(missions);
|
||||
}
|
||||
|
||||
function projectReadiness(missions: Array<MissionReadiness & { name: string }>) {
|
||||
const active = missions.filter(m => ["active", "release-candidate"].includes(m.historicalStatus ?? ""));
|
||||
return { revision: hash(active), state: active.length && active.every(m => m.state === "ready") ? "ready" : active.some(m => m.state === "unknown") ? "unknown" : "not-ready", missions, basis: "Active mission readiness; distinct outcome judgment and publication remain separate" };
|
||||
}
|
||||
@@ -3,7 +3,7 @@ import * as path from "node:path";
|
||||
import { createHash } from "node:crypto";
|
||||
import type { EventBus } from "../event-bus.js";
|
||||
import type { ScopeFsDeps } from "../scope/scope-view-projection.js";
|
||||
import { createProofPolicyRead, proofFs, readProjectReadiness, resolveProjectRoot } from "./judgments.js";
|
||||
import { createProofPolicyRead, proofFs, readProjectReadinessAsync, resolveProjectRoot } from "./judgments.js";
|
||||
|
||||
export interface ProofSourceWatch {
|
||||
close(): void;
|
||||
@@ -76,16 +76,15 @@ function shallowWatches(missionsRoot: string, realRoot: string, top: string, pro
|
||||
* including absent nearer ones, and accepted evidence elsewhere in the project) is watched at its parent directory,
|
||||
* not recursively, so sibling checkouts and build trees are never subscribed to.
|
||||
*/
|
||||
export function watchProofSources(missionsRoot: string, invalidate: () => void, bus: EventBus, scopeChanged?: () => void): ProofSourceWatch {
|
||||
export async function watchProofSources(missionsRoot: string, invalidate: () => void, bus: EventBus, scopeChanged?: () => void): Promise<ProofSourceWatch> {
|
||||
missionsRoot = path.resolve(missionsRoot);
|
||||
let projectRoot = path.dirname(missionsRoot);
|
||||
try { projectRoot = resolveProjectRoot(missionsRoot) ?? projectRoot; } catch { /* Legacy roots keep the missions root's parent. */ }
|
||||
const top = path.dirname(path.resolve(projectRoot));
|
||||
// The authority boundary in real terms: what the readers' containment check accepts.
|
||||
const project = canonical(projectRoot) ?? path.resolve(projectRoot);
|
||||
// ponytail: bounded local workspaces use a full semantic read after a file burst.
|
||||
// Keep this workload measured; subtree indexing is warranted only when that bound is exceeded.
|
||||
const basis = () => {
|
||||
// Keep the full semantic read, but let queue/send requests run between missions.
|
||||
const basis = async () => {
|
||||
const read = new Set<string>();
|
||||
const note = (p: string) => { read.add(p); return p; };
|
||||
const io: ScopeFsDeps = {
|
||||
@@ -95,7 +94,8 @@ export function watchProofSources(missionsRoot: string, invalidate: () => void,
|
||||
listDir: (p) => proofFs.listDir(note(p)),
|
||||
isDirectory: (p) => proofFs.isDirectory(note(p)),
|
||||
};
|
||||
const hash = createHash("sha256").update(JSON.stringify(readProjectReadiness(missionsRoot, createProofPolicyRead(), undefined, io, note).missions.map(m => [m.name, m.revision]))).digest("hex");
|
||||
const result = await readProjectReadinessAsync(missionsRoot, createProofPolicyRead(), undefined, io, note);
|
||||
const hash = createHash("sha256").update(JSON.stringify(result.missions.map(m => [m.name, m.revision]))).digest("hex");
|
||||
// A file read through a link inside the project depends on its real location too: observe both.
|
||||
for (const file of [...read]) {
|
||||
const real = canonical(file);
|
||||
@@ -107,6 +107,7 @@ export function watchProofSources(missionsRoot: string, invalidate: () => void,
|
||||
let closed = false, failed = false, covered = false, readable = false;
|
||||
let revision = "unavailable", timer: NodeJS.Timeout | undefined;
|
||||
let scopeDirty = false;
|
||||
let reading = true, dirty = false;
|
||||
const notifyScope = () => { try { scopeChanged?.(); } catch { /* A later event or explicit refresh retries the view read. */ } };
|
||||
let wanted: Shallow = new Map();
|
||||
const shallow = new Map<string, fs.FSWatcher>();
|
||||
@@ -163,25 +164,32 @@ export function watchProofSources(missionsRoot: string, invalidate: () => void,
|
||||
/** Read the sources and keep the subscriptions in step with what was read, even when the result is unchanged.
|
||||
* Coverage is certified only by a read whose every dependency was already watched before it; if the dependencies
|
||||
* keep moving past a small budget, the sources are reported unobserved rather than read forever. */
|
||||
const evaluate = (): string => {
|
||||
let read = basis();
|
||||
const evaluate = async (): Promise<{ hash: string; covered: boolean }> => {
|
||||
let read = await basis();
|
||||
for (let i = 0; ; i++) {
|
||||
if (closed) return { hash: revision, covered: false };
|
||||
// A root whose real location is unknown or outside the project has no anchors there: never certified.
|
||||
if (!resubscribe(read.watches)) { covered = recursiveReal !== null && within(recursiveReal, project); return read.hash; }
|
||||
if (i === 2) { covered = false; return read.hash; }
|
||||
read = basis();
|
||||
// Coverage belongs to this candidate, just like the hash; neither escapes a discarded read.
|
||||
if (!resubscribe(read.watches)) return { hash: read.hash, covered: recursiveReal !== null && within(recursiveReal, project) };
|
||||
if (i === 2) return { hash: read.hash, covered: false };
|
||||
read = await basis();
|
||||
}
|
||||
};
|
||||
const observed = () => !closed && !failed && covered && readable;
|
||||
const schedule = () => {
|
||||
if (timer || closed) return;
|
||||
timer = setTimeout(() => {
|
||||
dirty = true;
|
||||
if (timer || reading || closed) return;
|
||||
timer = setTimeout(async () => {
|
||||
timer = undefined;
|
||||
if (closed) return;
|
||||
reading = true; dirty = false;
|
||||
try {
|
||||
if (scopeDirty) { scopeDirty = false; notifyScope(); }
|
||||
const before = observed();
|
||||
const next = evaluate();
|
||||
const candidate = await evaluate();
|
||||
if (closed || dirty) return;
|
||||
const next = candidate.hash;
|
||||
covered = candidate.covered && !failed;
|
||||
readable = true;
|
||||
// Deduplicate on what clients see: the revision and whether the sources are observed. Losing or regaining
|
||||
// coverage is news even when the proof result itself did not change.
|
||||
@@ -190,12 +198,16 @@ export function watchProofSources(missionsRoot: string, invalidate: () => void,
|
||||
invalidate();
|
||||
notify(observed() ? revision : "unavailable");
|
||||
} catch (e) {
|
||||
if (closed) return;
|
||||
// Never report the sources as watched after a failed read or a watch that could not be attached.
|
||||
readable = false;
|
||||
if (e instanceof WatchFailure) covered = false;
|
||||
revision = "unavailable";
|
||||
invalidate();
|
||||
notify("unavailable");
|
||||
} finally {
|
||||
reading = false;
|
||||
if (dirty && !closed) schedule();
|
||||
}
|
||||
}, 40);
|
||||
timer.unref();
|
||||
@@ -203,7 +215,10 @@ export function watchProofSources(missionsRoot: string, invalidate: () => void,
|
||||
try {
|
||||
recursiveReal = canonical(missionsRoot);
|
||||
recursive = watchDir(missionsRoot, true);
|
||||
try { revision = evaluate(); readable = true; } catch (e) {
|
||||
try {
|
||||
const candidate = await evaluate();
|
||||
if (!dirty) { revision = candidate.hash; covered = candidate.covered && !failed; readable = true; }
|
||||
} catch (e) {
|
||||
if (e instanceof WatchFailure) throw e;
|
||||
// Direct reads name unavailable inputs; watching may recover them.
|
||||
covered = true;
|
||||
@@ -214,6 +229,9 @@ export function watchProofSources(missionsRoot: string, invalidate: () => void,
|
||||
for (const watcher of shallow.values()) watcher.close();
|
||||
invalidate(); notify("unavailable"); notifyScope();
|
||||
return { close() {}, observation: () => ({ state: "unavailable", revision }) };
|
||||
} finally {
|
||||
reading = false;
|
||||
if (dirty && !closed) schedule();
|
||||
}
|
||||
return {
|
||||
observation: () => ({ state: observed() ? "watching" : "unavailable", revision }),
|
||||
|
||||
@@ -219,6 +219,8 @@ export type ReactiveEventInput = {
|
||||
// Required: a reactive event without a freshness bound could never pass BR-2, and a momentary
|
||||
// event must always carry when it expires. Staleness itself stays the shared predicate's job.
|
||||
staleAfter: string;
|
||||
/** #99: when the provider's own failure says the limit resets. Absent, the pool falls back to the bounded retry. */
|
||||
resetsAt?: string;
|
||||
} & ({ provider: ProviderKind; accountRef: string; seatSession?: never }
|
||||
| { provider: "claude"; seatSession: string; accountRef?: never });
|
||||
|
||||
@@ -236,6 +238,7 @@ export function reactiveEventSignal(input: ReactiveEventInput): ProviderSignal {
|
||||
// At-limit exhaustion is an actionable switch trigger; stream/stop errors are advisory.
|
||||
automationUse: input.kind === "at_limit" ? "allow_switch_decision" : "advisory_only",
|
||||
// No usedPercent — a reactive event is exhaustion evidence, not a remaining meter.
|
||||
...(input.resetsAt !== undefined ? { resetsAt: input.resetsAt } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -319,8 +322,8 @@ export function deriveUsageLimitPools(input: {
|
||||
const statedResets = pool.signals
|
||||
.map((signal) => Date.parse(signal.resetsAt ?? ""))
|
||||
.filter((value) => Number.isFinite(value) && value > nowMs);
|
||||
// A typed Claude failure has no reset timestamp. Use the existing bounded retry interval,
|
||||
// as for Codex events; a Claude meter missing its reset still cannot invent one.
|
||||
// A typed Claude failure carries a reset only when Claude's own message stated one (#99). Without it, use
|
||||
// the existing bounded retry interval, as for Codex events; a Claude meter missing its reset still cannot invent one.
|
||||
const fallbackResets = pool.signals
|
||||
.filter((signal) => pool.provider === "codex" || (signal.sourceClass === "provider_event" && signal.authority === "reactive_error"))
|
||||
.map((signal) => Date.parse(signal.asOf) + input.fallbackSeconds * 1000)
|
||||
|
||||
@@ -4,6 +4,9 @@ import type { CodexAuthMetadata } from "./codex-auth-reader.js";
|
||||
import { reactiveEventSignal, type ReactiveEventKind } from "./provider-signals.js";
|
||||
import type { ProviderSignal } from "./provider-types.js";
|
||||
|
||||
/** #99: the farthest a stated Claude reset may sit after its failure: the relay's own cap (activity-relay.cjs). */
|
||||
const CLAUDE_STATED_RESET_MAX_MS = 8 * 24 * 3600 * 1000;
|
||||
|
||||
interface ReactiveSeat {
|
||||
seatSession: string;
|
||||
runtime: string;
|
||||
@@ -98,9 +101,13 @@ export function collectReactiveEventSignals(deps: ReactiveTapDeps): ReactiveTapR
|
||||
|| isVerificationReason(event.reason) || event.stale) continue;
|
||||
const eventAtMs = Date.parse(event.eventAt ?? "");
|
||||
if (!Number.isFinite(eventAtMs) || eventAtMs > nowMs || nowMs >= eventAtMs + deps.freshnessMs) continue;
|
||||
// #99: the reset Claude's message stated, when the relay could read one, after the failure and within the relay's
|
||||
// eight-day cap. The cap is enforced here too, so no other sender can park a pool longer than the relay could.
|
||||
const resetMs = Date.parse(event.resetsAt ?? "");
|
||||
signals.push(reactiveEventSignal({
|
||||
provider: "claude", seatSession: seat.seatSession, kind: "at_limit",
|
||||
asOf: event.eventAt!, staleAfter: new Date(eventAtMs + deps.freshnessMs).toISOString(),
|
||||
...(Number.isFinite(resetMs) && resetMs > eventAtMs && resetMs <= eventAtMs + CLAUDE_STATED_RESET_MAX_MS ? { resetsAt: new Date(resetMs).toISOString() } : {}),
|
||||
}));
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -4272,7 +4272,9 @@ export class QueueRepository {
|
||||
// The person learns who decides and in what recorded state, so a delay is shown: one update per request.
|
||||
const toldId = `qitem-runner-permission-passed-${item.qitemId.replace(/^qitem-runner-permission-/, "").replace(/^qitem-/, "")}`;
|
||||
setImmediate(() => {
|
||||
// The request the person answered is what they'd judge (convention C3: a note to a person names its evidence).
|
||||
void this.create({ qitemId: toldId, sourceSession: root.sourceSession, destinationSession: person, humanIntent: "update",
|
||||
evidenceRef: `rig queue show ${root.qitemId}`,
|
||||
summary: `Your answer went to ${decider} to decide; ${root.sourceSession} is waiting`,
|
||||
body: [`Your answer to ${root.sourceSession}'s permission request ${root.qitemId} went to ${decider}, which decides it while ${root.sourceSession} waits. Nothing runs until it decides.`,
|
||||
...(chosen?.describe ? [`Receiving agent: ${chosen.describe}`] : []),
|
||||
@@ -4323,6 +4325,7 @@ export class QueueRepository {
|
||||
// After the answer's own write settles: one receipt per request (its id is the request's), told to who answered.
|
||||
setImmediate(() => {
|
||||
void this.create({ qitemId: receiptId, sourceSession: requester, destinationSession: person, humanIntent: "update",
|
||||
evidenceRef: `rig queue show ${item.qitemId}`,
|
||||
summary: `Your answer to ${requester}'s permission request is kept; the tool hasn't run`,
|
||||
body: [`Your answer to ${requester}'s permission request ${item.qitemId} is kept, and the request is still waiting: ${unreachable ? `it couldn't be passed to ${unreachable}, the receiving agent, just now (answer again to retry)` : `no receiving agent on this host can read it before the tool runs (none is set, or the only one is ${requester} itself)`}. The tool hasn't run, and nothing was permitted or refused.`,
|
||||
`To decide now: rig seat decide-permission ${item.qitemId} --option <id>${options.length ? ` (options: ${options.map((o) => `${o.id}, ${o.label}`).join("; ")})` : ""}. Or set up a receiving agent that isn't ${requester}: rig config set agents.operator_session <seat>@<rig>.`,
|
||||
|
||||
@@ -331,8 +331,15 @@ export class RunnerPermissions {
|
||||
const changed = new Promise<void>((resolve) => { release = resolve; });
|
||||
let liveId = askId; // after a route or handoff, the colleague's row is where the answer lands
|
||||
const unsubscribe = this.deps.eventBus.subscribe((event) => {
|
||||
const changedId = (event as { type?: string; qitemId?: string }).type === "queue.updated" ? (event as { qitemId?: string }).qitemId : undefined;
|
||||
if (changedId === askId || changedId === liveId) release();
|
||||
const { type, qitemId } = event as { type?: string; qitemId?: string };
|
||||
// A handoff while this read waits (the person's answer going to the receiving agent) moves the request to a new
|
||||
// row: follow it, so the decision recorded there wakes this read rather than its next one.
|
||||
if (type === "queue.handed_off" && (qitemId === askId || qitemId === liveId)) {
|
||||
const asked = this.deps.queueRepo.getById(askId);
|
||||
liveId = (asked ? this.liveEnd(asked)?.qitemId : undefined) ?? liveId;
|
||||
return;
|
||||
}
|
||||
if (type === "queue.updated" && (qitemId === askId || qitemId === liveId)) release();
|
||||
});
|
||||
try {
|
||||
const now = this.withWords(askId, this.read(seat, askId));
|
||||
|
||||
@@ -12,6 +12,7 @@ import { SessionAddressConflict, type SessionRegistry } from "./session-registry
|
||||
import type { DiscoveryRepository } from "./discovery-repository.js";
|
||||
import type { EventBus } from "./event-bus.js";
|
||||
import type { TmuxAdapter } from "../adapters/tmux.js";
|
||||
import { shellQuote } from "../adapters/shell-quote.js";
|
||||
import type { TmuxOptionDefaultsApplier } from "./tmux-option-defaults.js";
|
||||
import { SeatStatusService, type SeatStatus, type SeatStatusResult } from "./seat-status-service.js";
|
||||
import { SeatHandoverPlanner, parseHandoverSource, SEAT_HANDOVER_SOURCE_CAPABILITIES, type SeatHandoverPlan, type SeatHandoverSource } from "./seat-handover-planner.js";
|
||||
@@ -118,7 +119,9 @@ export type SeatHandoverResult =
|
||||
| { ok: false; code: "missing_reason" | "invalid_source" | "successor_creation_not_implemented" | "source_not_supported" | "resume_token_unavailable" | "fork_source_not_found"; message: string; guidance: string }
|
||||
| { ok: false; code: "address_conflict" | "current_occupant_required" | "discovered_not_active" | "successor_tmux_absent" | "successor_already_managed" | "successor_is_current" | "runtime_mismatch"; message: string; guidance: string }
|
||||
| { ok: false; code: "discovered_not_found"; message: string; guidance: string }
|
||||
| { ok: false; code: "tmux_probe_failed" | "handover_commit_failed" | "successor_create_failed" | "context_delivery_failed"; message: string; guidance: string }
|
||||
| { ok: false; code: "tmux_probe_failed" | "handover_commit_failed" | "context_delivery_failed"; message: string; guidance: string }
|
||||
// #2985: once the successor was launched in the respawned pane, the failure says so and names the pane.
|
||||
| { ok: false; code: "successor_create_failed"; message: string; guidance: string; replacementStarted?: true; successorPane?: string }
|
||||
| Extract<SeatStatusResult, { ok: false }>;
|
||||
|
||||
type ClaudeRestorePacketObservation =
|
||||
@@ -588,6 +591,24 @@ export class SeatHandoverService {
|
||||
},
|
||||
});
|
||||
if (!launch.ok) {
|
||||
// #2985: the harness was launched into the respawned pane and only readiness failed. The previous occupant is
|
||||
// already ended and the successor may come up (or already be working) there, but the handover never commits,
|
||||
// so its generation is never registered. Retrying the handover would end it, so the guidance says what may be
|
||||
// in the pane instead of "retry". It claims only what holds for every runtime (a Codex successor's hooks are
|
||||
// not held back by generation_uncommitted, which is Claude-only).
|
||||
if (launch.step === "start_agent" && launch.replacementStarted && launch.successorLaunched) {
|
||||
const pane = launch.tmuxPane ?? "unknown";
|
||||
const session = latestSession.session_name;
|
||||
const quoted = shellQuote(session);
|
||||
return {
|
||||
ok: false,
|
||||
code: "successor_create_failed",
|
||||
message: `Handover failed at step "${launch.step}": ${launch.message}`,
|
||||
replacementStarted: true,
|
||||
...(launch.tmuxPane ? { successorPane: launch.tmuxPane } : {}),
|
||||
guidance: `The handover was not committed: the seat's registry binding is unchanged and OpenRig has not registered the successor as the seat's occupant. The previous occupant was already ended, and a successor was launched in pane "${pane}" of ${session}; it may still be starting there or already working. Look at the pane before acting: rig capture ${quoted}. Another handover, or rig seat launch ${quoted} --fresh --stop --reason "<why>", ends that successor if it goes ahead. Shell-looking output alone does not confirm that the successor has stopped; check the launch and process state before another handover.`,
|
||||
};
|
||||
}
|
||||
return {
|
||||
ok: false,
|
||||
code: "successor_create_failed",
|
||||
@@ -833,12 +854,12 @@ export class SeatHandoverService {
|
||||
}
|
||||
// #981: a composer-launched successor respawns into the seat's own pane, whose discovery row the
|
||||
// previous handover claimed for this same node, and a rescan keeps that claim (#237). That claim is
|
||||
// this seat's own, so commit re-claims it. A row claimed by any other node, or a discovered-source
|
||||
// successor, still needs an active, unclaimed record.
|
||||
// this seat's own, so commit re-claims it. #2861: a deleted owner or a terminal owner in an archived
|
||||
// earlier rig can also leave that pane claimed. Other owners and discovered-source successors still refuse.
|
||||
const ownPaneClaim = input.reportedSource.mode !== "discovered"
|
||||
&& discovered.status === "claimed"
|
||||
&& discovered.claimedNodeId === input.node.id
|
||||
&& discovered.tmuxSession === input.latestSession.session_name;
|
||||
&& discovered.tmuxSession === input.latestSession.session_name
|
||||
&& (discovered.claimedNodeId === input.node.id || this.isStalePaneClaimOwner(discovered.claimedNodeId));
|
||||
if (discovered.status !== "active" && !ownPaneClaim) {
|
||||
return fail({
|
||||
ok: false,
|
||||
@@ -1256,6 +1277,15 @@ export class SeatHandoverService {
|
||||
return findOtherSessionOwner(this.db, tmuxSession, targetNodeId, { ignoreArchived });
|
||||
}
|
||||
|
||||
private isStalePaneClaimOwner(nodeId: string | null): boolean {
|
||||
if (nodeId === null) return true; // ON DELETE SET NULL retains the claim after its owner is removed.
|
||||
return this.db.prepare(`
|
||||
SELECT 1 FROM nodes n JOIN rigs r ON r.id = n.rig_id
|
||||
WHERE n.id = ? AND r.archived_at IS NOT NULL
|
||||
AND (SELECT status FROM sessions WHERE node_id = n.id ORDER BY id DESC LIMIT 1) IN ('superseded', 'detached', 'exited')
|
||||
`).pluck().get(nodeId) === 1;
|
||||
}
|
||||
|
||||
private commit(input: {
|
||||
seatRef: string;
|
||||
status: SeatStatus;
|
||||
@@ -1277,6 +1307,15 @@ export class SeatHandoverService {
|
||||
}): SeatHandoverResult {
|
||||
const handoverAt = this.now().toISOString();
|
||||
const tx = this.db.transaction(() => {
|
||||
// The presence probe awaited after validation. Reclaim only the same still-stale foreign claim,
|
||||
// inside the transaction that assigns it, so a resumed owner or a new claimant cannot be overwritten.
|
||||
if (input.discovered.status === "claimed" && input.discovered.claimedNodeId !== input.node.id) {
|
||||
const current = this.discoveryRepo.getDiscoveredSession(input.discovered.id);
|
||||
if (current?.status !== "claimed" || current.claimedNodeId !== input.discovered.claimedNodeId
|
||||
|| !this.isStalePaneClaimOwner(current.claimedNodeId)) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
const rows = this.db.prepare(
|
||||
"SELECT id FROM sessions WHERE node_id = ? AND status NOT IN ('superseded', 'detached', 'exited') ORDER BY id"
|
||||
).all(input.node.id) as Array<{ id: string }>;
|
||||
@@ -1392,7 +1431,16 @@ export class SeatHandoverService {
|
||||
|
||||
let committed: { newSessionId: string; previousSessionIdsSuperseded: string[]; event: PersistedEvent; provenanceRecordWritten: boolean };
|
||||
try {
|
||||
committed = tx();
|
||||
const outcome = tx();
|
||||
if (outcome === null) {
|
||||
return {
|
||||
ok: false,
|
||||
code: "discovered_not_active",
|
||||
message: `Discovery record "${input.discovered.id}" no longer has a stale owner claim.`,
|
||||
guidance: "Inspect the current discovery owner and successor state before retrying.",
|
||||
};
|
||||
}
|
||||
committed = outcome;
|
||||
// S19 ruling 01530 — the SOLE narrow call: after the commit lands, the activity
|
||||
// oracle sees the swap as its own event, keyed by the durable node id, identified
|
||||
// by the successor tenure (never the retiree's). In-memory, post-commit, optional.
|
||||
|
||||
@@ -73,7 +73,8 @@ const SEAT_LOOKUP_GUIDANCE = "List seats with: rig ps --nodes";
|
||||
|
||||
/**
|
||||
* The supported route for a session registered as adopted (origin=claimed): `rig unclaim` releases
|
||||
* it and leaves its process running, so ending that process in its own terminal ends only this seat.
|
||||
* it and leaves its process running, so ending that process and its tmux session ends only this seat. Closing only
|
||||
* a viewing window detaches from the tmux session and leaves it running.
|
||||
* `rig down` stops only a session carrying the seat's OpenRig marker, which OpenRig sets on the sessions
|
||||
* it starts and an adopted, bound or self-attached session lacks, so it isn't a route here and the
|
||||
* guidance says why. Only the guidance names the route; the refusal stands. A seat handed over before
|
||||
@@ -85,9 +86,13 @@ export function claimedSessionRoutes(rigName: string, sessionName: string, exter
|
||||
const release = `To release it from OpenRig and leave its process running, run rig unclaim ${session}.`;
|
||||
if (externalCli) return `This seat was joined with rig attach as an external CLI, so rig down leaves it running. ${release} End the process in its own terminal.`;
|
||||
return `To end this session, run rig unclaim ${session} (it releases the session from OpenRig and leaves its process running), `
|
||||
+ `then end that process in its own terminal. ${rigDownLeavesIt(rigName)}`;
|
||||
+ `then end that process and the underlying tmux session, including any remaining shell, so no session keeps the seat's name. `
|
||||
+ `${VIEWER_LEAVES_IT} ${rigDownLeavesIt(rigName)}`;
|
||||
}
|
||||
|
||||
/** Closing a viewer detaches; the tmux session, its shell and the seat's name stay. */
|
||||
const VIEWER_LEAVES_IT = "Closing only its viewing window leaves the tmux session running.";
|
||||
|
||||
/** Why `rig down <rig>` isn't the route for an adopted session. */
|
||||
function rigDownLeavesIt(rigName: string): string {
|
||||
return `rig down ${shellQuote(rigName)} leaves a session OpenRig didn't start running.`;
|
||||
@@ -1381,6 +1386,8 @@ export class SeatLifecycleService {
|
||||
ok: false,
|
||||
code: "unmanaged_session_collision",
|
||||
message: `Canonical tmux session "${canonicalSessionName}" exists but is not owned by this seat's current managed rows; refusing to overwrite it.`,
|
||||
guidance: "If you released this session with rig unclaim, end that underlying tmux session, including any remaining shell, "
|
||||
+ "then launch again. Closing only its viewing window doesn't end the session.",
|
||||
};
|
||||
}
|
||||
if (!stop) {
|
||||
@@ -1424,8 +1431,10 @@ export class SeatLifecycleService {
|
||||
return {
|
||||
ok: false, code: "claimed_session",
|
||||
message: `Seat "${seatRef}" has an adopted/operator-owned occupant; fresh launch refuses even with --stop.`,
|
||||
guidance: `Release it with rig unclaim ${shellQuote(claimed.session_name)} (its process keeps running), end that process yourself, then launch fresh.`
|
||||
+ (externalCli ? "" : ` ${rigDownLeavesIt(seat.rigName)}`),
|
||||
guidance: `Release it with rig unclaim ${shellQuote(claimed.session_name)} (its process keeps running), `
|
||||
+ (externalCli
|
||||
? "end that process and close its terminal, then launch fresh."
|
||||
: `end that process and the underlying tmux session, including any remaining shell, then launch fresh. ${VIEWER_LEAVES_IT} ${rigDownLeavesIt(seat.rigName)}`),
|
||||
};
|
||||
}
|
||||
const binding = this.sessionRegistry.getBindingForNode(nodeId);
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type Database from "better-sqlite3";
|
||||
import type { PaneCapture, TmuxAdapter } from "../adapters/tmux.js";
|
||||
import { communityHarnessPaneActivity, classifyPaneActivity, codexWorkingElapsed, isCodexLiveWorkingFrame, readSeatBeforePaste, type PaneActivityClassification } from "./session-transport.js";
|
||||
import { communityHarnessPaneActivity, classifyPaneActivity, codexWorkingElapsed, isCodexLiveWorkingFrame, namesAnotherClientVersion, readSeatBeforePaste, type PaneActivityClassification } from "./session-transport.js";
|
||||
import { communityHarnessOf } from "./built-in-runtimes.js";
|
||||
import { detectCodexLimitBanner, type CodexLimitBanner } from "./provider/codex-limit-banner.js";
|
||||
import { IDLE_FULL_CAPTURE_MS, windowSnapshot } from "./transcript-rotation.js";
|
||||
@@ -129,6 +129,9 @@ export class SeatStructuralActivityService {
|
||||
return null;
|
||||
}
|
||||
const c = communityHarnessPaneActivity(classifyPaneActivity(content), communityHarness, content);
|
||||
// Redraws from an unrecorded Codex layout cannot establish work or completed idle.
|
||||
// Keep positive work and attention visible; native hooks are resolved separately by the activity fold.
|
||||
const unrecognizedVersion = runtime === "codex" && c.state !== "attention" && c.state !== "agent_active" && namesAnotherClientVersion(content, "codex");
|
||||
const at = observedAt ?? this.now();
|
||||
const latest = this.latestBySession.get(sessionName);
|
||||
// A counter or proof read from another pane or process says nothing about this one, so compare only within one.
|
||||
@@ -146,8 +149,8 @@ export class SeatStructuralActivityService {
|
||||
...(advanced ? { previousCaptureStartedAt: previous!.codexWorkingClock!.captureStartedAt } : {}) }
|
||||
: priorProof;
|
||||
const obs: StructuralObservation = {
|
||||
state: c.state,
|
||||
reason: c.reason,
|
||||
state: unrecognizedVersion ? "unknown" : c.state,
|
||||
reason: unrecognizedVersion ? "unrecognized_client_version" : c.reason,
|
||||
evidence: c.evidence,
|
||||
observedAt: at.toISOString(),
|
||||
...(captureStartedAt ? { captureStartedAt: captureStartedAt.toISOString() } : {}),
|
||||
@@ -167,7 +170,7 @@ export class SeatStructuralActivityService {
|
||||
// Null/failed, blank and unrecognized captures cannot establish recovery.
|
||||
if (runtime === "codex" && this.bannerEmitter) {
|
||||
const banner = detectCodexLimitBanner(content);
|
||||
if (banner || (content.trim().length > 0 && c.state !== "unknown")) {
|
||||
if (banner || (content.trim().length > 0 && obs.state !== "unknown")) {
|
||||
this.bannerEmitter(sessionName, banner, at);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -280,7 +280,8 @@ function findCurrentClaudeQuestion(paneContent: string): string | null {
|
||||
// Claude can leave these noninteractive warnings BELOW the input box and mode bar.
|
||||
// Recognize the complete input block, never a warning or historical prompt alone.
|
||||
const CLAUDE_STATUS_WARNINGS = [
|
||||
/^✘ Auto-update failed: no write permission to npm prefix · Run claude doctor$/,
|
||||
/^✘ Auto-update failed(?:: [^·\r\n]+)? · Run claude doctor$/,
|
||||
/^✘ Auto-update failed · Try claude doctor or (?:npm i -g @anthropic-ai\/claude-code|cd ~\/\.claude\/local && npm update @anthropic-ai\/claude-code)$/,
|
||||
/^tmux focus-events off · add 'set -g focus-events on' to ~\/\.tmux\.conf and re…$/,
|
||||
/^You've used (?:\d|[1-9]\d)% of your weekly limit · resets \d{1,2}(?::\d{2})?(?:am|pm) \(UTC\)$/,
|
||||
// #1641: Claude 2.1.293's update banner, whatever installer it names.
|
||||
@@ -1834,8 +1835,9 @@ const SEAT_IDLE_SHAPES: {
|
||||
},
|
||||
},
|
||||
codex: {
|
||||
// The hint row under the status line of an empty box (product-path-idle.J, idle).
|
||||
hints: [/^\? for shortcuts$/, /^\? for shortcuts\s+⚠ \d+ warnings? · f2 to view$/],
|
||||
// Empty-box hints: 0.160 product-path-idle.J/idle, and the captured 0.167 completed frame.
|
||||
hints: [/^\? for shortcuts$/, /^\? for shortcuts\s+⚠ \d+ warnings? · f2 to view$/,
|
||||
/^← for agents · \? for shortcuts\s+⚠ \d+ warnings? · f2 to view$/],
|
||||
// The last non-blank row above the empty box: the completion row Codex adds when a turn completes
|
||||
// (after-enter-15s.J; its grammar is CODEX_COMPLETION_ROW above), or the greeting under the session header
|
||||
// before any turn (idle, product-path-idle.J, cleared-without-enter.J). A running or streaming turn draws rows
|
||||
@@ -1908,7 +1910,7 @@ const VISIBLE_CLIENT_VERSION: Record<"claude-code" | "codex", RegExp[]> = {
|
||||
};
|
||||
|
||||
/** Whether a frame visibly names a client version other than the recorded one (see RECORDED_CLIENT_VERSION). */
|
||||
function namesAnotherClientVersion(pane: string, runtime: "claude-code" | "codex"): boolean {
|
||||
export function namesAnotherClientVersion(pane: string, runtime: "claude-code" | "codex"): boolean {
|
||||
return VISIBLE_CLIENT_VERSION[runtime].some((pattern) =>
|
||||
[...pane.matchAll(pattern)].some((match) => match[1] !== RECORDED_CLIENT_VERSION[runtime]));
|
||||
}
|
||||
|
||||
@@ -116,6 +116,11 @@ export type SuccessorLaunchResult =
|
||||
message: string;
|
||||
/** True once the predecessor process has been replaced in its pane. */
|
||||
replacementStarted: boolean;
|
||||
/** The pane the successor was being started in, once it is resolved (start_agent failures). */
|
||||
tmuxPane?: string;
|
||||
/** True when the harness launch was sent into the pane and only readiness failed (#2985): the successor may
|
||||
* still come up there, and it is not adopted. */
|
||||
successorLaunched?: boolean;
|
||||
};
|
||||
|
||||
export class SuccessorSessionLauncher {
|
||||
@@ -293,11 +298,14 @@ export class SuccessorSessionLauncher {
|
||||
// 3. Route the successor through REAL runtime startup (launchHarness send-keys + readiness) so it
|
||||
// becomes a LIVE, READY agent in the reused pane BEFORE it can be committed. UNWIND INVARIANT:
|
||||
// on ANY launch/readiness failure we do NOT killSession the preserved seat — that would destroy
|
||||
// the retiree's recoverable state. We return the structured failure and leave the re-wakeable
|
||||
// shell in the pane; commit never runs, so the binding is not repointed.
|
||||
// the retiree's recoverable state. We return the structured failure; commit never runs, so the binding is
|
||||
// not repointed. When the harness launch was already sent and only readiness failed, the pane is NOT left as
|
||||
// a bare shell: the successor may still come up there, unadopted (#2985). The failure says so, so the caller
|
||||
// can tell the operator that a retry would end it.
|
||||
const started = await this.startAgent(input.node, departingSession, pane.id, cwd, input.forkSource, input.occupantGeneration);
|
||||
if (!started.ok) {
|
||||
return { ok: false, code: started.code, step: "start_agent", message: started.message, replacementStarted: true };
|
||||
return { ok: false, code: started.code, step: "start_agent", message: started.message, replacementStarted: true,
|
||||
tmuxPane: pane.id, ...(started.launched ? { successorLaunched: true } : {}) };
|
||||
}
|
||||
|
||||
const discovered = this.discoveryRepo.upsertDiscoveredSession({
|
||||
@@ -386,7 +394,7 @@ export class SuccessorSessionLauncher {
|
||||
forkSource?: ForkSource,
|
||||
launchGeneration?: string | null,
|
||||
staged?: StagedLaunch,
|
||||
): Promise<{ ok: true; resumeToken?: string; resumeType?: string; appliedLaunch?: AppliedLaunchObservation; notes?: string[] } | { ok: false; code: string; message: string }> {
|
||||
): Promise<{ ok: true; resumeToken?: string; resumeType?: string; appliedLaunch?: AppliedLaunchObservation; notes?: string[] } | { ok: false; code: string; message: string; launched?: true }> {
|
||||
const adapter = node.runtime ? this.runtimeAdapters[node.runtime] : undefined;
|
||||
if (!adapter) {
|
||||
return {
|
||||
@@ -472,9 +480,9 @@ export class SuccessorSessionLauncher {
|
||||
// createSession already succeeded (adapter/socket/permission errors rethrow).
|
||||
// A thrown exception must NOT reject createSuccessor before its kill/unwind
|
||||
// runs (that would LEAK the unmanaged successor + surface an unstructured 500).
|
||||
// Catch it here and return a STRUCTURED ok:false so the caller kills the
|
||||
// just-created session (killBestEffort) and fails loudly at step=start_agent —
|
||||
// exactly like a returned readiness failure.
|
||||
// Catch it here and return a STRUCTURED ok:false so the caller fails loudly at
|
||||
// step=start_agent, exactly like a returned readiness failure. Nothing kills the
|
||||
// pane (unwind invariant above): the launched successor may still be running there.
|
||||
let readiness: ReadinessResult;
|
||||
try {
|
||||
readiness = await this.waitForReady(adapter, binding);
|
||||
@@ -483,6 +491,7 @@ export class SuccessorSessionLauncher {
|
||||
ok: false,
|
||||
code: "successor_readiness_failed",
|
||||
message: `Successor readiness probe threw: ${err instanceof Error ? err.message : String(err)}`,
|
||||
launched: true,
|
||||
};
|
||||
}
|
||||
if (!readiness.ready) {
|
||||
@@ -490,6 +499,7 @@ export class SuccessorSessionLauncher {
|
||||
ok: false,
|
||||
code: isAttentionRequiredReadinessCode(readiness.code) ? "successor_attention_required" : "successor_not_ready",
|
||||
message: `Successor did not become a ready agent: ${readiness.reason ?? "readiness timeout"}`,
|
||||
launched: true,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -651,6 +651,9 @@ export interface AgentActivity {
|
||||
sourceSequence?: number | null;
|
||||
/** How often the emitter promises to report while idle; an idle report older than two of these no longer holds. */
|
||||
livenessSeconds?: number | null;
|
||||
/** #99: when a Claude usage limit (StopFailure/rate_limit) says it resets, read by the relay from Claude's own
|
||||
* message. Only the instant is kept, never the message. */
|
||||
resetsAt?: string | null;
|
||||
rawEvent?: string | null;
|
||||
rawSubtype?: string | null;
|
||||
runtime?: string | null;
|
||||
|
||||
@@ -384,6 +384,7 @@ export async function startServer(port?: number) {
|
||||
// tree post-bind so the HTTP surface is ready before the first
|
||||
// tick (matches contextMonitor pattern).
|
||||
deps.watchdogScheduler?.start();
|
||||
deps.networkConnector?.start();
|
||||
deps.sessionTransport?.guardedDelivery?.recover();
|
||||
// S10 — start the gateway subsystem's NETWORK services (replay, outbound poll,
|
||||
// Socket Mode inbound) post-bind, contextMonitor pattern: the composed wire is
|
||||
@@ -452,6 +453,7 @@ export async function startServer(port?: number) {
|
||||
["proof-source-watch", () => deps.proofSourceWatch?.close()],
|
||||
["health-diagnosis", () => deps.healthDiagnosis?.stop()],
|
||||
["watchdog", () => deps.watchdogScheduler?.stop()],
|
||||
["network-connector", () => deps.networkConnector?.stop()],
|
||||
["stream-intake", () => deps.streamIntakeCoordinator?.stop()],
|
||||
["draft-delivery", () => deps.sessionTransport?.guardedDelivery?.stop()],
|
||||
["seat-activity", () => deps.seatActivityService?.stop()],
|
||||
|
||||
@@ -116,6 +116,16 @@ export function hookRelayRoute(value: unknown): EvidenceRoute | undefined {
|
||||
const hookEvidenceSeq = new Map<string, number>();
|
||||
// What each seat's runner has said about its turns, for deciding whether a working beat counts (the runner contract §4).
|
||||
const workingLiveness = new WorkingLivenessGate();
|
||||
/** #99: the reset instant the relay read from a Claude usage limit, as a canonical ISO timestamp, or null. The store
|
||||
* keeps it only on the typed StopFailure/rate_limit row; the provider lane decides whether it is still ahead. */
|
||||
function limitResetsAt(body: Record<string, unknown>): string | null {
|
||||
if (typeof body.resetsAt !== "string" || body.resetsAt.length > 40) return null;
|
||||
const ms = Date.parse(body.resetsAt);
|
||||
// The relay's eight-day horizon, enforced at ingress too: no sender can park a pool for longer than the relay could.
|
||||
return Number.isFinite(ms) && ms <= Date.now() + LIMIT_RESET_MAX_MS ? new Date(ms).toISOString() : null;
|
||||
}
|
||||
const LIMIT_RESET_MAX_MS = 8 * 24 * 3600 * 1000;
|
||||
|
||||
function sourceSequence(body: Record<string, unknown>): number | null {
|
||||
return typeof body.sequence === "number" && Number.isSafeInteger(body.sequence) && body.sequence > 0 ? body.sequence : null;
|
||||
}
|
||||
@@ -727,6 +737,8 @@ function recordHookActivity(c: Context, store: AgentActivityStore, body: Record<
|
||||
livenessSeconds: beat ? workingLivenessSeconds(body.livenessSeconds)
|
||||
: typeof body.livenessSeconds === "number" && Number.isSafeInteger(body.livenessSeconds)
|
||||
&& body.livenessSeconds > 0 && body.livenessSeconds <= 3600 ? body.livenessSeconds : null,
|
||||
resetsAt: limitResetsAt(body),
|
||||
sessionId: stringOrNull(body.sessionId),
|
||||
});
|
||||
|
||||
if (!result.ok) {
|
||||
|
||||
@@ -50,11 +50,10 @@ import { mkdirSync, writeFileSync, rmSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { randomUUID, randomInt } from "node:crypto";
|
||||
import { connect } from "node:net";
|
||||
import { execFile } from "node:child_process";
|
||||
import { authBearerTokenMiddleware } from "../middleware/auth-bearer-token.js";
|
||||
import { getOpenRigHome } from "../openrig-compat.js";
|
||||
import { addHostEntry, updateHostEntry, removeHostEntry } from "../domain/hosts/hosts-registry-writer.js";
|
||||
import { defaultHostRegistryPath, loadHostRegistry, validateHostRegistry, type HostEntry, type SshHostEntry } from "../domain/hosts/hosts-registry-reader.js";
|
||||
import { defaultHostRegistryPath, loadHostRegistry, validateHostRegistry, type HostEntry } from "../domain/hosts/hosts-registry-reader.js";
|
||||
import type { SettingsStore } from "../domain/user-settings/settings-store.js";
|
||||
import { UNATTRIBUTED_ACTOR, type QueueItem, type QueueRepository } from "../domain/queue-repository.js";
|
||||
import { KERNEL_OPERATOR_SESSION, resolveDefaultReceiver } from "../domain/default-receiver.js";
|
||||
@@ -332,41 +331,16 @@ export function hostsRoutes(opts?: { bearerToken?: string | null; humanRegistry?
|
||||
// POINTERS by construction) + `selected` + a bounded coarse `status`.
|
||||
// ---------------------------------------------------------------------
|
||||
|
||||
// Where ssh would connect for this host, from `ssh -G` (reads ssh config, no connection); null when
|
||||
// ssh can't resolve it or a ProxyJump/ProxyCommand sits in between. Twin of the CLI's resolveSshEndpoint.
|
||||
function resolveSshEndpoint(host: SshHostEntry, timeoutMs: number): Promise<{ hostname: string; port: number } | null> {
|
||||
// `--` before the target, the same separator the CLI twin and the executor pass: a
|
||||
// destination that begins with `-` is read as the host, not as an option.
|
||||
const args = [...(host.user ? ["-l", host.user] : []), "-G", "--", host.target];
|
||||
return new Promise((resolve) => {
|
||||
execFile("ssh", args, { timeout: timeoutMs }, (error, stdout) => {
|
||||
if (error) return resolve(null);
|
||||
const conf = new Map(stdout.split("\n").map((line) => {
|
||||
const i = line.indexOf(" ");
|
||||
return [line.slice(0, i).toLowerCase(), line.slice(i + 1).trim()] as const;
|
||||
}));
|
||||
const proxied = ["proxyjump", "proxycommand"].some((k) => (conf.get(k) ?? "none") !== "none");
|
||||
const hostname = conf.get("hostname");
|
||||
const port = Number(conf.get("port"));
|
||||
resolve(hostname && Number.isInteger(port) && !proxied ? { hostname, port } : null);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function probeHost(host: HostEntry, timeoutMs = 1500): Promise<"reachable" | "unreachable" | "unknown"> {
|
||||
try {
|
||||
let target: string;
|
||||
let port: number;
|
||||
if (host.transport === "ssh") {
|
||||
const endpoint = await resolveSshEndpoint(host, timeoutMs);
|
||||
if (!endpoint) return "unknown";
|
||||
({ hostname: target, port } = endpoint);
|
||||
} else {
|
||||
const u = new URL(host.url);
|
||||
// WHATWG keeps the brackets on an IPv6 hostname; net.connect wants the bare address.
|
||||
target = u.hostname.replace(/^\[(.*)\]$/, "$1");
|
||||
port = u.port ? Number(u.port) : u.protocol === "https:" ? 443 : 80;
|
||||
// Passive SSH rows have no authority to read SSH config or probe sshd.
|
||||
return "unknown";
|
||||
}
|
||||
const u = new URL(host.url);
|
||||
// WHATWG keeps the brackets on an IPv6 hostname; net.connect wants the bare address.
|
||||
const target = u.hostname.replace(/^\[(.*)\]$/, "$1");
|
||||
const port = u.port ? Number(u.port) : u.protocol === "https:" ? 443 : 80;
|
||||
return new Promise((resolve) => {
|
||||
const sock = connect({ host: target, port, timeout: timeoutMs });
|
||||
sock.on("connect", () => { sock.destroy(); resolve("reachable"); });
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
import { Hono, type Context } from "hono";
|
||||
import type { RigRepository } from "../domain/rig-repository.js";
|
||||
import { NetworkBindingError, NetworkBindingStore } from "../domain/network/network-bindings.js";
|
||||
import { networkClosedElsewhere } from "../domain/network/network-connector.js";
|
||||
import { authBearerTokenMiddleware } from "../middleware/auth-bearer-token.js";
|
||||
import { OPENRIG_HOME } from "../openrig-compat.js";
|
||||
|
||||
/** A local seat's connections to a remote hub (`rig network`). Reads list every binding with its state and last contact;
|
||||
* the token is accepted on connect and never returned. Writes are gated like the hosts routes: enforced when an operator
|
||||
* bearer is configured, passed through on a loopback or tailnet-trusted daemon. */
|
||||
export function networkRoutes(opts?: { bearerToken?: string | null; home?: string }): Hono {
|
||||
const router = new Hono();
|
||||
const requireAuth = authBearerTokenMiddleware({ expectedToken: opts?.bearerToken ?? null });
|
||||
router.use("/bindings", async (c, next) => (c.req.method === "GET" ? next() : requireAuth(c, next)));
|
||||
router.use("/bindings/:id/disconnect", requireAuth);
|
||||
|
||||
const store = (c: Context) => new NetworkBindingStore((c.get("rigRepo" as never) as RigRepository).db, opts?.home ?? OPENRIG_HOME);
|
||||
const refused = (c: Context, error: unknown) => {
|
||||
if (error instanceof NetworkBindingError) return c.json({ error: error.code, message: error.message }, error.status);
|
||||
throw error;
|
||||
};
|
||||
|
||||
// The connector's own view (next pass, backoff, last pass) when it runs in this daemon; null otherwise.
|
||||
router.get("/bindings", (c) => {
|
||||
const connector = c.get("networkConnector" as never) as { snapshot(): unknown } | undefined;
|
||||
const db = (c.get("rigRepo" as never) as RigRepository).db;
|
||||
// Hub items someone else closed before a binding's result reached them: a conflict the person sees.
|
||||
const bindings = store(c).list().map((b) => ({ ...b, closedElsewhere: networkClosedElsewhere(db, b.id) }));
|
||||
return c.json({ bindings, connector: connector?.snapshot() ?? null });
|
||||
});
|
||||
|
||||
router.post("/bindings", async (c) => {
|
||||
let body: Record<string, unknown>;
|
||||
try { body = await c.req.json(); } catch { return c.json({ error: "invalid_body", message: "The body must be JSON." }, 400); }
|
||||
try {
|
||||
return c.json(store(c).create({ session: body.session, hubUrl: body.hubUrl, hubSeat: body.hubSeat, token: body.token, allowHttp: body.allowHttp }), 201);
|
||||
} catch (error) { return refused(c, error); }
|
||||
});
|
||||
|
||||
router.post("/bindings/:id/disconnect", async (c) => {
|
||||
let body: Record<string, unknown> = {};
|
||||
try { body = await c.req.json(); } catch { /* no reason given */ }
|
||||
try {
|
||||
return c.json(store(c).disconnect(c.req.param("id"), typeof body.reason === "string" ? body.reason : undefined));
|
||||
} catch (error) { return refused(c, error); }
|
||||
});
|
||||
|
||||
return router;
|
||||
}
|
||||
@@ -3,7 +3,7 @@ import { proofSourceObservation } from "../domain/proof/source-watch.js";
|
||||
import * as path from "node:path";
|
||||
import type { SliceIndexer } from "../domain/slices/slice-indexer.js";
|
||||
import type { EventBus } from "../domain/event-bus.js";
|
||||
import { JudgmentError, evidenceAt, readSliceReadiness, readMissionReadiness, readProjectReadiness, recordJudgment, resolveProofScope, resolveProjectRoot, type JudgeInput } from "../domain/proof/judgments.js";
|
||||
import { JudgmentError, evidenceAt, readSliceReadiness, readMissionReadiness, readProjectReadinessAsync, recordJudgment, resolveProofScope, resolveProjectRoot, type JudgeInput } from "../domain/proof/judgments.js";
|
||||
import { requireSenderIdentity, resolveRecordedProvenance } from "./require-sender-identity.js";
|
||||
import { projectById, sameProjectRoot } from "../domain/workspace/project-read.js";
|
||||
import { ProjectReadError } from "../domain/workspace/project-catalog.js";
|
||||
@@ -78,12 +78,15 @@ export function proofRoutes(): Hono {
|
||||
// freshness is reported as unverified rather than borrowing the selected workspace's state.
|
||||
const observation = (c: Parameters<typeof indexer>[0], project: { id: string; root: string } | null) =>
|
||||
project ? { state: "unavailable" as const, revision: "unverified" } : proofSourceObservation(c);
|
||||
app.get("/", c => {
|
||||
app.get("/", async c => {
|
||||
const { root, evidenceRoot, scope, project } = target(c, c.req.query("scope"), c.req.query("project"), c.req.query("projectRoot"));
|
||||
const common = { sourceObservation: observation(c, project), ...(project ? { project } : {}) };
|
||||
// A catalog project's own root bounds policy, scope identity and evidence (never an ancestor's project.yaml).
|
||||
const bound = project?.root;
|
||||
if (!scope) return c.json({ ...readProjectReadiness(root, undefined, bound), ...common });
|
||||
if (!scope) {
|
||||
const result = await readProjectReadinessAsync(root, undefined, bound);
|
||||
return c.json({ ...result, ...common, sourceObservation: observation(c, project) });
|
||||
}
|
||||
const dir = resolveProofScope(root, scope);
|
||||
if (path.basename(path.dirname(dir)) !== "slices") return c.json({ ...readMissionReadiness(dir, undefined, bound), ...common });
|
||||
const refs = c.req.queries("evidence") ?? [];
|
||||
|
||||
@@ -18,7 +18,7 @@ import { inferCatalogProject, ProjectReadError } from "../domain/workspace/proje
|
||||
// avoid `/proof-asset` being parsed as a slice-name.
|
||||
|
||||
import { Hono, type Context } from "hono";
|
||||
import { readSliceReadiness, readProjectReadiness } from "../domain/proof/judgments.js";
|
||||
import { readSliceReadiness, readProjectReadinessAsync } from "../domain/proof/judgments.js";
|
||||
import * as fs from "node:fs";
|
||||
import * as path from "node:path";
|
||||
import { SliceIndexer, SliceListEntry, SliceStatus } from "../domain/slices/slice-indexer.js";
|
||||
@@ -37,7 +37,7 @@ export function slicesRoutes(): Hono {
|
||||
|
||||
// 1) Static literal `/` BEFORE dynamic `/:name` so the list isn't
|
||||
// shadowed by a slice named "list" or similar.
|
||||
app.get("/", (c) => {
|
||||
app.get("/", async (c) => {
|
||||
const deps = getDeps(c);
|
||||
if (!deps) return c.json({ error: "slices_indexer_unavailable" }, 503);
|
||||
if (!deps.indexer.isReady()) {
|
||||
@@ -61,7 +61,7 @@ export function slicesRoutes(): Hono {
|
||||
// operation: the list rebuild, the boundToWorkflow per-slice get loop,
|
||||
// and the mission sidecar share ONE membership batch (pre-scope, each
|
||||
// uncached get built its own 2-scan batch: 2+2N total queue scans).
|
||||
return deps.indexer.withMembershipBatch(() => {
|
||||
const listing = deps.indexer.withMembershipBatch(() => {
|
||||
const all = deps.indexer.list();
|
||||
let filtered = filter === "all" ? all : all.filter((s) => s.status === filter);
|
||||
// Workflows in Spec Library v0: optional lens filter — narrow to
|
||||
@@ -102,7 +102,7 @@ export function slicesRoutes(): Hono {
|
||||
complete: projectLookup.complete,
|
||||
});
|
||||
const authored = deps.indexer.missionAuthoredStatuses();
|
||||
return c.json({
|
||||
return {
|
||||
slices: filtered.map((s) => ({
|
||||
...s,
|
||||
queueCounts: queueCountsByPath.get(s.slicePath) ?? null,
|
||||
@@ -114,13 +114,17 @@ export function slicesRoutes(): Hono {
|
||||
// VM-005 (release-0.4.7): additive authored mission-status sidecar so
|
||||
// chip surfaces can honor authored-wins precedence without a second
|
||||
// round-trip. The `slices` array itself is byte-untouched.
|
||||
missions: { ...authored, ...Object.fromEntries(readProjectReadiness(deps.indexer.slicesRoot).missions.map(m => [m.name, {
|
||||
...authored[m.name],
|
||||
authoredStatus: m.historicalStatus ?? authored[m.name]?.authoredStatus ?? null,
|
||||
readiness: m,
|
||||
}])) },
|
||||
});
|
||||
missions: authored,
|
||||
};
|
||||
});
|
||||
if (listing instanceof Response) return listing;
|
||||
// The membership batch is synchronous and request-local; finish it before yielding to other requests.
|
||||
const readiness = await readProjectReadinessAsync(deps.indexer.slicesRoot);
|
||||
return c.json({ ...listing, missions: { ...listing.missions, ...Object.fromEntries(readiness.missions.map(m => [m.name, {
|
||||
...listing.missions[m.name],
|
||||
authoredStatus: m.historicalStatus ?? listing.missions[m.name]?.authoredStatus ?? null,
|
||||
readiness: m,
|
||||
}])) } });
|
||||
});
|
||||
|
||||
// V0.3.1 slice 17 founder-walk-workspace-state-correctness (walk item 8 — Explorer auto-show): explicit cache invalidation surface.
|
||||
|
||||
@@ -84,6 +84,7 @@ import { skillsRoutes } from "./routes/skills.js";
|
||||
import type { SkillLibraryDiscoveryService } from "./domain/skill-library-discovery.js";
|
||||
import { configRoutes } from "./routes/config.js";
|
||||
import { hostsRoutes } from "./routes/hosts.js";
|
||||
import { networkRoutes } from "./routes/network.js";
|
||||
import { hostReadThrough } from "./domain/hosts/read-through.js";
|
||||
import { jsonBodyErrorHandler, trackJsonBodyParseErrors } from "./middleware/json-body-error.js";
|
||||
import { getSelfHostId, getSelfHostIdSource } from "./domain/hosts/fanout-contract.js";
|
||||
@@ -242,6 +243,8 @@ export interface AppDeps {
|
||||
watchdogHistoryLog?: WatchdogHistoryLog;
|
||||
watchdogPolicyEngine?: WatchdogPolicyEngine;
|
||||
watchdogScheduler?: WatchdogScheduler;
|
||||
/** The outbound network connector (`rig network`); started after the listener binds. */
|
||||
networkConnector?: import("./domain/network/network-connector.js").NetworkConnectorService;
|
||||
/** B8 / slice-07 A3 — the model-divergence monitor (effective-vs-pinned, four-channel proclaim). */
|
||||
modelDivergenceMonitor?: import("./domain/model-divergence/model-divergence-monitor.js").ModelDivergenceMonitor;
|
||||
/** S10 — the in-daemon gateway subsystem (amended M1 §3: in-process, no second deployable).
|
||||
@@ -634,6 +637,7 @@ export function createApp(deps: AppDeps): Hono {
|
||||
c.set("watchdogHistoryLog" as never, deps.watchdogHistoryLog);
|
||||
c.set("watchdogPolicyEngine" as never, deps.watchdogPolicyEngine);
|
||||
c.set("watchdogScheduler" as never, deps.watchdogScheduler);
|
||||
c.set("networkConnector" as never, deps.networkConnector);
|
||||
c.set("workflowRuntime" as never, deps.workflowRuntime);
|
||||
c.set("workflowBuiltinSpecsDir" as never, deps.workflowBuiltinSpecsDir);
|
||||
c.set("workflowsFolderDir" as never, deps.workflowsFolderDir);
|
||||
@@ -872,6 +876,9 @@ export function createApp(deps: AppDeps): Hono {
|
||||
"/api/hosts",
|
||||
hostsRoutes({ bearerToken: deps.missionControlBearerToken ?? null }),
|
||||
);
|
||||
// A selected local seat's connections to a remote hub (`rig network`): bindings, status and disconnect. Its writes carry
|
||||
// a participant's token, so they take the same operator-bearer posture as the hosts routes.
|
||||
app.route("/api/network", networkRoutes({ bearerToken: deps.missionControlBearerToken ?? null }));
|
||||
// Slice Story View v0 — slice indexer + per-tab payload routes.
|
||||
app.route("/api/slices", slicesRoutes());
|
||||
// Living Notes Packet 2 (OPR.0.4.4.20) — composed-review read contract.
|
||||
|
||||
@@ -2118,7 +2118,7 @@ export async function createDaemon(opts?: DaemonOptions): Promise<DaemonResult>
|
||||
deps.sliceIndexer = sliceIndexer;
|
||||
if (sliceIndexer.isReady()) {
|
||||
const { watchProofSources } = await import("./domain/proof/source-watch.js");
|
||||
deps.proofSourceWatch = watchProofSources(sliceIndexer.slicesRoot, () => sliceIndexer.invalidate(), eventBus, () => {
|
||||
deps.proofSourceWatch = await watchProofSources(sliceIndexer.slicesRoot, () => sliceIndexer.invalidate(), eventBus, () => {
|
||||
sliceIndexer.invalidate();
|
||||
eventBus.emit({ type: "scope.sources_changed", scope: sliceIndexer.slicesRoot });
|
||||
});
|
||||
@@ -2544,6 +2544,19 @@ export async function createDaemon(opts?: DaemonOptions): Promise<DaemonResult>
|
||||
});
|
||||
deps.watchdogPolicyEngine = watchdogPolicyEngine;
|
||||
deps.watchdogScheduler = watchdogScheduler;
|
||||
// The outbound network connector: a pass per active `rig network` binding. With none, it makes no network call.
|
||||
{
|
||||
const { NetworkConnectorService } = await import("./domain/network/network-connector.js");
|
||||
const { NetworkBindingStore } = await import("./domain/network/network-bindings.js");
|
||||
const pollSeconds = Number(process.env.OPENRIG_NETWORK_POLL_SECONDS);
|
||||
deps.networkConnector = new NetworkConnectorService({
|
||||
db: queueRepoInstance.db,
|
||||
queueRepo: queueRepoInstance,
|
||||
bindings: new NetworkBindingStore(queueRepoInstance.db, OPENRIG_HOME),
|
||||
home: OPENRIG_HOME,
|
||||
log: (line) => console.log(`[network] ${line}`),
|
||||
}, Number.isFinite(pollSeconds) && pollSeconds >= 5 ? { intervalMs: pollSeconds * 1000 } : {});
|
||||
}
|
||||
|
||||
// OPR.0.5.6.24 F-14 — ONE parked-owner supervisor job per rig (mini-req 1):
|
||||
// ensured idempotently on the (policy, target_session) tuple; the anchor is
|
||||
|
||||
@@ -16,6 +16,8 @@ const oldIdle = { state: "idle", reason: "stop_hook", evidenceSource: "runtime_h
|
||||
const CNAME = "review-r1@sample-rig";
|
||||
const cframe = readFileSync(new URL("./fixtures/input-box/claude-code-2.1.296/openrig-seat-accept-edits-idle.txt", import.meta.url), "utf8");
|
||||
const codexFrame = readFileSync(new URL("./fixtures/input-box/codex-0.160.0/after-enter-15s.J.txt", import.meta.url), "utf8");
|
||||
const codexCompleted = readFileSync(new URL("./fixtures/input-box/codex-0.167.0/completed.txt", import.meta.url), "utf8");
|
||||
const codexWorking = readFileSync(new URL("./fixtures/input-box/codex-0.167.0/working.txt", import.meta.url), "utf8");
|
||||
const busyFrame = cframe.replace(/^✻ Churned for 4s · done 5:24 PM$/m, "✻ Working… (3s · thinking)");
|
||||
async function adapterFold(frame: string, opts: { age?: number; sourceName?: string; sourceRuntime?: "claude-code" | "codex"; runtime?: string; hook?: object | null; lastWriteAge?: number; motion?: boolean } = {}) {
|
||||
const capturedAt = new Date(NOW.getTime() - (opts.age ?? 0));
|
||||
@@ -57,6 +59,63 @@ describe("issue824 existing strict adapter idle frame (behavioral red-first)", (
|
||||
const streaming = codexFrame.replace(/^ {2}Worked for[^\n]+$/m, "• Streaming an unfinished answer");
|
||||
expect((await adapterFold(streaming, { sourceRuntime: "codex" })).activity.state).toBe("running");
|
||||
});
|
||||
it.each(["0.167.0", "99.0.0"])("an unrecorded Codex %s frame stays unknown under redraw motion", async (version) => {
|
||||
// Synthetic version edits of the recorded 0.160 fixture, not native newer-version captures.
|
||||
const frame = codexFrame.replaceAll("0.160.0", version);
|
||||
for (const pane of [frame, frame.replace(/^ {2}Worked for[^\n]+$/m, "• Streaming an unfinished answer")]) {
|
||||
expect(readSeatBeforePaste(pane, "codex")).toBe("unknown");
|
||||
const result = await adapterFold(pane, { sourceRuntime: "codex", hook: null, lastWriteAge: 2000 });
|
||||
expect(result.activity.state).toBe("unknown");
|
||||
expect(result.activity.reason).not.toBe("window_activity_motion");
|
||||
}
|
||||
});
|
||||
it("the captured Codex agents hint preserves completed idle despite redraws", async () => {
|
||||
expect.soft(readSeatBeforePaste(codexCompleted, "codex")).toBe("idle");
|
||||
for (const hook of [null, oldIdle]) {
|
||||
const result = await adapterFold(codexCompleted, { sourceRuntime: "codex", hook });
|
||||
expect.soft(result.activity.state).toBe("idle");
|
||||
expect(result.activity.evidenceSource).toBe("pane_heuristic");
|
||||
}
|
||||
});
|
||||
it("the captured working frame remains running without motion", async () => {
|
||||
expect(readSeatBeforePaste(codexWorking, "codex")).toBe("not-idle");
|
||||
expect((await adapterFold(codexWorking, { sourceRuntime: "codex", hook: null, motion: false })).activity.state).toBe("running");
|
||||
});
|
||||
it("the agents footer alone cannot prove idle while output is unfinished", async () => {
|
||||
// Synthetic spinner-hidden stream, not a native streaming capture.
|
||||
const stream = codexCompleted.replace(/^ {2}Worked for[^\n]+$/m, "• Streaming an unfinished answer");
|
||||
expect(readSeatBeforePaste(stream, "codex")).toBe("unknown");
|
||||
expect((await adapterFold(stream, { sourceRuntime: "codex", hook: null })).activity.state).toBe("running");
|
||||
const badHint = codexCompleted.replace("← for agents", "unknown hint");
|
||||
expect(readSeatBeforePaste(badHint, "codex")).toBe("unknown");
|
||||
});
|
||||
it("an unsupported Codex frame cannot turn an idle or stale hook into running, while positive work and attention remain", async () => {
|
||||
const frame = codexFrame.replaceAll("0.160.0", "99.0.0");
|
||||
for (const hook of [oldIdle, { ...oldIdle, state: "unknown", stale: true }]) {
|
||||
expect((await adapterFold(frame, { sourceRuntime: "codex", hook })).activity.state).toBe("unknown");
|
||||
}
|
||||
for (const state of ["running", "needs_input"]) {
|
||||
expect((await adapterFold(frame, { sourceRuntime: "codex", hook: { ...oldIdle, state } })).activity.state).toBe(state);
|
||||
}
|
||||
});
|
||||
it.each([" >_ OpenAI Codex (v99.0.0)", "Update available! 99.0.0 -> 99.1.0"])("a fresh idle hook wins without motion on a version-named screen: %s", async (header) => {
|
||||
const frame = `${header}\n${codexCompleted}`;
|
||||
const hook = { ...oldIdle, sampledAt: NOW.toISOString() };
|
||||
expect(readSeatBeforePaste(frame, "codex")).toBe("unknown");
|
||||
const still = await adapterFold(frame, { sourceRuntime: "codex", hook, motion: false });
|
||||
expect(still.observation).toMatchObject({ state: "unknown", reason: "unrecognized_client_version" });
|
||||
expect(still.activity).toEqual(hook);
|
||||
const moving = await adapterFold(frame, { sourceRuntime: "codex", hook, motion: true });
|
||||
expect(moving.activity).toMatchObject({ state: "unknown", reason: "unrecognized_client_version" });
|
||||
});
|
||||
it.each([false, true])("positive pane work survives a version header with motion=%s", async (motion) => {
|
||||
// Synthetic version header over the recorded working frame.
|
||||
const frame = ` >_ OpenAI Codex (v99.0.0)\n${codexWorking}`;
|
||||
expect(readSeatBeforePaste(frame, "codex")).toBe("not-idle");
|
||||
const result = await adapterFold(frame, { sourceRuntime: "codex", hook: null, motion });
|
||||
expect(result.observation).toMatchObject({ state: "agent_active", reason: "mid_work_pattern" });
|
||||
expect(result.activity).toMatchObject({ state: "running", evidenceSource: "pane_heuristic", reason: "mid_work_pattern" });
|
||||
});
|
||||
it("a newer redraw after an old idle capture keeps D2 running", async () => {
|
||||
expect((await adapterFold(cframe, { age: 2000 })).activity.state).toBe("running");
|
||||
// The write and capture are in the same second: ordering is unknown,
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
// Reuses the released handover fixture. Process startup and pane bytes are modeled; no native Claude.
|
||||
import { NativePermissionStore } from "../src/domain/native-permission-store.js";
|
||||
import { verifyClaudePaneProcess } from "../src/domain/native-process-lineage.js";
|
||||
import { ClaudeCodeAdapter } from "../src/adapters/claude-code-adapter.js";
|
||||
import { mkdirSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
@@ -385,6 +386,92 @@ describe("Claude fresh in-place handover readiness", () => {
|
||||
expect(killSession).not.toHaveBeenCalled();
|
||||
|
||||
});
|
||||
|
||||
// #2985: a rebuild handover whose readiness window (30 s) closes before the new Claude is proven in the pane. The
|
||||
// retiree is already ended and the pane respawned, so the failure leaves the successor that comes up a few seconds
|
||||
// later running, with the binding on the previous conversation and its generation uncommitted. The result must say
|
||||
// so instead of "retry", which would end that successor.
|
||||
it("2985 rebuild: readiness times out at 30 s, the successor comes up after it and keeps running unadopted; the failure says so", async () => {
|
||||
const startedAtMs = Date.parse("2026-11-10T16:21:00Z");
|
||||
vi.useFakeTimers({ toFake: ["Date"] });
|
||||
vi.setSystemTime(startedAtMs);
|
||||
const { node, sessionId } = seedSeat({ runtime: "claude-code" });
|
||||
const previousToken = "11111111-1111-4111-8111-111111111111";
|
||||
sessionRegistry.updateResumeToken(sessionId!, "claude_id", previousToken, "hook");
|
||||
const successorToken = "22222222-2222-4222-8222-222222222222";
|
||||
const nativeAt = 33_000; // the issue: the new Claude was seen about 3 s after the 30 s window closed
|
||||
let elapsed = 0, requested = false;
|
||||
const running = () => requested && elapsed >= nativeAt;
|
||||
const oldFrame = "Claude Code v2.1.322\n❯ predecessor finished turn\n ⏵⏵ accept edits on (shift+tab to cycle)";
|
||||
// The new Claude's frame is drawn from the launch on, so readiness waits on the process proof
|
||||
// ("Waiting to confirm the new Claude process in this pane."), which the 30 s window never sees.
|
||||
const freshFrame = "───────── dev-impl@seat-rig ─\n❯ \n─────────\n ⏵⏵ accept edits on (shift+tab to cycle)";
|
||||
getPaneCommand.mockImplementation(async () => running() ? "claude" : "zsh");
|
||||
capturePaneScreen.mockImplementation(async () => requested ? freshFrame : `${oldFrame}\n❯ `);
|
||||
const capturePaneContent = vi.fn(async () => requested ? freshFrame : `${oldFrame}\n❯ `);
|
||||
const target = { ...tmux(), capturePaneContent, getPanePid: vi.fn(async () => 100),
|
||||
sendShellCommand: async (session: string, text: string) => sendText(session, text) } as unknown as TmuxAdapter;
|
||||
const startedAt = "Tue Nov 10 16:21:00 2026";
|
||||
const listProcesses = vi.fn(() => [
|
||||
{ pid: 100, ppid: 1, pgid: 100, tpgid: running() ? 102 : 100, command: "-zsh", executableName: "zsh", startedAt },
|
||||
...(running() ? [{ pid: 102, ppid: 100, pgid: 102, tpgid: 102,
|
||||
command: `claude --session-id ${successorToken} --name dev-impl@seat-rig`, executableName: "claude", startedAt }] : []),
|
||||
]);
|
||||
sendText.mockImplementation(async (_session: string, text: string) => {
|
||||
if (text.includes("claude ") && text.includes(`--session-id ${successorToken}`)) requested = true;
|
||||
return { ok: true };
|
||||
});
|
||||
const fsError = () => { throw new Error("unexpected filesystem access"); };
|
||||
const claude = new ClaudeCodeAdapter({ tmux: target, fsOps: { exists: () => false, readFile: fsError,
|
||||
writeFile: fsError, mkdirp: fsError, copyFile: fsError }, listProcesses, sleep: async () => {}, sessionIdFactory: () => successorToken });
|
||||
const handover = new SeatHandoverService({ db, rigRepo, sessionRegistry, discoveryRepo, eventBus,
|
||||
tmuxAdapter: target, now: () => new Date("2026-11-10T16:21:00Z"), newSuccessorId: () => "01SUCCID0",
|
||||
runtimeAdapters: { "claude-code": claude as unknown as RuntimeAdapter },
|
||||
contextUsageStore: { readSidecar } as never, resumeTokenCapturer: { captureCodexThreadId } as never,
|
||||
occupantInvalidator: { invalidateRetiringOccupant }, activityOracle: { declareOccupantSwap },
|
||||
predecessorRecapResolver: resolvePredecessorRecap as never, readinessTimeoutMs: 30_000,
|
||||
sleep: async ms => { elapsed += ms; vi.setSystemTime(startedAtMs + elapsed); },
|
||||
});
|
||||
const sessionsBefore = db.prepare("SELECT session_name,status,resume_token FROM sessions ORDER BY id").all();
|
||||
|
||||
const result = await handover.handover({ seatRef: "dev-impl@seat-rig", source: "rebuild", reason: "context-wall" });
|
||||
|
||||
// The real case: the in-place respawn ran, readiness timed out at 30 s, and the handover failed at start_agent.
|
||||
expect(result).toMatchObject({ ok: false, code: "successor_create_failed" });
|
||||
if (result.ok) throw new Error("expected failure");
|
||||
expect(result.message).toContain('Handover failed at step "start_agent"');
|
||||
expect(result.message).toContain("readiness timeout after 30s: Waiting to confirm the new Claude process in this pane.");
|
||||
expect(elapsed).toBeLessThan(nativeAt);
|
||||
expect(respawnPane).toHaveBeenCalledTimes(1);
|
||||
expect(killSession).not.toHaveBeenCalled();
|
||||
// The binding is unchanged: the previous conversation is still the resume target, nothing completed, and the
|
||||
// successor's reserved generation was never committed (its hooks are answered generation_uncommitted).
|
||||
expect(db.prepare("SELECT session_name,status,resume_token FROM sessions ORDER BY id").all()).toEqual(sessionsBefore);
|
||||
expect(sessionRegistry.getBindingForNode(node.id)?.tmuxSession).toBe("dev-impl@seat-rig");
|
||||
expect(db.prepare("SELECT COUNT(*) AS n FROM events WHERE type = 'seat.handover_completed'").get()).toEqual({ n: 0 });
|
||||
const generation = (respawnPane.mock.calls[0]?.[2] as { env?: Record<string, string> } | undefined)?.env?.OPENRIG_OCCUPANT_GENERATION;
|
||||
expect(generation).toBeTruthy();
|
||||
expect(sessionRegistry.isOccupantGenerationRegistered(node.id, generation!)).toBe(false);
|
||||
|
||||
// The fixture then models what the issue observed: a few seconds after the window closed, the successor is proven
|
||||
// in the pane under its own session id, while the seat still holds the previous one (asserted above).
|
||||
elapsed = nativeAt + 1000;
|
||||
expect(await verifyClaudePaneProcess({ target: "%9", tmux: target, listProcesses, expectedToken: successorToken })).not.toBeNull();
|
||||
expect(await verifyClaudePaneProcess({ target: "%9", tmux: target, listProcesses, expectedToken: previousToken })).toBeNull();
|
||||
|
||||
// The fix: the failure says a successor was launched and may be running unadopted, names its pane, and warns that
|
||||
// a retry or a fresh launch with --stop ends it, instead of telling the operator to retry.
|
||||
expect(result).toMatchObject({ replacementStarted: true, successorPane: "%9" });
|
||||
expect(result.guidance).not.toMatch(/\bretry\b/i);
|
||||
expect(result.guidance).toContain("binding is unchanged");
|
||||
expect(result.guidance).toContain('a successor was launched in pane "%9"');
|
||||
expect(result.guidance).toContain("OpenRig has not registered the successor as the seat's occupant");
|
||||
expect(result.guidance).toContain("rig capture 'dev-impl@seat-rig'");
|
||||
expect(result.guidance).toContain(`Another handover, or rig seat launch 'dev-impl@seat-rig' --fresh --stop --reason "<why>", ends that successor if it goes ahead`);
|
||||
// A readiness failure doesn't prove the process exited: Claude can still be starting behind shell-looking output.
|
||||
expect(result.guidance).not.toContain("no successor is running");
|
||||
expect(result.guidance).toContain("Shell-looking output alone does not confirm that the successor has stopped; check the launch and process state before another handover.");
|
||||
});
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
|
||||
@@ -23,11 +23,14 @@ import { WatchdogJobsRepository } from "../src/domain/watchdog-jobs-repository.j
|
||||
const require = createRequire(import.meta.url);
|
||||
const relay = require("../assets/plugins/openrig-core/hooks/scripts/activity-relay.cjs") as {
|
||||
buildOpenRigPayload: (payload: Record<string, unknown>, env: Record<string, string>, now: () => Date) => Record<string, unknown>;
|
||||
claudeLimitResetAt: (text: string, now: Date) => string | null;
|
||||
};
|
||||
const SESSION = "dev-worker@limit-fixture";
|
||||
const NODE = "node-limit-fixture";
|
||||
const GENERATION = "generation-current";
|
||||
const T0 = "2025-01-01T12:00:00.000Z";
|
||||
// The seat's own Claude session: its resume token, which a hook from the seat's Claude reports as session_id.
|
||||
const OWN_CLAUDE_SESSION = "11111111-1111-4111-8111-111111111111";
|
||||
const identity = { OPENRIG_SESSION_NAME: SESSION, OPENRIG_NODE_ID: NODE, OPENRIG_RUNTIME: "claude-code", OPENRIG_OCCUPANT_GENERATION: GENERATION };
|
||||
|
||||
describe("Claude provider interruptions (#99)", () => {
|
||||
@@ -48,7 +51,8 @@ describe("Claude provider interruptions (#99)", () => {
|
||||
db.prepare("INSERT INTO rigs (id, name) VALUES ('rig-limit', 'limit-fixture')").run();
|
||||
db.prepare("INSERT INTO pods (id, rig_id, namespace, label) VALUES ('pod-limit', 'rig-limit', 'dev', 'Dev')").run();
|
||||
db.prepare("INSERT INTO nodes (id, rig_id, logical_id, runtime, pod_id) VALUES (?, 'rig-limit', 'dev.worker', 'claude-code', 'pod-limit')").run(NODE);
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status, startup_status) VALUES ('session-limit', ?, ?, 'running', 'ready')").run(NODE, SESSION);
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status, startup_status, resume_type, resume_token) VALUES ('session-limit', ?, ?, 'running', 'ready', 'claude_id', ?)")
|
||||
.run(NODE, SESSION, OWN_CLAUDE_SESSION);
|
||||
db.prepare("INSERT INTO bindings (id, node_id, tmux_session) VALUES ('binding-limit', ?, ?)").run(NODE, SESSION);
|
||||
const eventBus = new EventBus(db);
|
||||
liveGeneration = GENERATION;
|
||||
@@ -125,6 +129,124 @@ describe("Claude provider interruptions (#99)", () => {
|
||||
expect(store.getLatestForNode({ sessionName: SESSION })).toMatchObject({ state: "unknown", reason: "usage_limit" });
|
||||
});
|
||||
|
||||
describe("the reset a Claude limit states (#99)", () => {
|
||||
const WEEKLY = "You've hit your weekly limit · resets Jan 2, 4pm (America/Sao_Paulo)";
|
||||
const reportStated = (text: string, field = "last_assistant_message", sessionId: string | null = OWN_CLAUDE_SESSION) =>
|
||||
report({ hook_event_name: "StopFailure", error: "rate_limit", [field]: text, ...(sessionId ? { session_id: sessionId } : {}) });
|
||||
const latestResetsAt = () => (store.getLatestForNode({ sessionName: SESSION }) as { resetsAt?: string | null } | null)?.resetsAt;
|
||||
|
||||
it.each([
|
||||
// Real messages from a Claude Code 2.1.296 limit, and the older formats.
|
||||
[WEEKLY, "2025-01-02T19:00:00.000Z"],
|
||||
["5-hour limit reached ∙ resets 8am (UTC)", "2025-01-02T08:00:00.000Z"],
|
||||
["5-hour limit reached ∙ resets 1:30pm (UTC)", "2025-01-01T13:30:00.000Z"],
|
||||
["rate limit · resets in 2h 30m", "2025-01-01T14:30:00.000Z"],
|
||||
["Claude AI usage limit reached|1735740000", "2025-01-01T14:00:00.000Z"],
|
||||
])("reads %j", (text, expected) => {
|
||||
expect(relay.claudeLimitResetAt(text, new Date(T0))).toBe(expected);
|
||||
});
|
||||
|
||||
it.each([
|
||||
"API Error: Rate limit reached",
|
||||
"resets 3",
|
||||
"resets 4pm (Not/AZone)",
|
||||
"resets 4pm (Not/AZone", // a truncated zone is not the seat's local time
|
||||
"resets 4pm ()",
|
||||
"resets in 2 months", // not two minutes
|
||||
"resets in 2h 30x",
|
||||
"resets in 2h, 30x", // a separator doesn't end the fragment
|
||||
"resets in 2h and 30x",
|
||||
"resets in soon",
|
||||
"resets Feb 30, 4pm (UTC)",
|
||||
"resets Mar 1, 4pm (UTC)", // more than a week away: not a usage window
|
||||
"Claude AI usage limit reached|1735730000", // already past
|
||||
])("states no reset for %j", text => {
|
||||
expect(relay.claudeLimitResetAt(text, new Date(T0))).toBeNull();
|
||||
});
|
||||
|
||||
it.each([
|
||||
// Close to the impossible date, where only a calendar check (not the distance cap) can reject it.
|
||||
["2026-02-28T12:00:00.000Z", "resets Feb 30, 4pm (UTC)", null],
|
||||
["2026-02-28T12:00:00.000Z", "resets Feb 29, 4pm (UTC)", null], // 2026 is not a leap year
|
||||
["2026-02-28T12:00:00.000Z", "resets Mar 2, 4pm (UTC)", "2026-03-02T16:00:00.000Z"],
|
||||
["2028-02-28T12:00:00.000Z", "resets Feb 29, 4pm (UTC)", "2028-02-29T16:00:00.000Z"],
|
||||
["2026-04-29T12:00:00.000Z", "resets Apr 31, 4pm (UTC)", null],
|
||||
// A wall time a DST change repeats or skips names no single instant: no reset rather than one a day late.
|
||||
["2025-11-02T05:00:00.000Z", "resets 1:30am (America/New_York)", null], // 1:30 happens twice that night
|
||||
["2025-11-02T05:00:00.000Z", "resets Nov 2, 1:30am (America/New_York)", null],
|
||||
["2025-03-09T06:00:00.000Z", "resets 2:30am (America/New_York)", null], // 2:30 never happens that night
|
||||
["2025-03-09T06:00:00.000Z", "resets 3:30am (America/New_York)", "2025-03-09T07:30:00.000Z"],
|
||||
["2025-11-02T05:00:00.000Z", "resets 3am (America/New_York)", "2025-11-02T08:00:00.000Z"],
|
||||
])("at %s reads %j as %j", (now, text, expected) => {
|
||||
expect(relay.claudeLimitResetAt(text, new Date(now))).toBe(expected);
|
||||
});
|
||||
|
||||
it("relays only the instant, never the message", () => {
|
||||
const payload = reportStated(WEEKLY);
|
||||
expect(payload).toMatchObject({ subtype: "rate_limit", resetsAt: "2025-01-02T19:00:00.000Z" });
|
||||
expect(JSON.stringify(payload)).not.toContain("weekly");
|
||||
expect(reportStated(WEEKLY, "error_details")).toMatchObject({ resetsAt: "2025-01-02T19:00:00.000Z" });
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ hook_event_name: "StopFailure", error: "overloaded", last_assistant_message: WEEKLY },
|
||||
{ hook_event_name: "Stop", last_assistant_message: WEEKLY },
|
||||
{ hook_event_name: "Notification", notification_type: "rate_limit", message: WEEKLY },
|
||||
])("reads no reset off another event: %j", payload => {
|
||||
expect(report(payload)).not.toHaveProperty("resetsAt");
|
||||
});
|
||||
|
||||
it("keeps a stated reset only from the seat's own Claude session, not a helper's", async () => {
|
||||
// A helper Claude started from the seat's shell posts as the seat, but under its own session (maybe another login).
|
||||
reportStated(WEEKLY, "last_assistant_message", "22222222-2222-4222-8222-222222222222");
|
||||
expect(latestResetsAt()).toBeUndefined();
|
||||
const model = await service.getReadModel();
|
||||
expect(deriveUsageLimitPools({ signals: model.signals, bindings: model.bindings, now: new Date(), fallbackSeconds: 300 }))
|
||||
.toEqual([expect.objectContaining({ poolKey: "claude:local", source: "config-fallback",
|
||||
expiresAt: new Date(Date.parse(T0) + 300_000).toISOString() })]);
|
||||
|
||||
reportStated(WEEKLY, "last_assistant_message", null);
|
||||
expect(latestResetsAt()).toBeUndefined();
|
||||
reportStated(WEEKLY);
|
||||
expect(latestResetsAt()).toBe("2025-01-02T19:00:00.000Z");
|
||||
});
|
||||
|
||||
it("drops a stated reset past the eight-day cap from any sender", async () => {
|
||||
const result = store.recordHookEvent({ runtime: "claude-code", sessionName: SESSION, hookEvent: "StopFailure", subtype: "rate_limit",
|
||||
occurredAt: T0, generation: GENERATION, sessionId: OWN_CLAUDE_SESSION, resetsAt: "2025-01-10T12:00:00.000Z" });
|
||||
expect(result.ok && result.activity.resetsAt).toBe("2025-01-10T12:00:00.000Z");
|
||||
expect(await events()).toEqual([expect.not.objectContaining({ resetsAt: expect.anything() })]);
|
||||
});
|
||||
|
||||
it("keeps a stated reset only on the typed limit row", () => {
|
||||
const result = store.recordHookEvent({ runtime: "claude-code", sessionName: SESSION, hookEvent: "Stop",
|
||||
resetsAt: "2025-01-02T19:00:00.000Z", generation: GENERATION, sessionId: OWN_CLAUDE_SESSION });
|
||||
expect(result.ok && result.activity).not.toHaveProperty("resetsAt");
|
||||
});
|
||||
|
||||
it("parks pending work until the stated reset instead of the bounded retry", async () => {
|
||||
reportStated(WEEKLY);
|
||||
const model = await service.getReadModel();
|
||||
expect(model.signals.filter(signal => signal.sourceClass === "provider_event"))
|
||||
.toEqual([expect.objectContaining({ provider: "claude", seatSession: SESSION, resetsAt: "2025-01-02T19:00:00.000Z" })]);
|
||||
expect(deriveUsageLimitPools({ signals: model.signals, bindings: model.bindings, now: new Date(), fallbackSeconds: 300 }))
|
||||
.toEqual([{ poolKey: "claude:local", provider: "claude", seatSessions: [SESSION], expiresAt: "2025-01-02T19:00:00.000Z", source: "provider-reset" }]);
|
||||
|
||||
queue.attachWatchdogJobsRepository(new WatchdogJobsRepository(db, () => new Date()));
|
||||
queue.attachOutbox(new OutboxHandler(db));
|
||||
const source = await queue.create({ sourceSession: "sender@limit-fixture", destinationSession: "relay@limit-fixture", body: "work", nudge: false });
|
||||
const { created: baton } = await queue.handoff({ qitemId: source.qitemId, fromSession: "relay@limit-fixture", toSession: SESSION, nudge: false });
|
||||
db.prepare("UPDATE queue_items SET last_nudge_attempt = ?, last_nudge_result = 'failed:provider limit' WHERE qitem_id = ?")
|
||||
.run(new Date(Date.parse(T0) - 600_000).toISOString(), baton.qitemId);
|
||||
const result = await runWakeLadderTick({ db, queueRepo: queue, attemptWake: vi.fn(async () => "typed"), resolveOrchestrator: () => null,
|
||||
retryIntervalSeconds: 300, usageLimitJitterSeconds: 0, getProviderReadModel: () => service.getReadModel(), log: () => {} });
|
||||
expect(result.actions).toContainEqual({ qitemId: baton.qitemId, action: "park-usage-limit" });
|
||||
const blocker = queue.getById(baton.qitemId)!.blockedOn!;
|
||||
expect(queue.getParkWakeStatus(blocker)).toMatchObject({ kind: "timer", live: true, expiresAt: "2025-01-02T19:00:00.000Z" });
|
||||
expect(queue.transitionLog.listForQitem(blocker).some(row => row.transitionNote?.includes("cause=provider-reset"))).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
it("reports a fresh same-occupant limit without inventing a Claude account, usage meter or reset", async () => {
|
||||
reportLimit();
|
||||
const model = await service.getReadModel();
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { describe, it, expect, vi } from "vitest";
|
||||
import {
|
||||
detectCodexLimitBanner,
|
||||
@@ -497,6 +498,11 @@ describe("codex-limit-banner — store mapping and seat status", () => {
|
||||
it.each([
|
||||
{ name: "whitespace-only capture", frame: " \n\t\n " },
|
||||
{ name: "unrecognized non-empty capture", frame: "unclassified output" },
|
||||
{
|
||||
name: "unsupported client version on a completed layout",
|
||||
frame: " >_ OpenAI Codex (v99.0.0)\n" + readFileSync(
|
||||
new URL("./fixtures/input-box/codex-0.167.0/completed.txt", import.meta.url), "utf8"),
|
||||
},
|
||||
{ name: "banner above draft composer", frame: `${BANNER}\n› Continue the task` },
|
||||
{ name: "banner above indented empty composer", frame: `${BANNER}\n › Ask Codex to do anything` },
|
||||
])("does not clear a current limit for $name", async ({ frame }) => {
|
||||
@@ -533,6 +539,35 @@ describe("codex-limit-banner — store mapping and seat status", () => {
|
||||
} finally { db.close(); }
|
||||
});
|
||||
|
||||
it("positive work on a version-named frame clears a prior limit durably", async () => {
|
||||
const db = createFullTestDb();
|
||||
try {
|
||||
const { sessionName } = seedCodexSeat(db);
|
||||
let clock = NOW;
|
||||
const deps = { db, eventBus: new EventBus(db), now: () => clock, resolveOccupantGeneration: () => "current-generation" };
|
||||
const store = new AgentActivityStore(deps);
|
||||
const capturePaneContent = vi.fn(async () => BANNER);
|
||||
const structuralActivity = new SeatStructuralActivityService({ capturePaneContent }, () => clock);
|
||||
structuralActivity.attachCodexLimitBanner((sessionName, banner, at) => {
|
||||
recordCodexLimitBanner({ store, resolveGeneration: () => "current-generation", sessionName, banner, now: () => at });
|
||||
});
|
||||
await structuralActivity.pollSeat(sessionName, null, "codex");
|
||||
expect(store.getLatestForNode({ sessionName })).toMatchObject({ reason: "usage_limit" });
|
||||
capturePaneContent.mockResolvedValue(" >_ OpenAI Codex (v99.0.0)\n" + readFileSync(
|
||||
new URL("./fixtures/input-box/codex-0.167.0/working.txt", import.meta.url), "utf8"));
|
||||
for (let poll = 1; poll <= 2; poll++) {
|
||||
clock = new Date(NOW.getTime() + poll * 1000);
|
||||
expect((await structuralActivity.pollSeat(sessionName, null, "codex"))?.state).toBe("agent_active");
|
||||
expect(new AgentActivityStore(deps).getLatestForNode({ sessionName })).toMatchObject({ reason: "usage_limit_cleared" });
|
||||
const out = await attachAgentActivity(entries(sessionName), {
|
||||
tmuxAdapter: { capturePaneContent } as never, activityStore: store, structuralActivity, now: clock,
|
||||
});
|
||||
expect(out[0]!.agentActivity).toMatchObject({ state: "running", reason: "mid_work_pattern" });
|
||||
}
|
||||
expect(db.prepare("SELECT COUNT(*) AS n FROM events WHERE type = 'agent.activity'").get()).toEqual({ n: 2 });
|
||||
} finally { db.close(); }
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ name: "submitted prompt followed by actual work", draft: "› Continue the task\n◦ Working (2s • esc to interrupt)", limit: false },
|
||||
...["Explain this example", ""].map(first => ({ name: `unsent nested draft: ${first || "blank first line"}`,
|
||||
|
||||
@@ -57,6 +57,15 @@ it("a fresh install has all six exact plugin hooks trusted before the first seat
|
||||
expect(Object.values(states).every((state: any) => state.trusted_content_hash === codexPluginContentHash(shipped))).toBe(true);
|
||||
expect(f.calls.map(a=>a.slice(0,2))).toEqual([["plugin","marketplace"],["plugin","add"]]);
|
||||
});
|
||||
it("refocus keeps Codex's default budget and main's declaration hash", () => {
|
||||
const manifest = JSON.parse(fs.readFileSync(path.join(shipped, "hooks/codex.json"), "utf8"));
|
||||
const handler = manifest.hooks.UserPromptSubmit[1].hooks[0];
|
||||
const { computeCodexHookTrust } = createRequire(import.meta.url)(path.join(shipped, "hooks/scripts/codex-hook-trust.cjs"));
|
||||
const identity = computeCodexHookTrust("UserPromptSubmit", { ...handler, timeoutSec: handler.timeout,
|
||||
keySource: "openrig-core@openrig:hooks/codex.json", groupIndex: 1, handlerIndex: 0 });
|
||||
expect(identity.hash).toBe("sha256:c648714b3a02675ba4ea029b8e62dbcd77569afc317ad4f1bc3e8cbfbe7b34ee");
|
||||
expect(handler.additionalContextLimit).toBeUndefined();
|
||||
});
|
||||
it.each(['[plugins."openrig-core@openrig"]\nenabled = false\n','[plugins._default]\nenabled = false\n'])("a disabled plugin stays disabled on restart and upgrade: %s", async config => {
|
||||
const f = fixture(config); await f.adapter().ensureCodexCorePlugin(); await f.adapter().ensureCodexCorePlugin();
|
||||
expect(fs.readFileSync(f.file,"utf8")).toBe(config); expect(f.calls).toEqual([]);
|
||||
|
||||
@@ -68,6 +68,27 @@ describe("cursor-bound composer input", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
"✘ Auto-update failed · Run claude doctor",
|
||||
"✘ Auto-update failed: no write permission to npm prefix · Run claude doctor",
|
||||
"✘ Auto-update failed: claude.exe in use (close other Claude Code sessions, including VS Code) · Run claude doctor",
|
||||
"✘ Auto-update failed · Try claude doctor or npm i -g @anthropic-ai/claude-code",
|
||||
"✘ Auto-update failed · Try claude doctor or cd ~/.claude/local && npm update @anthropic-ai/claude-code",
|
||||
])("reads a complete auto-update failure footer row (%s), preserving draft text", warning => {
|
||||
const frame = (body: string, footer = warning) => {
|
||||
const snapshot = claude(body, undefined, 200);
|
||||
snapshot.screen = snapshot.screen.replace("? for shortcuts\n", `? for shortcuts\n${footer}\n`);
|
||||
return snapshot;
|
||||
};
|
||||
expect(inspectComposerInput(frame(""), "claude-code").state).toBe("empty");
|
||||
expect(inspectComposerInput(frame("person's draft"), "claude-code").state).toBe("text");
|
||||
expect(inspectComposerInput(claude(warning, undefined, 200), "claude-code").state).toBe("text");
|
||||
for (const nearMiss of [warning + " now", warning.replace("failed", "succeeded"), warning.replace("doctor", "doctors")]) {
|
||||
expect(inspectComposerInput(claude(nearMiss, undefined, 200), "claude-code").state).toBe("text");
|
||||
expect(inspectComposerInput(frame("", nearMiss), "claude-code").state).toBe("unknown");
|
||||
}
|
||||
});
|
||||
|
||||
it("reads Claude's context hint below a named composer without skipping arbitrary rows", () => {
|
||||
const frame = (body: string, styled = false, hint = "~66k uncached · /clear to start fresh"): ComposerSnapshot => ({
|
||||
screen: [...Array<string>(22).fill(""),
|
||||
|
||||
@@ -133,7 +133,7 @@ describe.skipIf(process.platform === "win32")("native draft-aware terminal deliv
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
it.each(["update-banner", "cursor-block", "context-hint"])("submits a multiline paste with %s and keeps a real draft through tmux", async variant => {
|
||||
it.each(["update-banner", "cursor-block", "context-hint", "auto-update-short", "auto-update-reason", "auto-update-try-global", "auto-update-try-local"])("submits a multiline paste with %s and keeps a real draft through tmux", async variant => {
|
||||
const scratch = mkdtempSync(join(tmpdir(), "draft-banner-"));
|
||||
const socket = join(scratch, "s");
|
||||
const env = { ...process.env }; delete env.TMUX; delete env.TMUX_TMPDIR;
|
||||
@@ -144,7 +144,8 @@ describe.skipIf(process.platform === "win32")("native draft-aware terminal deliv
|
||||
let transport: SessionTransport | undefined;
|
||||
try {
|
||||
const command = [process.execPath, composer, "claude", stateFile, variant].map(shellQuote).join(" ");
|
||||
const [width, height] = variant === "context-hint" ? ["68", "30"] : variant === "cursor-block" ? ["68", "31"] : ["120", "40"];
|
||||
const wide = variant === "update-banner" || (variant.startsWith("auto-update-") && variant !== "auto-update-short");
|
||||
const [width, height] = wide ? ["120", "40"] : ["68", variant === "cursor-block" ? "31" : "30"];
|
||||
const pane = (await tmux(["-f", "/dev/null", "new-session", "-d", "-P", "-F", "#{pane_id}", "-x", width, "-y", height, "-s", "worker@test", command])).trim();
|
||||
await vi.waitFor(() => expect(state().input).toBe(""), { timeout: 5000 });
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
@@ -161,13 +162,22 @@ describe.skipIf(process.platform === "win32")("native draft-aware terminal deliv
|
||||
if (variant !== "update-banner") {
|
||||
await vi.waitFor(async () => {
|
||||
const snapshot = await adapter.captureComposerSnapshot(pane);
|
||||
const row = variant === "context-hint" ? 23 : 26;
|
||||
expect(snapshot?.cursor).toEqual({ x: 2, y: row, width: 68, height: Number(height) });
|
||||
const row = variant === "cursor-block" ? 26 : 23;
|
||||
expect(snapshot?.cursor).toEqual({ x: 2, y: row, width: Number(width), height: Number(height) });
|
||||
expect(snapshot!.screen.split("\n")[row]).toContain("❯\u00a0\x1b[7m ");
|
||||
expect(snapshot!.screen.split("\n")[row - 1]).toContain("@moonjoy-workshop ─");
|
||||
const plain = await tmux(["capture-pane", "-p", "-t", pane]);
|
||||
expect(plain.split("\n")[row]).toBe("❯\u00a0");
|
||||
if (variant === "context-hint") expect(plain.split("\n")[26].trim()).toBe("~66k uncached · /clear to start fresh");
|
||||
if (variant.startsWith("auto-update")) {
|
||||
const expected: Record<string, string> = {
|
||||
"auto-update-short": "✘ Auto-update failed · Run claude doctor",
|
||||
"auto-update-reason": "✘ Auto-update failed: no write permission to npm prefix · Run claude doctor",
|
||||
"auto-update-try-global": "✘ Auto-update failed · Try claude doctor or npm i -g @anthropic-ai/claude-code",
|
||||
"auto-update-try-local": "✘ Auto-update failed · Try claude doctor or cd ~/.claude/local && npm update @anthropic-ai/claude-code",
|
||||
};
|
||||
expect(plain.split("\n")[26].trim()).toBe(expected[variant]);
|
||||
}
|
||||
expect(annotateInputBox(snapshot!.screen, snapshot!.cursor, "claude-code")).toMatchObject({ status: "empty", confidence: "high" });
|
||||
});
|
||||
} else await vi.waitFor(async () => expect(inspectComposerInput(await adapter.captureComposerSnapshot(pane), "claude-code").state).toBe("empty"));
|
||||
|
||||
@@ -24,6 +24,7 @@ import { queueItemsSchema } from "../src/db/migrations/024_queue_items.js";
|
||||
import { queueTransitionsSchema } from "../src/db/migrations/025_queue_transitions.js";
|
||||
import { queueItemSummarySchema } from "../src/db/migrations/044_queue_item_summary.js";
|
||||
import { buildExecutionView, type ExecutionViewDeps } from "../src/domain/execution-view.js";
|
||||
import { readProjectReadiness } from "../src/domain/proof/judgments.js";
|
||||
|
||||
const SHA = "a".repeat(40);
|
||||
const REGISTRY_FILES = 450;
|
||||
@@ -120,4 +121,25 @@ describe("execution view review registry", () => {
|
||||
expect(turnsAtRead[REGISTRY_FILES - 1]! - turnsAtRead[0]!).toBeGreaterThan(0);
|
||||
expect(turnsAtRead).toHaveLength(REGISTRY_FILES);
|
||||
});
|
||||
|
||||
it("also yields inside the later whole-project readiness read, preserving its projection", async () => {
|
||||
const deps = fixture("turns", 2), missions = deps.slicesRoot()!;
|
||||
for (const name of ["beta", "gamma"]) {
|
||||
fs.mkdirSync(path.join(missions, name));
|
||||
fs.writeFileSync(path.join(missions, name, "mission.yaml"), "kind: mission\ncomposition: { slices: [] }\n");
|
||||
}
|
||||
const first = fs.readdirSync(missions).slice(0, -1).find(name => name !== "turns")!;
|
||||
let started = false, yielded = false;
|
||||
hooks.onRead = file => {
|
||||
if (!started && file === path.join(missions, first, "mission.yaml")) {
|
||||
started = true;
|
||||
setImmediate(() => { yielded = true; });
|
||||
}
|
||||
};
|
||||
const doc = await buildExecutionView(deps, { mission: "turns" });
|
||||
expect(started).toBe(true);
|
||||
expect(yielded, "earlier registry yields must not hide a later synchronous project walk").toBe(true);
|
||||
hooks.onRead = null;
|
||||
expect(doc.project_readiness).toEqual(readProjectReadiness(missions));
|
||||
});
|
||||
});
|
||||
+15
-7
@@ -5,18 +5,26 @@ import { StringDecoder } from "node:string_decoder";
|
||||
import stringWidth from "string-width";
|
||||
|
||||
const [runtime, stateFile, footerVariant] = process.argv.slice(2);
|
||||
const contextHint = footerVariant === "context-hint";
|
||||
const cursorBlock = footerVariant === "cursor-block" || contextHint;
|
||||
const rule = cursorBlock ? `\x1b[38;5;244m${"─".repeat(68)}\x1b[39m` : "────────────────────────────────────────";
|
||||
const footerRows = {
|
||||
"context-hint": "~66k uncached · /clear to start fresh",
|
||||
"auto-update-short": "✘ Auto-update failed · Run claude doctor",
|
||||
"auto-update-reason": "✘ Auto-update failed: no write permission to npm prefix · Run claude doctor",
|
||||
"auto-update-try-global": "✘ Auto-update failed · Try claude doctor or npm i -g @anthropic-ai/claude-code",
|
||||
"auto-update-try-local": "✘ Auto-update failed · Try claude doctor or cd ~/.claude/local && npm update @anthropic-ai/claude-code",
|
||||
};
|
||||
const footerRow = footerRows[footerVariant];
|
||||
const cursorBlock = footerVariant === "cursor-block" || Boolean(footerRow);
|
||||
const frameWidth = footerVariant?.startsWith("auto-update-") && footerVariant !== "auto-update-short" ? 120 : 68;
|
||||
const rule = cursorBlock ? `\x1b[38;5;244m${"─".repeat(frameWidth)}\x1b[39m` : "────────────────────────────────────────";
|
||||
let input = "", pending = "", pasting = false, cursor = 0, ghost = false;
|
||||
const submissions = [];
|
||||
const decoder = new StringDecoder("utf8");
|
||||
|
||||
function render() {
|
||||
const rows = ["Editable delivery fixture", ...submissions.slice(-3).flatMap(text => text.split("\n"))];
|
||||
if (cursorBlock) while (rows.length < (contextHint ? 22 : 25)) rows.push("");
|
||||
const label = contextHint ? "dev-docs@moonjoy-workshop" : "orch-lead@moonjoy-workshop";
|
||||
const topRule = cursorBlock ? `\x1b[38;5;244m${"─".repeat(68 - stringWidth(label) - 3)} ${label} ─\x1b[39m` : rule;
|
||||
if (cursorBlock) while (rows.length < (footerRow ? 22 : 25)) rows.push("");
|
||||
const label = footerRow ? "dev-docs@moonjoy-workshop" : "orch-lead@moonjoy-workshop";
|
||||
const topRule = cursorBlock ? `\x1b[38;5;244m${"─".repeat(frameWidth - stringWidth(label) - 3)} ${label} ─\x1b[39m` : rule;
|
||||
if (runtime === "claude") rows.push(topRule);
|
||||
const start = rows.length;
|
||||
const displayed = ghost && cursor === input.length ? `${input}\x1b[2m suggested follow-up\x1b[22m`
|
||||
@@ -25,7 +33,7 @@ function render() {
|
||||
rows.push(`${runtime === "claude" ? "❯\u00a0" : "› "}${body[0]}`, ...body.slice(1).map(line => ` ${line}`));
|
||||
rows.push(...(runtime === "claude" ? [rule, cursorBlock
|
||||
? " \x1b[38;5;211m⏵⏵ bypass permissions on\x1b[38;5;246m (shift+tab to cycle)\x1b[0m" : "? for shortcuts"] : ["", "fixture · 90% context left"]));
|
||||
if (runtime === "claude" && contextHint) rows.push(" ~66k uncached · /clear to start fresh");
|
||||
if (runtime === "claude" && footerRow) rows.push(footerRow.padStart(66));
|
||||
if (runtime === "claude" && ghost) {
|
||||
rows[rows.length - 1] = "⏵⏵ accept edits on (shift+tab to cycle) · ← for agents";
|
||||
rows.push("✘ Auto-update failed: no write permission to npm prefix · Run claude doctor");
|
||||
|
||||
@@ -5,6 +5,10 @@ Real frames of Claude Code 2.1.296 (`claude-code-2.1.296/`) and Codex 0.160.0 (`
|
||||
(`claude-code-2.1.296/openrig-seat-*`, see below). `texts/` holds the texts that were sent.
|
||||
`input-box-reading.test.ts` and `session-transport-input-box.test.ts` read them.
|
||||
|
||||
`codex-0.167.0/` adds redacted completed and working tails from a managed terminal.
|
||||
Its README records their provenance and the native streaming evidence still missing.
|
||||
The activity tests use these tails; they do not establish a staged-input/send sequence.
|
||||
|
||||
## Versions and their limits
|
||||
|
||||
Every frame here is of one client version, named by its folder; the idle allow-list and the box readers were built
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
# Codex 0.167.0 activity frames
|
||||
|
||||
Captured from an OpenRig-managed Codex terminal on Linux, at 120 columns. The
|
||||
runtime reported `codex-cli 0.167.0` separately; neither captured tail displays
|
||||
the client version. These frames establish the drawn layout, not a general
|
||||
allow-list for every screen of that version.
|
||||
|
||||
- `completed.txt`: another agent captured the terminal at the completed prompt
|
||||
with `rig capture`, before waking it, on 2026-11-11. The transcript was replaced
|
||||
by a generic response, and the local path and task title were redacted. The
|
||||
completion, blank rows, composer, model and hint/warning rows are retained.
|
||||
- `working.txt`: captured during a tool turn with `rig capture`. Command text,
|
||||
output, local path and task title were redacted. The working row, tip, blank
|
||||
rows, composer and footer are retained.
|
||||
|
||||
No spinner-hidden streaming frame was captured. Tests that replace the completion
|
||||
row with unfinished response text are synthetic controls, not native captures.
|
||||
No fresh-session header, staged input or submit sequence is represented here.
|
||||
@@ -0,0 +1,10 @@
|
||||
• Finished the task.
|
||||
|
||||
Worked for 50s • 12:32 AM
|
||||
|
||||
|
||||
› Ask Codex to do anything
|
||||
|
||||
GPT-6.1-Sol high · ~/example/work · Finish task
|
||||
← for agents · ? for shortcuts ⚠ 1 warning · f2 to view
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
• Ran a command
|
||||
└ Command output
|
||||
|
||||
• Working (4m 34s • esc to interrupt) · 1 background terminal running · /ps to view · /stop to close
|
||||
└ Tip: Use /feedback to send logs to the maintainers when something looks off.
|
||||
|
||||
|
||||
› Ask Codex to do anything
|
||||
|
||||
GPT-6.1-Sol high · ~/example/work · Finish task
|
||||
← for agents · ? for shortcuts ⚠ 1 warning · f2 to view
|
||||
|
||||
@@ -4,6 +4,8 @@ import { queueErasureSchema } from "../../src/db/migrations/155_queue_erasure.js
|
||||
import { queueErasureBatchesSchema } from "../../src/db/migrations/157_queue_erasure_batches.js";
|
||||
import { queueInboxCheckinsSchema } from "../../src/db/migrations/158_queue_inbox_checkins.js";
|
||||
import { queueItemsHandedOffFromIndexSchema } from "../../src/db/migrations/159_queue_items_handed_off_from_index.js";
|
||||
import { networkBindingsSchema } from "../../src/db/migrations/160_network_bindings.js";
|
||||
import { networkReceiptsSchema } from "../../src/db/migrations/161_network_receipts.js";
|
||||
import { activityGrantsSchema } from "../../src/db/migrations/143_activity_grants.js";
|
||||
import { environmentBindingsSchema } from "../../src/db/migrations/144_environment_bindings.js";
|
||||
import { nodePermissionsToSchema } from "../../src/db/migrations/145_node_permissions_to.js";
|
||||
@@ -166,7 +168,7 @@ import fs from "node:fs";
|
||||
|
||||
/** Seam B R6: the canonical full-fixture migration list, exported so file-backed
|
||||
* DB-reopen tests migrate IDENTICALLY to createFullTestDb. */
|
||||
export const migrationsForFullTestDb = [coreSchema, bindingsSessionsSchema, eventsSchema, snapshotsSchema, checkpointsSchema, resumeMetadataSchema, nodeSpecFieldsSchema, packagesSchema, installJournalSchema, journalSeqSchema, bootstrapSchema, discoverySchema, discoveryFkFix, agentspecRebootSchema, startupContextSchema, chatMessagesSchema, podNamespaceSchema, contextUsageSchema, externalCliAttachmentSchema, rigServicesSchema, seatHandoverObservabilitySchema, nodeCodexConfigProfileSchema, nodePermissionPolicySchema, rigPermissionPolicySchema, nodePolicyProvenanceSchema, rigPolicyProvenanceSchema, streamItemsSchema, queueItemsSchema, queueTransitionsSchema, rigPolicySchema, rigArchiveSchema, resumeProvenanceSchema, resumeVerificationSchema, seatIdentityVerdictsSchema, selfHostIdentitySchema, occupantTenuresSchema, daemonLifecycleSchema, watchdogJobsSchema, occupantGenerationStampsSchema, projectionManifestSchema, watchdogTargetGenerationSchema, appliedLaunchObservationsSchema, appliedLaunchObservationInvalidationsSchema, threadSeatMapSchema, queueTransitionWakesSchema, nodeSessionSourceSchema, scopedOperatingPostureSchema, humanNotificationIntentSchema, reviewReadIndexesSchema, inventoryEventIndexesSchema, rigClaudeManagedBlockFileSchema, nodePermissionSelectionsSchema, humanReplyToSchema, humanQuestionsSchema, nodeEffortSchema, rigNonInterruptiveSchema, rigInstallRootSchema, threadPartMapSchema, queueTransitionsQitemIdOrderSchema, queueItemsRecoveryTagsSchema, resumeRotationSchema, sessionStopIntentSchema, nodeEffortChoiceSchema, modePostureCoexistSchema, seatEnvSchema, appliedLaunchDaemonOptOutSchema, sessionCommunityHarnessSchema, threadSeatMapChannelKeySchema, queueItemWakeInputBoxSchema, providerAccountReadingsSchema, humanQuietWindowsSchema, seatIdentityPanePidSchema, seatIdentityProcessEvidenceSchema, rigStartupLayersSchema, humanOperatingModesSchema, queueDelegatedDecisionSchema, nodeAdvisorSchema, operationCorrelationSchema, terminalPolicySchema, seatRuntimeSwitchesSchema, claudeTokenReadingsSchema, nodeLaunchRecipeSchema, nodeClaudeTokenRefSchema, nodeClaudePermissionsSchema, nodeCodexPermissionsSchema, escalationTargetRefsSchema, sessionAgentRunnerSchema, nodeAccountHomeSchema, nodeClaudeMcpSchema, activityGrantsSchema, environmentBindingsSchema, nodePermissionsToSchema, nodeStartupGuidanceFileSchema, managedBlockWritesSchema, streamIntakeCursorSchema, nodeLaunchWrapperSchema, piAccountSwitchesSchema, eventsRetentionMarksSchema, queueErasureSchema, queueErasureBatchesSchema, queueInboxCheckinsSchema, queueItemsHandedOffFromIndexSchema];
|
||||
export const migrationsForFullTestDb = [coreSchema, bindingsSessionsSchema, eventsSchema, snapshotsSchema, checkpointsSchema, resumeMetadataSchema, nodeSpecFieldsSchema, packagesSchema, installJournalSchema, journalSeqSchema, bootstrapSchema, discoverySchema, discoveryFkFix, agentspecRebootSchema, startupContextSchema, chatMessagesSchema, podNamespaceSchema, contextUsageSchema, externalCliAttachmentSchema, rigServicesSchema, seatHandoverObservabilitySchema, nodeCodexConfigProfileSchema, nodePermissionPolicySchema, rigPermissionPolicySchema, nodePolicyProvenanceSchema, rigPolicyProvenanceSchema, streamItemsSchema, queueItemsSchema, queueTransitionsSchema, rigPolicySchema, rigArchiveSchema, resumeProvenanceSchema, resumeVerificationSchema, seatIdentityVerdictsSchema, selfHostIdentitySchema, occupantTenuresSchema, daemonLifecycleSchema, watchdogJobsSchema, occupantGenerationStampsSchema, projectionManifestSchema, watchdogTargetGenerationSchema, appliedLaunchObservationsSchema, appliedLaunchObservationInvalidationsSchema, threadSeatMapSchema, queueTransitionWakesSchema, nodeSessionSourceSchema, scopedOperatingPostureSchema, humanNotificationIntentSchema, reviewReadIndexesSchema, inventoryEventIndexesSchema, rigClaudeManagedBlockFileSchema, nodePermissionSelectionsSchema, humanReplyToSchema, humanQuestionsSchema, nodeEffortSchema, rigNonInterruptiveSchema, rigInstallRootSchema, threadPartMapSchema, queueTransitionsQitemIdOrderSchema, queueItemsRecoveryTagsSchema, resumeRotationSchema, sessionStopIntentSchema, nodeEffortChoiceSchema, modePostureCoexistSchema, seatEnvSchema, appliedLaunchDaemonOptOutSchema, sessionCommunityHarnessSchema, threadSeatMapChannelKeySchema, queueItemWakeInputBoxSchema, providerAccountReadingsSchema, humanQuietWindowsSchema, seatIdentityPanePidSchema, seatIdentityProcessEvidenceSchema, rigStartupLayersSchema, humanOperatingModesSchema, queueDelegatedDecisionSchema, nodeAdvisorSchema, operationCorrelationSchema, terminalPolicySchema, seatRuntimeSwitchesSchema, claudeTokenReadingsSchema, nodeLaunchRecipeSchema, nodeClaudeTokenRefSchema, nodeClaudePermissionsSchema, nodeCodexPermissionsSchema, escalationTargetRefsSchema, sessionAgentRunnerSchema, nodeAccountHomeSchema, nodeClaudeMcpSchema, activityGrantsSchema, environmentBindingsSchema, nodePermissionsToSchema, nodeStartupGuidanceFileSchema, managedBlockWritesSchema, streamIntakeCursorSchema, nodeLaunchWrapperSchema, piAccountSwitchesSchema, eventsRetentionMarksSchema, queueErasureSchema, queueErasureBatchesSchema, queueInboxCheckinsSchema, queueItemsHandedOffFromIndexSchema, networkBindingsSchema, networkReceiptsSchema];
|
||||
|
||||
/**
|
||||
* P24 — the DECLARED exclusions for {@link migrationsForFullTestDb}. That list is deliberately a
|
||||
|
||||
@@ -151,26 +151,9 @@ describe("POST /api/hosts/add — the narrow named add seam", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("GET /api/hosts dials where ssh resolves an ssh host (alias and port), and reads unknown behind a ProxyJump", async () => {
|
||||
const server = net.createServer((sock) => sock.destroy());
|
||||
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
|
||||
const bin = mkdtempSync(join(tmpdir(), "hosts-ssh-bin-"));
|
||||
const savedPath = process.env["PATH"];
|
||||
try {
|
||||
const { port } = server.address() as net.AddressInfo;
|
||||
// A stand-in `ssh -G`: the alias resolves to this listener; the other host sits behind a jump.
|
||||
writeFileSync(join(bin, "ssh"), `#!/bin/sh\ncase "$*" in\n *devbox*) printf 'hostname 127.0.0.1\\nport ${port}\\n' ;;\n *) printf 'hostname jumpy\\nport 22\\nproxyjump bastion\\n' ;;\nesac\n`, { mode: 0o755 });
|
||||
process.env["PATH"] = `${bin}:${savedPath}`;
|
||||
writeFileSync(join(home, "hosts.yaml"), "hosts:\n - id: devbox\n transport: ssh\n target: devbox\n - id: jumpy\n transport: ssh\n target: jumpy\n");
|
||||
const res = await app.request("/api/hosts", { headers: auth });
|
||||
const body = (await res.json()) as { hosts: Array<{ id: string; status: string }> };
|
||||
expect(body.hosts.map((h) => [h.id, h.status])).toEqual([["devbox", "reachable"], ["jumpy", "unknown"]]);
|
||||
} finally {
|
||||
process.env["PATH"] = savedPath;
|
||||
rmSync(bin, { recursive: true, force: true });
|
||||
await new Promise((resolve) => server.close(resolve));
|
||||
}
|
||||
});
|
||||
// hosts-list-ssh-probe-inert.test.ts pins that passive SSH rows start no
|
||||
// process and open no socket. Keep excluding a loopback characterization that
|
||||
// marks a directly resolved SSH host as reachable.
|
||||
|
||||
it("writes a valid entry through the writer twin and the CLI-visible registry file", async () => {
|
||||
const res = await app.request("/api/hosts/add", {
|
||||
@@ -309,17 +292,38 @@ describe("POST /api/hosts/add — the narrow named add seam", () => {
|
||||
});
|
||||
|
||||
it("POST /api/hosts/remove deletes only the registry entry, returns previous entry", async () => {
|
||||
await app.request("/api/hosts/add", { method: "POST", headers: auth, body: JSON.stringify({ id: "vm-a", transport: "ssh", target: "vm-a.local" }) });
|
||||
await app.request("/api/hosts/add", { method: "POST", headers: auth, body: JSON.stringify({ id: "vm-b", transport: "http", url: "http://vm-b" }) });
|
||||
const probeServer = net.createServer((socket) => socket.destroy());
|
||||
try {
|
||||
const port = await new Promise<number>((resolve, reject) => {
|
||||
probeServer.once("error", reject);
|
||||
probeServer.listen(0, "127.0.0.1", () => {
|
||||
const address = probeServer.address();
|
||||
if (!address || typeof address === "string") {
|
||||
reject(new Error("loopback probe fixture did not expose an address"));
|
||||
return;
|
||||
}
|
||||
resolve(address.port);
|
||||
});
|
||||
});
|
||||
|
||||
const res = await app.request("/api/hosts/remove", { method: "POST", headers: auth, body: JSON.stringify({ id: "vm-a" }) });
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json() as { entry: { id: string } };
|
||||
expect(body.entry).toMatchObject({ id: "vm-a", transport: "ssh" });
|
||||
await app.request("/api/hosts/add", { method: "POST", headers: auth, body: JSON.stringify({ id: "vm-a", transport: "ssh", target: "vm-a.local" }) });
|
||||
await app.request("/api/hosts/add", { method: "POST", headers: auth, body: JSON.stringify({ id: "vm-b", transport: "http", url: `http://127.0.0.1:${port}` }) });
|
||||
|
||||
const loaded = await app.request("/api/hosts", { headers: auth });
|
||||
const body2 = await loaded.json() as { hosts: Array<{ id: string }> };
|
||||
expect(body2.hosts.map(h => h.id)).toEqual(["vm-b"]);
|
||||
const res = await app.request("/api/hosts/remove", { method: "POST", headers: auth, body: JSON.stringify({ id: "vm-a" }) });
|
||||
expect(res.status).toBe(200);
|
||||
const body = await res.json() as { entry: { id: string } };
|
||||
expect(body.entry).toMatchObject({ id: "vm-a", transport: "ssh" });
|
||||
|
||||
const loaded = await app.request("/api/hosts", { headers: auth });
|
||||
const body2 = await loaded.json() as { hosts: Array<{ id: string }> };
|
||||
expect(body2.hosts.map(h => h.id)).toEqual(["vm-b"]);
|
||||
} finally {
|
||||
if (probeServer.listening) {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
probeServer.close((error) => error ? reject(error) : resolve());
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("POST /api/hosts/remove refuses an unknown id", async () => {
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { EventEmitter } from "node:events";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { Hono } from "hono";
|
||||
import { hostsRoutes } from "../src/routes/hosts.js";
|
||||
|
||||
const { execFileMock, connectMock } = vi.hoisted(() => ({
|
||||
execFileMock: vi.fn(),
|
||||
connectMock: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("node:child_process", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("node:child_process")>();
|
||||
return { ...actual, execFile: execFileMock };
|
||||
});
|
||||
vi.mock("node:net", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("node:net")>();
|
||||
return { ...actual, connect: connectMock };
|
||||
});
|
||||
|
||||
type SocketEvent = "connect" | "timeout" | "error";
|
||||
interface SocketTrace {
|
||||
events: string[];
|
||||
writes: number;
|
||||
ends: number;
|
||||
destroys: number;
|
||||
}
|
||||
interface ProbeTrace {
|
||||
execFileCalls: Array<{ command: string; args: string[]; timeout: number | undefined }>;
|
||||
connects: Array<{ host: string | undefined; port: number | undefined; timeout: number | undefined }>;
|
||||
sockets: SocketTrace[];
|
||||
}
|
||||
|
||||
class FakeSocket extends EventEmitter {
|
||||
constructor(private readonly trace: SocketTrace) {
|
||||
super();
|
||||
}
|
||||
|
||||
destroy(): this {
|
||||
this.trace.events.push("destroy");
|
||||
this.trace.destroys += 1;
|
||||
return this;
|
||||
}
|
||||
|
||||
write(..._args: unknown[]): boolean {
|
||||
this.trace.writes += 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
end(..._args: unknown[]): this {
|
||||
this.trace.ends += 1;
|
||||
return this;
|
||||
}
|
||||
}
|
||||
|
||||
let home: string;
|
||||
let savedHome: string | undefined;
|
||||
let trace: ProbeTrace;
|
||||
let socketEvent: SocketEvent;
|
||||
|
||||
function app(): Hono {
|
||||
const result = new Hono();
|
||||
result.route("/api/hosts", hostsRoutes());
|
||||
return result;
|
||||
}
|
||||
|
||||
function writeRegistry(hosts: unknown[]): void {
|
||||
writeFileSync(join(home, "hosts.yaml"), JSON.stringify({ hosts }));
|
||||
}
|
||||
|
||||
function singleStatus(body: { hosts: Array<{ status: string }> }): string | undefined {
|
||||
return body.hosts[0]?.status;
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
home = mkdtempSync(join(tmpdir(), "ssh-list-inert-"));
|
||||
savedHome = process.env["OPENRIG_HOME"];
|
||||
process.env["OPENRIG_HOME"] = home;
|
||||
trace = { execFileCalls: [], connects: [], sockets: [] };
|
||||
socketEvent = "connect";
|
||||
|
||||
execFileMock.mockReset();
|
||||
execFileMock.mockImplementation((
|
||||
command: string,
|
||||
args: string[],
|
||||
options: { timeout?: number },
|
||||
callback: (error: Error | null, stdout: string) => void,
|
||||
) => {
|
||||
trace.execFileCalls.push({ command, args: [...args], timeout: options.timeout });
|
||||
queueMicrotask(() => callback(null, ""));
|
||||
});
|
||||
|
||||
connectMock.mockReset();
|
||||
connectMock.mockImplementation((options: { host?: string; port?: number; timeout?: number }) => {
|
||||
trace.connects.push({ host: options.host, port: options.port, timeout: options.timeout });
|
||||
const lifecycle: SocketTrace = { events: [], writes: 0, ends: 0, destroys: 0 };
|
||||
trace.sockets.push(lifecycle);
|
||||
const socket = new FakeSocket(lifecycle);
|
||||
queueMicrotask(() => {
|
||||
lifecycle.events.push(socketEvent);
|
||||
socket.emit(socketEvent, ...(socketEvent === "error" ? [new Error("inert socket error")] : []));
|
||||
});
|
||||
return socket;
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
if (savedHome === undefined) delete process.env["OPENRIG_HOME"];
|
||||
else process.env["OPENRIG_HOME"] = savedHome;
|
||||
});
|
||||
|
||||
describe("GET /api/hosts status probes at inert external boundaries", () => {
|
||||
it("reports SSH entries unknown without invoking a process or opening a TCP socket", async () => {
|
||||
writeRegistry([
|
||||
{ id: "devbox", transport: "ssh", target: "devbox", user: "ops" },
|
||||
{ id: "another", transport: "ssh", target: "host.example" },
|
||||
]);
|
||||
|
||||
const response = await app().request("/api/hosts");
|
||||
const body = await response.json() as { hosts: Array<{ status: string }> };
|
||||
const observed = {
|
||||
httpStatus: response.status,
|
||||
statuses: body.hosts.map((host) => host.status),
|
||||
execFileCalls: trace.execFileCalls,
|
||||
connects: trace.connects,
|
||||
sockets: trace.sockets,
|
||||
};
|
||||
|
||||
expect(observed).toEqual({
|
||||
httpStatus: 200,
|
||||
statuses: ["unknown", "unknown"],
|
||||
execFileCalls: [],
|
||||
connects: [],
|
||||
sockets: [],
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ label: "HTTP default port", url: "http://example.test", host: "example.test", port: 80 },
|
||||
{ label: "HTTP explicit port", url: "http://example.test:7433", host: "example.test", port: 7433 },
|
||||
{ label: "HTTPS default port", url: "https://example.test", host: "example.test", port: 443 },
|
||||
{ label: "bracketed IPv6 HTTPS", url: "https://[2001:db8::7]:9443", host: "2001:db8::7", port: 9443 },
|
||||
])("preserves $label TCP status behavior using only the fake socket", async ({ url, host, port }) => {
|
||||
writeRegistry([{ id: "web", transport: "http", url }]);
|
||||
|
||||
const response = await app().request("/api/hosts");
|
||||
const body = await response.json() as { hosts: Array<{ status: string }> };
|
||||
|
||||
expect({
|
||||
httpStatus: response.status,
|
||||
status: singleStatus(body),
|
||||
execFileCalls: trace.execFileCalls,
|
||||
connects: trace.connects,
|
||||
sockets: trace.sockets,
|
||||
}).toEqual({
|
||||
httpStatus: 200,
|
||||
status: "reachable",
|
||||
execFileCalls: [],
|
||||
connects: [{ host, port, timeout: 1500 }],
|
||||
sockets: [{ events: ["connect", "destroy"], writes: 0, ends: 0, destroys: 1 }],
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["timeout", "error"] as const)("keeps HTTP %s results unreachable and destroys the fake socket", async (event) => {
|
||||
socketEvent = event;
|
||||
writeRegistry([{ id: "web", transport: "http", url: "http://example.test:7433" }]);
|
||||
|
||||
const response = await app().request("/api/hosts");
|
||||
const body = await response.json() as { hosts: Array<{ status: string }> };
|
||||
|
||||
expect({ status: singleStatus(body), sockets: trace.sockets }).toEqual({
|
||||
status: "unreachable",
|
||||
sockets: [{ events: [event, "destroy"], writes: 0, ends: 0, destroys: 1 }],
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,51 @@
|
||||
// A bind whose credential write fails partway leaves neither a credential nor a binding behind.
|
||||
import { afterEach, beforeEach, expect, it, vi } from "vitest";
|
||||
|
||||
const failWrite = { on: false };
|
||||
vi.mock("node:fs", async (importOriginal) => {
|
||||
const real = await importOriginal<typeof import("node:fs")>();
|
||||
return {
|
||||
...real,
|
||||
writeFileSync: (...args: Parameters<typeof real.writeFileSync>) => {
|
||||
// A partial write: some bytes reach the file, then the disk is full.
|
||||
if (failWrite.on && typeof args[0] === "number") {
|
||||
real.writeSync(args[0], "{\"version\":1,");
|
||||
throw Object.assign(new Error("ENOSPC: no space left on device, write"), { code: "ENOSPC" });
|
||||
}
|
||||
return real.writeFileSync(...args);
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
const { default: Database } = await import("better-sqlite3");
|
||||
const { existsSync, mkdtempSync, rmSync } = await import("node:fs");
|
||||
const { tmpdir } = await import("node:os");
|
||||
const { join } = await import("node:path");
|
||||
const { ALL_MIGRATIONS } = await import("../src/db/all-migrations.js");
|
||||
const { migrate } = await import("../src/db/migrate.js");
|
||||
const { RigRepository } = await import("../src/domain/rig-repository.js");
|
||||
const { NetworkBindingStore, networkCredentialFile } = await import("../src/domain/network/network-bindings.js");
|
||||
|
||||
let db: InstanceType<typeof Database>;
|
||||
let home: string;
|
||||
beforeEach(() => {
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
const rigs = new RigRepository(db);
|
||||
const node = rigs.addNode(rigs.createRig("lab").id, "impl", { runtime: "claude-code" });
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES ('s1', ?, 'a@lab', 'running')").run(node.id);
|
||||
home = mkdtempSync(join(tmpdir(), "openrig-network-write-"));
|
||||
});
|
||||
afterEach(() => { failWrite.on = false; db.close(); rmSync(home, { recursive: true, force: true }); });
|
||||
|
||||
it("removes the partly written credential and its folder, and records no binding", () => {
|
||||
const id = "netb-00000000000000000003";
|
||||
const store = new NetworkBindingStore(db, home, () => new Date(), () => id);
|
||||
failWrite.on = true;
|
||||
expect(() => store.create({ session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok" })).toThrow(/ENOSPC/);
|
||||
expect(existsSync(networkCredentialFile(home, id))).toBe(false);
|
||||
expect(existsSync(join(networkCredentialFile(home, id), ".."))).toBe(false);
|
||||
expect(store.list()).toHaveLength(0);
|
||||
failWrite.on = false;
|
||||
expect(store.create({ session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok" }).state).toBe("active");
|
||||
});
|
||||
@@ -0,0 +1,260 @@
|
||||
// `rig network` bindings: a selected local seat joined to one hub participant. The credential lives in a private file
|
||||
// and never comes back over the API; disconnect stops the binding and removes the credential, and local work stays.
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import Database from "better-sqlite3";
|
||||
import { Hono } from "hono";
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync, statSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { ALL_MIGRATIONS } from "../src/db/all-migrations.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { RigRepository } from "../src/domain/rig-repository.js";
|
||||
import { NetworkBindingError, NetworkBindingStore, networkCredentialFile, readNetworkCredential } from "../src/domain/network/network-bindings.js";
|
||||
import { networkRoutes } from "../src/routes/network.js";
|
||||
|
||||
describe("network bindings", () => {
|
||||
let db: Database.Database;
|
||||
let rigs: RigRepository;
|
||||
let home: string;
|
||||
let store: NetworkBindingStore;
|
||||
let seq = 0;
|
||||
beforeEach(() => {
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
rigs = new RigRepository(db);
|
||||
home = mkdtempSync(join(tmpdir(), "openrig-network-"));
|
||||
store = new NetworkBindingStore(db, home);
|
||||
});
|
||||
afterEach(() => { db.close(); rmSync(home, { recursive: true, force: true }); });
|
||||
|
||||
function seat(session: string) {
|
||||
const rig = rigs.listRigs().find((r) => r.name === "lab") ?? rigs.createRig("lab");
|
||||
const node = rigs.addNode(rig.id, `n${++seq}`, { runtime: "claude-code" });
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES (?, ?, ?, 'running')").run(`s${seq}`, node.id, session);
|
||||
return node;
|
||||
}
|
||||
const connect = (session: string, over: Record<string, unknown> = {}) =>
|
||||
store.create({ session, hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok-123", ...over });
|
||||
|
||||
it("binds a local seat by its node, keeps the token in a private file, and never lists it", () => {
|
||||
const node = seat("dev-impl@lab");
|
||||
const binding = connect("dev-impl@lab");
|
||||
expect(binding).toMatchObject({ nodeId: node.id, localSession: "dev-impl@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team",
|
||||
state: "active", lastContactAt: null });
|
||||
expect(binding.id).toMatch(/^netb-[0-9a-f]{20}$/);
|
||||
const file = networkCredentialFile(home, binding.id);
|
||||
expect(statSync(file).mode & 0o777).toBe(0o600);
|
||||
expect(statSync(join(file, "..")).mode & 0o777).toBe(0o700);
|
||||
expect(readNetworkCredential(home, binding.id)).toBe("tok-123");
|
||||
expect(JSON.stringify(store.list())).not.toContain("tok-123");
|
||||
});
|
||||
|
||||
it("refuses an unknown seat, a second binding for a seat or a participant, and a credential in the URL", () => {
|
||||
expect(() => connect("nobody@lab")).toThrow(expect.objectContaining({ code: "seat_not_found", status: 404 }));
|
||||
seat("a@lab"); seat("b@lab");
|
||||
connect("a@lab");
|
||||
expect(() => connect("a@lab", { hubSeat: "other@team" })).toThrow(expect.objectContaining({ code: "seat_already_connected", status: 409 }));
|
||||
expect(() => connect("b@lab")).toThrow(expect.objectContaining({ code: "participant_already_connected", status: 409 }));
|
||||
expect(() => connect("b@lab", { hubSeat: "c@team", hubUrl: "https://user:[email protected]/mcp" })).toThrow(expect.objectContaining({ code: "invalid_hub_url" }));
|
||||
expect(() => connect("b@lab", { hubSeat: "c@team", token: "two words" })).toThrow(expect.objectContaining({ code: "invalid_token" }));
|
||||
expect(() => connect("b@lab", { hubSeat: "not-an-address" })).toThrow(expect.objectContaining({ code: "invalid_hub_seat" }));
|
||||
// Every refusal above left no credential behind.
|
||||
expect(store.list()).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("plain http needs the person's choice: refused with the remedy, then recorded and marked unencrypted with --allow-http", () => {
|
||||
seat("a@lab");
|
||||
const http = { hubUrl: "http://hub.internal:7433/mcp" };
|
||||
let message = "";
|
||||
try { connect("a@lab", http); } catch (error) {
|
||||
expect(error).toMatchObject({ code: "http_not_allowed", status: 400 });
|
||||
message = (error as Error).message;
|
||||
}
|
||||
expect(message).toContain("unencrypted");
|
||||
expect(message).toContain("https URL");
|
||||
expect(message).toContain("--allow-http");
|
||||
expect(store.list()).toHaveLength(0);
|
||||
expect(existsSync(join(home, "network-secrets"))).toBe(false);
|
||||
|
||||
const binding = connect("a@lab", { ...http, allowHttp: true });
|
||||
// The scheme is kept as given, never switched.
|
||||
expect(binding).toMatchObject({ hubUrl: "http://hub.internal:7433/mcp", allowHttp: true, unencrypted: true });
|
||||
expect(store.list()).toEqual([expect.objectContaining({ id: binding.id, allowHttp: true, unencrypted: true })]);
|
||||
expect(JSON.stringify(store.list())).not.toContain("tok-123");
|
||||
// https needs no choice and isn't marked.
|
||||
seat("b@lab");
|
||||
expect(connect("b@lab", { hubSeat: "b@team" })).toMatchObject({ hubUrl: "https://hub.example/mcp", allowHttp: false, unencrypted: false });
|
||||
});
|
||||
|
||||
it("disconnect ends the binding by id or by seat, removes the credential, and frees the seat to connect again", () => {
|
||||
seat("a@lab");
|
||||
const first = connect("a@lab");
|
||||
const ended = store.disconnect("a@lab", "moving to another account");
|
||||
expect(ended).toMatchObject({ id: first.id, state: "disconnected", stateReason: "moving to another account" });
|
||||
expect(existsSync(networkCredentialFile(home, first.id))).toBe(false);
|
||||
expect(() => readNetworkCredential(home, first.id)).toThrow(expect.objectContaining({ code: "credential_missing" }));
|
||||
// A second disconnect is the same answer, not an error.
|
||||
expect(store.disconnect(first.id).state).toBe("disconnected");
|
||||
const again = connect("a@lab");
|
||||
expect(again.id).not.toBe(first.id);
|
||||
expect(store.active().map((b) => b.id)).toEqual([again.id]);
|
||||
// Two binds in the same millisecond list in id order, so compare by id.
|
||||
expect(Object.fromEntries(store.list().map((b) => [b.id, b.state]))).toEqual({ [first.id]: "disconnected", [again.id]: "active" });
|
||||
});
|
||||
|
||||
it("a revoked binding is ended the same way, and contact and errors are recorded on the active one", () => {
|
||||
seat("a@lab");
|
||||
const binding = connect("a@lab");
|
||||
store.recordError(binding.id, "hub answered 503");
|
||||
expect(store.get(binding.id)!.lastError).toBe("hub answered 503");
|
||||
// Hub text can carry terminal escapes; status prints it, so control characters are replaced.
|
||||
store.recordError(binding.id, "hub said \u001b[31mred\u0007");
|
||||
expect(store.get(binding.id)!.lastError).toBe("hub said [31mred ");
|
||||
store.recordContact(binding.id, new Date("2026-11-11T18:30:00.000Z"));
|
||||
expect(store.get(binding.id)).toMatchObject({ lastContactAt: "2026-11-11T18:30:00.000Z", lastError: null });
|
||||
expect(store.markRevoked(binding.id, "the hub refused the credential")).toMatchObject({ state: "revoked", stateReason: "the hub refused the credential" });
|
||||
expect(existsSync(networkCredentialFile(home, binding.id))).toBe(false);
|
||||
});
|
||||
|
||||
it("refuses a credential file that others can read", async () => {
|
||||
seat("a@lab");
|
||||
const binding = connect("a@lab");
|
||||
const { chmodSync } = await import("node:fs");
|
||||
chmodSync(networkCredentialFile(home, binding.id), 0o644);
|
||||
expect(() => readNetworkCredential(home, binding.id)).toThrow(NetworkBindingError);
|
||||
expect(readFileSync(networkCredentialFile(home, binding.id), "utf8")).toContain("tok-123");
|
||||
});
|
||||
|
||||
it("a refused bind removes only what it created: a file already at the path stays, and a failed insert takes its credential with it", async () => {
|
||||
const { mkdirSync, writeFileSync } = await import("node:fs");
|
||||
seat("a@lab");
|
||||
const fixed = "netb-00000000000000000001";
|
||||
const pinned = new NetworkBindingStore(db, home, () => new Date(), () => fixed);
|
||||
const path = networkCredentialFile(home, fixed);
|
||||
mkdirSync(join(path, ".."), { recursive: true, mode: 0o700 });
|
||||
writeFileSync(path, "someone else's file", { mode: 0o600 });
|
||||
expect(() => pinned.create({ session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok" })).toThrow(/EEXIST/);
|
||||
expect(readFileSync(path, "utf8")).toBe("someone else's file");
|
||||
expect(store.list()).toHaveLength(0);
|
||||
|
||||
db.exec("CREATE TRIGGER refuse_bind BEFORE INSERT ON network_bindings BEGIN SELECT RAISE(ABORT, 'refused for the test'); END;");
|
||||
const other = new NetworkBindingStore(db, home, () => new Date(), () => "netb-00000000000000000002");
|
||||
expect(() => other.create({ session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok" })).toThrow(/refused for the test/);
|
||||
expect(existsSync(networkCredentialFile(home, "netb-00000000000000000002"))).toBe(false);
|
||||
expect(store.list()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("a disconnect whose credential can't be removed still ends the binding, says so, and the retry it names removes it", async () => {
|
||||
const { chmodSync } = await import("node:fs");
|
||||
seat("a@lab");
|
||||
const binding = connect("a@lab");
|
||||
const secrets = join(home, "network-secrets");
|
||||
chmodSync(secrets, 0o500);
|
||||
let message = "";
|
||||
try {
|
||||
try { store.disconnect("a@lab"); } catch (error) {
|
||||
expect(error).toMatchObject({ code: "credential_cleanup_failed", status: 500 });
|
||||
message = (error as Error).message;
|
||||
}
|
||||
expect(store.get(binding.id)!.state).toBe("disconnected");
|
||||
expect(store.active()).toHaveLength(0);
|
||||
} finally {
|
||||
chmodSync(secrets, 0o700);
|
||||
}
|
||||
// The seat's name no longer finds an ended binding, so the message names the id to retry with.
|
||||
expect(() => store.disconnect("a@lab")).toThrow(expect.objectContaining({ code: "binding_not_found" }));
|
||||
const retry = /`rig network disconnect (\S+)`/.exec(message)?.[1];
|
||||
expect(retry).toBe(binding.id);
|
||||
expect(store.disconnect(retry!).state).toBe("disconnected");
|
||||
expect(existsSync(networkCredentialFile(home, binding.id))).toBe(false);
|
||||
});
|
||||
|
||||
it("a failed bind into a folder that was already there removes only its own credential file", async () => {
|
||||
const { mkdirSync, readFileSync: read, writeFileSync } = await import("node:fs");
|
||||
seat("a@lab");
|
||||
const id = "netb-00000000000000000004";
|
||||
const path = networkCredentialFile(home, id);
|
||||
mkdirSync(join(path, ".."), { recursive: true, mode: 0o700 });
|
||||
writeFileSync(join(path, "..", "other.txt"), "keep me");
|
||||
db.exec("CREATE TRIGGER refuse_bind BEFORE INSERT ON network_bindings BEGIN SELECT RAISE(ABORT, 'refused for the test'); END;");
|
||||
const pinned = new NetworkBindingStore(db, home, () => new Date(), () => id);
|
||||
expect(() => pinned.create({ session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok" })).toThrow(/refused for the test/);
|
||||
expect(existsSync(path)).toBe(false);
|
||||
expect(read(join(path, "..", "other.txt"), "utf8")).toBe("keep me");
|
||||
});
|
||||
|
||||
it("the credential sweep removes folders whose binding isn't active and leaves the active one and other names", async () => {
|
||||
const { mkdirSync, writeFileSync } = await import("node:fs");
|
||||
seat("a@lab"); seat("b@lab");
|
||||
const kept = connect("a@lab");
|
||||
const ended = connect("b@lab", { hubSeat: "b@team" });
|
||||
store.disconnect(ended.id);
|
||||
// An ended binding whose removal failed, a folder a stop left before its row, and a name that isn't a binding.
|
||||
const leftovers = [ended.id, "netb-00000000000000000009"];
|
||||
for (const id of leftovers) {
|
||||
mkdirSync(join(networkCredentialFile(home, id), ".."), { recursive: true, mode: 0o700 });
|
||||
writeFileSync(networkCredentialFile(home, id), "{}", { mode: 0o600 });
|
||||
}
|
||||
mkdirSync(join(home, "network-secrets", "notes"));
|
||||
expect(store.sweepCredentials()).toEqual({ removed: [...leftovers].sort(), failed: [] });
|
||||
for (const id of leftovers) expect(existsSync(join(networkCredentialFile(home, id), ".."))).toBe(false);
|
||||
expect(readNetworkCredential(home, kept.id)).toBe("tok-123");
|
||||
expect(existsSync(join(home, "network-secrets", "notes"))).toBe(true);
|
||||
});
|
||||
|
||||
it("a corrupt credential file is refused by its path alone, without quoting what it holds", async () => {
|
||||
const { writeFileSync } = await import("node:fs");
|
||||
seat("a@lab");
|
||||
const binding = connect("a@lab");
|
||||
writeFileSync(networkCredentialFile(home, binding.id), "tok-SECRET-not-json", { mode: 0o600 });
|
||||
let message = "";
|
||||
try { readNetworkCredential(home, binding.id); } catch (error) {
|
||||
expect(error).toMatchObject({ code: "credential_invalid" });
|
||||
message = (error as Error).message;
|
||||
}
|
||||
expect(message).toContain(networkCredentialFile(home, binding.id));
|
||||
expect(message).not.toContain("SECRET");
|
||||
});
|
||||
|
||||
describe("routes", () => {
|
||||
const app = (bearerToken: string | null = null) => {
|
||||
const router = new Hono();
|
||||
router.use("*", async (c, next) => { c.set("rigRepo" as never, rigs as never); await next(); });
|
||||
router.route("/api/network", networkRoutes({ bearerToken, home }));
|
||||
return router;
|
||||
};
|
||||
const post = (a: Hono, path: string, body: unknown, auth?: string) => a.request(path, {
|
||||
method: "POST", headers: { "Content-Type": "application/json", ...(auth ? { Authorization: `Bearer ${auth}` } : {}) }, body: JSON.stringify(body),
|
||||
});
|
||||
|
||||
it("connects, lists without the token, refuses with the store's code, and disconnects", async () => {
|
||||
seat("a@lab");
|
||||
const a = app();
|
||||
const created = await post(a, "/api/network/bindings", { session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok-9" });
|
||||
expect(created.status).toBe(201);
|
||||
const binding = await created.json() as { id: string };
|
||||
const listed = await (await a.request("/api/network/bindings")).text();
|
||||
expect(listed).toContain(binding.id);
|
||||
expect(listed).not.toContain("tok-9");
|
||||
const dup = await post(a, "/api/network/bindings", { session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "x@team", token: "tok-9" });
|
||||
expect(dup.status).toBe(409);
|
||||
expect(await dup.json()).toMatchObject({ error: "seat_already_connected" });
|
||||
const ended = await post(a, `/api/network/bindings/${binding.id}/disconnect`, { reason: "done" });
|
||||
expect(ended.status).toBe(200);
|
||||
expect(await ended.json()).toMatchObject({ state: "disconnected", stateReason: "done" });
|
||||
});
|
||||
|
||||
it("with an operator bearer configured, writes need it and reads don't", async () => {
|
||||
seat("a@lab");
|
||||
const a = app("op-secret");
|
||||
const body = { session: "a@lab", hubUrl: "https://hub.example/mcp", hubSeat: "alex@team", token: "tok-9" };
|
||||
expect((await post(a, "/api/network/bindings", body)).status).toBe(401);
|
||||
expect((await a.request("/api/network/bindings")).status).toBe(200);
|
||||
const created = await post(a, "/api/network/bindings", body, "op-secret");
|
||||
expect(created.status).toBe(201);
|
||||
const { id } = await created.json() as { id: string };
|
||||
expect((await post(a, `/api/network/bindings/${id}/disconnect`, {})).status).toBe(401);
|
||||
expect((await post(a, `/api/network/bindings/${id}/disconnect`, {}, "op-secret")).status).toBe(200);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,566 @@
|
||||
// The outbound connector against a hub that behaves like an external grant's queue: it lists the participant's own items,
|
||||
// a claim moves an item out of pending, and a close stores the resolution. Each test counts the local queue items that
|
||||
// carry a hub item: a lost answer, a restart, a rebinding or a revoke must never make a second one. No agent consumes
|
||||
// these items here, so the cells prove one local item per hub item, not one agent run.
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import Database from "better-sqlite3";
|
||||
import { existsSync, mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { ALL_MIGRATIONS } from "../src/db/all-migrations.js";
|
||||
import { migrate } from "../src/db/migrate.js";
|
||||
import { RigRepository } from "../src/domain/rig-repository.js";
|
||||
import { EventBus } from "../src/domain/event-bus.js";
|
||||
import { QueueRepository, QueueRepositoryError } from "../src/domain/queue-repository.js";
|
||||
import { NetworkBindingStore, networkCredentialFile } from "../src/domain/network/network-bindings.js";
|
||||
import { Hono } from "hono";
|
||||
import { connectMcpHub, HubError, type HubConnect, type HubItem, type HubSession } from "../src/domain/network/hub-client.js";
|
||||
import { networkRoutes } from "../src/routes/network.js";
|
||||
import { NetworkConnectorService, networkLocalItemId, runNetworkBindingPass, type NetworkConnectorDeps } from "../src/domain/network/network-connector.js";
|
||||
|
||||
const PARTICIPANT = "alex@team";
|
||||
|
||||
class FakeHub {
|
||||
readonly items = new Map<string, HubItem & { notes: string[] }>();
|
||||
readonly tokens: string[] = [];
|
||||
revoked = false;
|
||||
down = false;
|
||||
/** The next claim or close of these ids takes effect, then its answer is lost. */
|
||||
readonly loseClaimAnswer = new Set<string>();
|
||||
readonly loseCloseAnswer = new Set<string>();
|
||||
/** The next note on these ids fails; later ones land. */
|
||||
readonly failNote = new Set<string>();
|
||||
/** Shows of these ids wait until the pass closes its connection. */
|
||||
readonly stallShow = new Set<string>();
|
||||
/** Runs while a show is waiting on the hub, before it answers. */
|
||||
duringShow: ((id: string) => void) | null = null;
|
||||
/** Someone else closes these items just as ours arrives: the hub keeps theirs and answers ours as done. */
|
||||
readonly raceClose = new Map<string, Partial<HubItem>>();
|
||||
closes = 0;
|
||||
|
||||
add(qitemId: string, body = "Summarize the release notes", destination = PARTICIPANT): void {
|
||||
this.items.set(qitemId, { qitemId, sourceSession: "assistant@team", destinationSession: destination, state: "pending", body, summary: body, resolution: null, closureReason: null, notes: [] });
|
||||
}
|
||||
|
||||
readonly connect: HubConnect = async (_url, token, signal) => {
|
||||
this.tokens.push(token);
|
||||
if (this.revoked) throw new HubError("revoked", "The hub refused this participant's credential.");
|
||||
if (this.down) throw new HubError("unavailable", "The hub couldn't be reached: connect ECONNREFUSED");
|
||||
const get = (id: string) => {
|
||||
const item = this.items.get(id);
|
||||
if (!item) throw new HubError("refused", `qitem_not_found ${id}`);
|
||||
return item;
|
||||
};
|
||||
const session: HubSession = {
|
||||
listOwned: async (state, limit) => [...this.items.values()].filter((i) => i.destinationSession === PARTICIPANT && i.state === state).slice(0, limit)
|
||||
.map(({ notes: _notes, ...item }) => ({ ...item, body: null })),
|
||||
claim: async (id) => {
|
||||
const item = get(id);
|
||||
if (item.state !== "pending") throw new HubError("refused", `qitem ${id} is ${item.state}`);
|
||||
item.state = "in-progress";
|
||||
if (this.loseClaimAnswer.delete(id)) throw new HubError("unavailable", "The hub couldn't be reached: socket hang up");
|
||||
},
|
||||
show: async (id) => {
|
||||
this.duringShow?.(id);
|
||||
if (this.stallShow.has(id)) {
|
||||
await new Promise((_resolve, reject) => signal?.addEventListener("abort", () => reject(new HubError("unavailable", "The hub couldn't be reached: closed")), { once: true }));
|
||||
}
|
||||
const { notes: _notes, ...item } = get(id);
|
||||
return { ...item };
|
||||
},
|
||||
note: async (id, note) => {
|
||||
if (this.failNote.delete(id)) throw new HubError("unavailable", "The hub couldn't be reached: socket hang up");
|
||||
get(id).notes.push(note);
|
||||
},
|
||||
close: async (id, input) => {
|
||||
this.closes++;
|
||||
const item = get(id);
|
||||
if (["done", "canceled", "denied", "failed"].includes(item.state)) throw new HubError("refused", `qitem ${id} is already closed`);
|
||||
const theirs = this.raceClose.get(id);
|
||||
if (theirs) { Object.assign(item, theirs); return; }
|
||||
Object.assign(item, { state: input.state, closureReason: input.closureReason, resolution: input.resolution });
|
||||
if (this.loseCloseAnswer.delete(id)) throw new HubError("unavailable", "The hub couldn't be reached: socket hang up");
|
||||
},
|
||||
end: async () => {},
|
||||
};
|
||||
return session;
|
||||
};
|
||||
}
|
||||
|
||||
describe("network connector", () => {
|
||||
let db: Database.Database;
|
||||
let home: string;
|
||||
let repo: QueueRepository;
|
||||
let bindings: NetworkBindingStore;
|
||||
let hub: FakeHub;
|
||||
let deps: NetworkConnectorDeps;
|
||||
beforeEach(() => {
|
||||
db = new Database(":memory:");
|
||||
migrate(db, ALL_MIGRATIONS);
|
||||
const rigs = new RigRepository(db);
|
||||
const node = rigs.addNode(rigs.createRig("lab").id, "impl", { runtime: "claude-code" });
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES ('s1', ?, 'dev-impl@lab', 'running')").run(node.id);
|
||||
home = mkdtempSync(join(tmpdir(), "openrig-connector-"));
|
||||
repo = new QueueRepository(db, new EventBus(db), { validateRig: () => true, loadHumanRegistry: () => ({ ok: true, entities: [] }) });
|
||||
bindings = new NetworkBindingStore(db, home);
|
||||
hub = new FakeHub();
|
||||
deps = { db, queueRepo: repo, bindings, home, connect: hub.connect };
|
||||
});
|
||||
afterEach(() => { vi.restoreAllMocks(); db.close(); rmSync(home, { recursive: true, force: true }); });
|
||||
|
||||
const bind = () => bindings.create({ session: "dev-impl@lab", hubUrl: "https://hub.example/mcp", hubSeat: PARTICIPANT, token: "tok-1" });
|
||||
const pass = (bindingId: string) => runNetworkBindingPass(deps, bindings.get(bindingId)!);
|
||||
/** The local items carrying this hub item. */
|
||||
const runs = (hubId: string) => repo.list({ limit: 500 }).filter((i) => (i.tags ?? []).includes(`network-hub-item:${hubId}`));
|
||||
|
||||
it("takes a hub item into the bound seat once, says where it came from first, then carries pickup and the result back in order", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-1");
|
||||
const first = await pass(b.id);
|
||||
expect(first).toMatchObject({ taken: 1 });
|
||||
expect(first.error).toBeUndefined();
|
||||
expect(hub.tokens).toEqual(["tok-1"]);
|
||||
const [local] = runs("hub-1");
|
||||
expect(local).toMatchObject({ qitemId: networkLocalItemId(b, "hub-1"), sourceSession: "daemon@kernel", destinationSession: "dev-impl@lab", state: "pending",
|
||||
summary: "From assistant@team via network hub: Summarize the release notes" });
|
||||
// Where it came from is read before the requester's own words.
|
||||
expect(local!.body!.split("\n")[0]).toBe("From assistant@team through the network hub https://hub.example/mcp, as alex@team (hub item hub-1).");
|
||||
expect(local!.body).toContain("Summarize the release notes");
|
||||
// The hub claim is custody, not pickup: the item says it's queued and not picked up.
|
||||
expect(hub.items.get("hub-1")).toMatchObject({ state: "in-progress" });
|
||||
expect(hub.items.get("hub-1")!.notes).toEqual([`Queued for the connected local seat as ${local!.qitemId}. Not picked up yet.`]);
|
||||
expect(bindings.get(b.id)!.lastContactAt).not.toBeNull();
|
||||
|
||||
await pass(b.id);
|
||||
expect(runs("hub-1")).toHaveLength(1);
|
||||
expect(hub.items.get("hub-1")!.notes).toHaveLength(1);
|
||||
|
||||
repo.claim({ qitemId: local!.qitemId, destinationSession: "dev-impl@lab" });
|
||||
expect(await pass(b.id)).toMatchObject({ pickupsSent: 1, resultsSent: 0 });
|
||||
expect(hub.items.get("hub-1")!.notes.at(-1)).toMatch(/^Picked up by the local seat at /);
|
||||
|
||||
await repo.update({ qitemId: local!.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "Three changes; none breaking." });
|
||||
expect(await pass(b.id)).toMatchObject({ resultsSent: 1 });
|
||||
expect(hub.items.get("hub-1")).toMatchObject({ state: "done", resolution: "Three changes; none breaking." });
|
||||
await pass(b.id);
|
||||
expect(hub.closes).toBe(1);
|
||||
expect(runs("hub-1")).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("a claim whose answer was lost: the next pass finds the item it holds and makes its one local item", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-2");
|
||||
hub.loseClaimAnswer.add("hub-2");
|
||||
expect(await pass(b.id)).toMatchObject({ taken: 0, error: "unavailable" });
|
||||
expect(runs("hub-2")).toHaveLength(0);
|
||||
expect(bindings.get(b.id)!.state).toBe("active");
|
||||
expect(await pass(b.id)).toMatchObject({ taken: 1 });
|
||||
expect(runs("hub-2")).toHaveLength(1);
|
||||
await pass(b.id);
|
||||
expect(runs("hub-2")).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("a restart between the local create and its receipt finds the same local item, not a second one", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-3");
|
||||
hub.items.get("hub-3")!.state = "in-progress"; // claimed before the restart
|
||||
await repo.create({ qitemId: networkLocalItemId(b, "hub-3"), sourceSession: "daemon@kernel", destinationSession: "dev-impl@lab", body: "already made",
|
||||
tags: [`network-binding:${b.id}`, "network-hub-item:hub-3"] });
|
||||
await pass(b.id);
|
||||
expect(runs("hub-3")).toHaveLength(1);
|
||||
expect(runs("hub-3")[0]!.body).toBe("already made");
|
||||
expect(db.prepare("SELECT count(*) AS n FROM network_receipts WHERE hub_qitem_id = 'hub-3'").get()).toEqual({ n: 1 });
|
||||
});
|
||||
|
||||
it("a close whose answer was lost is seen closed on the next pass and recorded as sent, without closing twice", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-4");
|
||||
await pass(b.id);
|
||||
const [local] = runs("hub-4");
|
||||
await repo.update({ qitemId: local!.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "done it" });
|
||||
hub.loseCloseAnswer.add("hub-4");
|
||||
expect(await pass(b.id)).toMatchObject({ error: "unavailable" });
|
||||
expect(hub.items.get("hub-4")).toMatchObject({ state: "done", resolution: "done it" });
|
||||
const settled = await pass(b.id);
|
||||
expect(settled).toMatchObject({ resultsSent: 1 });
|
||||
expect(settled.error).toBeUndefined();
|
||||
expect(hub.items.get("hub-4")!.resolution).toBe("done it");
|
||||
expect(db.prepare("SELECT outcome, settled_at IS NOT NULL AS sent FROM network_receipts WHERE hub_qitem_id = 'hub-4'").get()).toEqual({ outcome: "done", sent: 1 });
|
||||
});
|
||||
|
||||
it("a revoked credential ends the binding: no new work comes in, and the work already taken stays", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-5");
|
||||
await pass(b.id);
|
||||
hub.revoked = true;
|
||||
expect(await pass(b.id)).toMatchObject({ error: "revoked" });
|
||||
expect(bindings.get(b.id)).toMatchObject({ state: "revoked" });
|
||||
expect(existsSync(networkCredentialFile(home, b.id))).toBe(false);
|
||||
hub.revoked = false;
|
||||
hub.add("hub-6");
|
||||
const service = new NetworkConnectorService(deps, { intervalMs: 1000 });
|
||||
await service.round();
|
||||
expect(runs("hub-6")).toHaveLength(0);
|
||||
expect(runs("hub-5")).toHaveLength(1);
|
||||
expect(hub.items.get("hub-6")!.state).toBe("pending");
|
||||
});
|
||||
|
||||
it("an unreachable hub is recorded, the binding stays active, and its next pass waits out a growing backoff", async () => {
|
||||
const b = bind();
|
||||
hub.down = true;
|
||||
let now = new Date("2026-11-11T19:00:00.000Z");
|
||||
const service = new NetworkConnectorService({ ...deps, now: () => now }, { intervalMs: 30_000, random: () => 0.5 });
|
||||
await service.round();
|
||||
expect(bindings.get(b.id)).toMatchObject({ state: "active", lastError: "The hub couldn't be reached: connect ECONNREFUSED" });
|
||||
expect(service.snapshot().bindings[0]).toMatchObject({ id: b.id, backoffMs: 60_000, lastPass: expect.objectContaining({ error: "unavailable" }) });
|
||||
const attempts = hub.tokens.length;
|
||||
now = new Date(now.getTime() + 30_000);
|
||||
await service.round();
|
||||
expect(hub.tokens.length).toBe(attempts);
|
||||
now = new Date(now.getTime() + 30_001);
|
||||
await service.round();
|
||||
expect(hub.tokens.length).toBe(attempts + 1);
|
||||
expect(service.snapshot().bindings[0]!.backoffMs).toBe(120_000);
|
||||
hub.down = false;
|
||||
now = new Date(now.getTime() + 120_001);
|
||||
await service.round();
|
||||
expect(service.snapshot().bindings[0]!.backoffMs).toBe(30_000);
|
||||
expect(bindings.get(b.id)!.lastError).toBeNull();
|
||||
});
|
||||
|
||||
it("a local item whose record is gone reads unknown on the hub and is never run again", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-7");
|
||||
await pass(b.id);
|
||||
const [local] = runs("hub-7");
|
||||
db.prepare("DELETE FROM queue_items WHERE qitem_id = ?").run(local!.qitemId);
|
||||
expect(await pass(b.id)).toMatchObject({ unknown: 1 });
|
||||
expect(hub.items.get("hub-7")!.notes.at(-1)).toBe(`This item's local record (${local!.qitemId}) is gone, so whether it ran is unknown. It won't be run again.`);
|
||||
expect(hub.items.get("hub-7")!.state).toBe("in-progress");
|
||||
await pass(b.id);
|
||||
expect(runs("hub-7")).toHaveLength(0);
|
||||
expect(db.prepare("SELECT outcome FROM network_receipts WHERE hub_qitem_id = 'hub-7'").get()).toEqual({ outcome: "unknown" });
|
||||
});
|
||||
|
||||
it("a seat with no session leaves its hub items pending and says why; once it has one, work goes to its current address", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-8");
|
||||
db.prepare("DELETE FROM sessions WHERE node_id = ?").run(b.nodeId);
|
||||
expect(await pass(b.id)).toMatchObject({ taken: 0, error: "no_session" });
|
||||
expect(hub.items.get("hub-8")).toMatchObject({ state: "pending", notes: [] });
|
||||
expect(runs("hub-8")).toHaveLength(0);
|
||||
expect(bindings.get(b.id)).toMatchObject({ state: "active", lastError: "dev-impl@lab has no session here, so its hub items stay pending until it has one." });
|
||||
// The seat comes back under another address; the binding follows its node.
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES ('s2', ?, 'impl-v2@lab', 'running')").run(b.nodeId);
|
||||
const taken = await pass(b.id);
|
||||
expect(taken).toMatchObject({ taken: 1 });
|
||||
expect(taken.error).toBeUndefined();
|
||||
expect(runs("hub-8")).toEqual([expect.objectContaining({ destinationSession: "impl-v2@lab" })]);
|
||||
expect(bindings.get(b.id)!.lastError).toBeNull();
|
||||
});
|
||||
|
||||
it("a revoke whose credential can't be removed still ends the binding, records why, and the next start sweeps it", async () => {
|
||||
const { chmodSync } = await import("node:fs");
|
||||
const b = bind();
|
||||
hub.revoked = true;
|
||||
const secrets = join(home, "network-secrets");
|
||||
chmodSync(secrets, 0o500);
|
||||
try {
|
||||
expect(await pass(b.id)).toMatchObject({ error: "revoked" });
|
||||
expect(bindings.get(b.id)!.state).toBe("revoked");
|
||||
expect(bindings.get(b.id)!.lastError).toContain(`Run \`rig network disconnect ${b.id}\` again to retry.`);
|
||||
expect(existsSync(networkCredentialFile(home, b.id))).toBe(true);
|
||||
} finally {
|
||||
chmodSync(secrets, 0o700);
|
||||
}
|
||||
const lines: string[] = [];
|
||||
const service = new NetworkConnectorService({ ...deps, log: (line) => lines.push(line) }, { intervalMs: 1000 });
|
||||
service.start();
|
||||
await service.stop();
|
||||
expect(existsSync(networkCredentialFile(home, b.id))).toBe(false);
|
||||
expect(lines).toEqual([`network connector: removed the credentials of ended bindings ${b.id}`]);
|
||||
});
|
||||
|
||||
it("a plain-http binding with its recorded choice is used pass after pass, and one without a choice sends no token", async () => {
|
||||
const b = bindings.create({ session: "dev-impl@lab", hubUrl: "http://192.0.2.7:7433/mcp", hubSeat: PARTICIPANT, token: "tok-1", allowHttp: true });
|
||||
hub.add("hub-9");
|
||||
expect((await pass(b.id)).error).toBeUndefined();
|
||||
expect((await pass(b.id)).error).toBeUndefined();
|
||||
expect(hub.tokens).toEqual(["tok-1", "tok-1"]);
|
||||
expect(runs("hub-9")).toHaveLength(1);
|
||||
|
||||
db.prepare("UPDATE network_bindings SET allow_http = 0 WHERE id = ?").run(b.id);
|
||||
expect(await pass(b.id)).toMatchObject({ error: "http_not_allowed" });
|
||||
expect(hub.tokens).toHaveLength(2);
|
||||
expect(bindings.get(b.id)!.lastError).toContain("--allow-http");
|
||||
});
|
||||
|
||||
it("a hub that redirects to another origin gets no token there, and the error names where it pointed", async () => {
|
||||
const { createServer } = await import("node:http");
|
||||
const listen = (handler: Parameters<typeof createServer>[1]) => new Promise<{ url: string; close: () => Promise<void> }>((resolve) => {
|
||||
const server = createServer(handler).listen(0, "127.0.0.1", () => {
|
||||
const port = (server.address() as { port: number }).port;
|
||||
resolve({ url: `http://127.0.0.1:${port}/mcp`, close: () => new Promise((done) => server.close(() => done())) });
|
||||
});
|
||||
});
|
||||
const elsewhere: Array<string | undefined> = [];
|
||||
const other = await listen((req, res) => { elsewhere.push(req.headers.authorization); res.writeHead(500).end(); });
|
||||
const asked: Array<string | undefined> = [];
|
||||
const hubServer = await listen((req, res) => { asked.push(req.headers.authorization); res.writeHead(307, { Location: other.url }).end(); });
|
||||
try {
|
||||
const b = bindings.create({ session: "dev-impl@lab", hubUrl: hubServer.url, hubSeat: PARTICIPANT, token: "tok-redirect", allowHttp: true });
|
||||
const result = await runNetworkBindingPass({ db, queueRepo: repo, bindings, home }, bindings.get(b.id)!);
|
||||
expect(result.error).toBe("unavailable");
|
||||
expect(asked[0]).toBe("Bearer tok-redirect");
|
||||
expect(elsewhere).toEqual([]);
|
||||
expect(bindings.get(b.id)!.lastError).toContain("redirected the connection somewhere else");
|
||||
expect(bindings.get(b.id)!.lastError).toContain(other.url);
|
||||
expect(bindings.get(b.id)!.state).toBe("active");
|
||||
} finally {
|
||||
await hubServer.close();
|
||||
await other.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("a local handoff's result comes back from its successor; failed and denied closes come back as themselves; no successor reads unknown", async () => {
|
||||
const b = bind();
|
||||
for (const id of ["hub-h1", "hub-h2", "hub-h3", "hub-h4"]) hub.add(id);
|
||||
await pass(b.id);
|
||||
const local = (id: string) => runs(id)[0]!;
|
||||
repo.claim({ qitemId: local("hub-h1").qitemId, destinationSession: "dev-impl@lab" });
|
||||
const { created } = await repo.handoff({ qitemId: local("hub-h1").qitemId, fromSession: "dev-impl@lab", toSession: "review@lab", nudge: false });
|
||||
repo.claim({ qitemId: created.qitemId, destinationSession: "review@lab" });
|
||||
await repo.update({ qitemId: created.qitemId, actorSession: "review@lab", state: "done", closureReason: "no-follow-on", resolution: "Reviewed: fine." });
|
||||
await repo.update({ qitemId: local("hub-h2").qitemId, actorSession: "dev-impl@lab", state: "failed", transitionNote: "The build broke." });
|
||||
// The queue's own refusal form: done with closure reason denied.
|
||||
await repo.update({ qitemId: local("hub-h3").qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "denied", resolution: "Not mine to do." });
|
||||
// Handed on with no successor here, as a handoff to another host leaves it.
|
||||
db.prepare("UPDATE queue_items SET state = 'handed-off' WHERE qitem_id = ?").run(local("hub-h4").qitemId);
|
||||
|
||||
expect(await pass(b.id)).toMatchObject({ resultsSent: 3, unknown: 1 });
|
||||
expect(hub.items.get("hub-h1")).toMatchObject({ state: "done", closureReason: "no-follow-on", resolution: "Reviewed: fine." });
|
||||
expect(hub.items.get("hub-h2")).toMatchObject({ state: "failed", closureReason: null, resolution: "The build broke." });
|
||||
expect(hub.items.get("hub-h3")).toMatchObject({ state: "done", closureReason: "denied", resolution: "Not mine to do." });
|
||||
expect(hub.items.get("hub-h4")).toMatchObject({ state: "in-progress" });
|
||||
expect(hub.items.get("hub-h4")!.notes.at(-1)).toBe(`This item was handed on from ${local("hub-h4").qitemId} without one successor here, so its outcome is unknown. It won't be run again.`);
|
||||
});
|
||||
|
||||
it("a disconnect while a hub answer is pending starts no local work, and work already queued stays", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-d1");
|
||||
await pass(b.id);
|
||||
hub.add("hub-d2");
|
||||
hub.duringShow = (id) => { if (id === "hub-d2") bindings.disconnect(b.id); };
|
||||
await pass(b.id);
|
||||
expect(runs("hub-d2")).toHaveLength(0);
|
||||
expect(runs("hub-d1")).toHaveLength(1);
|
||||
expect(hub.items.get("hub-d2")).toMatchObject({ state: "in-progress", notes: [] });
|
||||
});
|
||||
|
||||
it("disconnecting and connecting the same seat, hub and participant again keeps one local item per hub item, unknown included", async () => {
|
||||
const first = bind();
|
||||
for (const id of ["r-queued", "r-picked", "r-done", "r-gone"]) hub.add(id);
|
||||
await pass(first.id);
|
||||
const local = (id: string) => runs(id)[0]!;
|
||||
repo.claim({ qitemId: local("r-picked").qitemId, destinationSession: "dev-impl@lab" });
|
||||
repo.claim({ qitemId: local("r-done").qitemId, destinationSession: "dev-impl@lab" });
|
||||
await repo.update({ qitemId: local("r-done").qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "Finished before the reconnect." });
|
||||
const gone = local("r-gone").qitemId;
|
||||
db.prepare("DELETE FROM queue_items WHERE qitem_id = ?").run(gone);
|
||||
bindings.disconnect(first.id);
|
||||
|
||||
const second = bind();
|
||||
expect(second.id).not.toBe(first.id);
|
||||
expect(await pass(second.id)).toMatchObject({ taken: 0, pickupsSent: 2, resultsSent: 1, unknown: 1 });
|
||||
for (const id of ["r-queued", "r-picked", "r-done"]) expect(runs(id)).toHaveLength(1);
|
||||
expect(runs("r-gone")).toHaveLength(0);
|
||||
expect(hub.items.get("r-picked")!.notes.at(-1)).toMatch(/^Picked up by the local seat at /);
|
||||
expect(hub.items.get("r-done")).toMatchObject({ state: "done", resolution: "Finished before the reconnect." });
|
||||
expect(hub.items.get("r-gone")!.notes.at(-1)).toBe(`This item's local record (${gone}) is gone, so whether it ran is unknown. It won't be run again.`);
|
||||
expect(db.prepare("SELECT DISTINCT binding_id FROM network_receipts").all()).toEqual([{ binding_id: second.id }]);
|
||||
await pass(second.id);
|
||||
expect(runs("r-gone")).toHaveLength(0);
|
||||
expect(hub.items.get("r-gone")!.notes.filter((n) => n.includes("is unknown"))).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("a stage note that fails is sent on the next pass, and an unknown outcome is marked sent only once its note lands", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-n1");
|
||||
hub.failNote.add("hub-n1");
|
||||
expect(await pass(b.id)).toMatchObject({ taken: 1, error: "unavailable" });
|
||||
expect(hub.items.get("hub-n1")!.notes).toEqual([]);
|
||||
await pass(b.id);
|
||||
expect(hub.items.get("hub-n1")!.notes).toEqual([`Queued for the connected local seat as ${runs("hub-n1")[0]!.qitemId}. Not picked up yet.`]);
|
||||
|
||||
// An id used before and erased: the queue refuses it, so whether it ran is unknown.
|
||||
hub.add("hub-n2");
|
||||
vi.spyOn(repo, "create").mockRejectedValueOnce(new QueueRepositoryError("qitem_erased", "erased"));
|
||||
hub.failNote.add("hub-n2");
|
||||
expect(await pass(b.id)).toMatchObject({ error: "unavailable" });
|
||||
const receipt = () => db.prepare("SELECT outcome, settled_at FROM network_receipts WHERE hub_qitem_id = 'hub-n2'").get() as { outcome: string; settled_at: string | null };
|
||||
expect(receipt()).toMatchObject({ outcome: "unknown", settled_at: null });
|
||||
await pass(b.id);
|
||||
expect(hub.items.get("hub-n2")!.notes).toEqual([expect.stringMatching(/is gone, so whether it ran is unknown\. It won't be run again\.$/)]);
|
||||
expect(receipt().settled_at).not.toBeNull();
|
||||
expect(runs("hub-n2")).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("a hub item someone else closed first isn't taken as our result: the receipt says so and the local result stays", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-c1");
|
||||
await pass(b.id);
|
||||
const [local] = runs("hub-c1");
|
||||
Object.assign(hub.items.get("hub-c1")!, { state: "done", closureReason: "canceled", resolution: "Not needed any more." });
|
||||
await repo.update({ qitemId: local!.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "Here it is." });
|
||||
const closes = hub.closes;
|
||||
expect(await pass(b.id)).toMatchObject({ resultsSent: 0 });
|
||||
// Checked before the close, so nothing is written over theirs.
|
||||
expect(hub.closes).toBe(closes);
|
||||
expect(hub.items.get("hub-c1")).toMatchObject({ state: "done", closureReason: "canceled", resolution: "Not needed any more." });
|
||||
expect(db.prepare("SELECT outcome, settled_at IS NOT NULL AS settled, last_error FROM network_receipts WHERE hub_qitem_id = 'hub-c1'").get()).toEqual({
|
||||
outcome: "closed_on_hub", settled: 1,
|
||||
last_error: `The hub item was closed elsewhere (done, canceled) before this result reached it. The result stays on ${local!.qitemId}.`,
|
||||
});
|
||||
expect(repo.getById(local!.qitemId)!.resolution).toBe("Here it is.");
|
||||
});
|
||||
|
||||
it("a hub item already failed elsewhere gets no write and isn't our result, and a close that loses a race is read back as closed elsewhere", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-f1");
|
||||
hub.add("hub-f2");
|
||||
await pass(b.id);
|
||||
const [f1] = runs("hub-f1");
|
||||
const [f2] = runs("hub-f2");
|
||||
Object.assign(hub.items.get("hub-f1")!, { state: "failed", closureReason: null, resolution: "Their failure." });
|
||||
hub.raceClose.set("hub-f2", { state: "done", closureReason: "no-follow-on", resolution: "Someone else's answer." });
|
||||
for (const item of [f1!, f2!]) {
|
||||
await repo.update({ qitemId: item.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", resolution: "Ours." });
|
||||
}
|
||||
const closes = hub.closes;
|
||||
expect(await pass(b.id)).toMatchObject({ resultsSent: 0 });
|
||||
expect(hub.closes).toBe(closes + 1);
|
||||
expect(hub.items.get("hub-f1")).toMatchObject({ state: "failed", resolution: "Their failure." });
|
||||
expect(hub.items.get("hub-f2")).toMatchObject({ state: "done", resolution: "Someone else's answer." });
|
||||
const outcomes = db.prepare("SELECT hub_qitem_id, outcome FROM network_receipts WHERE hub_qitem_id IN ('hub-f1', 'hub-f2') ORDER BY hub_qitem_id").all();
|
||||
expect(outcomes).toEqual([{ hub_qitem_id: "hub-f1", outcome: "closed_on_hub" }, { hub_qitem_id: "hub-f2", outcome: "closed_on_hub" }]);
|
||||
|
||||
// The person sees both conflicts on the binding.
|
||||
const app = new Hono();
|
||||
app.use("*", async (c, next) => { c.set("rigRepo" as never, new RigRepository(db) as never); await next(); });
|
||||
app.route("/api/network", networkRoutes({ home }));
|
||||
// A later pass's ordinary contact doesn't clear it, and it's read from the receipts, so a restart keeps it.
|
||||
expect((await pass(b.id)).error).toBeUndefined();
|
||||
const listed = await (await app.request("/api/network/bindings")).json() as { bindings: Array<{ id: string; closedElsewhere: unknown[] }> };
|
||||
expect(listed.bindings.find((x) => x.id === b.id)!.closedElsewhere).toEqual([
|
||||
{ hubItem: "hub-f1", localItem: f1!.qitemId, note: `The hub item was closed elsewhere (failed) before this result reached it. The result stays on ${f1!.qitemId}.` },
|
||||
{ hubItem: "hub-f2", localItem: f2!.qitemId, note: `The hub item was closed elsewhere (done, no-follow-on) before this result reached it. The result stays on ${f2!.qitemId}.` },
|
||||
].sort((x, y) => x.localItem.localeCompare(y.localItem)));
|
||||
});
|
||||
|
||||
it("a close without a resolution sends only the closer's own closing note, never another note on the item", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-w1");
|
||||
hub.add("hub-w2");
|
||||
await pass(b.id);
|
||||
const [w1] = runs("hub-w1");
|
||||
const [w2] = runs("hub-w2");
|
||||
db.prepare("INSERT INTO queue_transitions (qitem_id, ts, state, transition_note, actor_session) VALUES (?, ?, 'pending', ?, 'daemon@kernel')")
|
||||
.run(w1!.qitemId, new Date().toISOString(), "Stuck sweep: dev-impl@lab idle 17 min, recovery qitem-recovery-x.");
|
||||
await repo.update({ qitemId: w1!.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on" });
|
||||
await repo.update({ qitemId: w2!.qitemId, actorSession: "dev-impl@lab", state: "done", closureReason: "no-follow-on", transitionNote: "Done; the notes are in the item." });
|
||||
// Notes after a close repeat its state: the closer's own later note isn't the result, and a later OpenRig note
|
||||
// doesn't hide the closer's note on the close.
|
||||
await repo.update({ qitemId: w1!.qitemId, actorSession: "dev-impl@lab", transitionNote: "Also: I left a scratch file in /tmp." });
|
||||
db.prepare("INSERT INTO queue_transitions (qitem_id, ts, state, transition_note, actor_session) VALUES (?, ?, 'done', ?, 'daemon@kernel')")
|
||||
.run(w2!.qitemId, new Date().toISOString(), "Delivery receipt: wake confirmed for dev-impl@lab.");
|
||||
await pass(b.id);
|
||||
expect(hub.items.get("hub-w1")!.resolution).toBe("Closed without a stated result.");
|
||||
expect(hub.items.get("hub-w2")!.resolution).toBe("Done; the notes are in the item.");
|
||||
});
|
||||
|
||||
it("a failure here is recorded as local, not as the hub's", async () => {
|
||||
const b = bind();
|
||||
hub.add("hub-l1");
|
||||
vi.spyOn(repo, "create").mockRejectedValueOnce(new Error("disk I/O error"));
|
||||
expect(await pass(b.id)).toMatchObject({ error: "local" });
|
||||
expect(bindings.get(b.id)!.lastError).toBe("A local error stopped this pass: disk I/O error");
|
||||
});
|
||||
|
||||
it("reporting looks at a bounded batch per pass and rotates it, so a later item isn't hidden behind older open ones", async () => {
|
||||
const b = bind();
|
||||
for (const id of ["hub-q1", "hub-q2", "hub-q3"]) hub.add(id);
|
||||
let tick = Date.parse("2026-11-11T19:00:00.000Z");
|
||||
const limited = { ...deps, reportLimit: 1, now: () => new Date(tick += 1000) };
|
||||
await runNetworkBindingPass(limited, bindings.get(b.id)!);
|
||||
// q1 and q2 stay unclaimed and open; q3 is picked up.
|
||||
repo.claim({ qitemId: runs("hub-q3")[0]!.qitemId, destinationSession: "dev-impl@lab" });
|
||||
for (let i = 0; i < 3; i++) await runNetworkBindingPass(limited, bindings.get(b.id)!);
|
||||
expect(hub.items.get("hub-q3")!.notes.at(-1)).toMatch(/^Picked up by the local seat at /);
|
||||
});
|
||||
|
||||
it("a pass whose hub stalls is closed at its deadline before any late local work, and the next binding still gets its turn", async () => {
|
||||
const node = new RigRepository(db).addNode(new RigRepository(db).listRigs()[0]!.id, "review", { runtime: "claude-code" });
|
||||
db.prepare("INSERT INTO sessions (id, node_id, session_name, status) VALUES ('s9', ?, 'dev-review@lab', 'running')").run(node.id);
|
||||
const slow = bind();
|
||||
hub.add("hub-s1");
|
||||
hub.stallShow.add("hub-s1");
|
||||
const other = new FakeHub();
|
||||
other.add("hub-o1");
|
||||
const fast = bindings.create({ session: "dev-review@lab", hubUrl: "https://other-hub.example/mcp", hubSeat: PARTICIPANT, token: "tok-2" });
|
||||
const connect: HubConnect = (url, token, signal) => (url === "https://other-hub.example/mcp" ? other : hub).connect(url, token, signal);
|
||||
const service = new NetworkConnectorService({ ...deps, connect, passTimeoutMs: 200 }, { intervalMs: 1000 });
|
||||
await service.round();
|
||||
expect(bindings.get(slow.id)!.lastError).toBe("This pass took longer than 200 ms, so its connection to the hub was closed. The next pass tries again.");
|
||||
expect(runs("hub-s1")).toHaveLength(0);
|
||||
// The other binding's pass ran in the same round and took its item.
|
||||
expect(runs("hub-o1")).toEqual([expect.objectContaining({ destinationSession: "dev-review@lab" })]);
|
||||
expect(bindings.get(fast.id)!.lastError).toBeNull();
|
||||
});
|
||||
|
||||
it("an initialize the hub answers but a stalled initialized notification is closed at the deadline, and stop doesn't wait for a pass", async () => {
|
||||
const { createServer } = await import("node:http");
|
||||
const waiting: Array<() => void> = [];
|
||||
const held: Array<import("node:http").ServerResponse> = [];
|
||||
const server = createServer((req, res) => {
|
||||
let raw = "";
|
||||
req.on("data", (chunk) => { raw += chunk; });
|
||||
req.on("end", () => {
|
||||
const message = raw ? JSON.parse(raw) as { id?: number; method?: string; params?: { protocolVersion?: string } } : {};
|
||||
if (message.method === "initialize") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" }).end(JSON.stringify({ jsonrpc: "2.0", id: message.id,
|
||||
result: { protocolVersion: message.params?.protocolVersion, capabilities: {}, serverInfo: { name: "stalling hub", version: "1" } } }));
|
||||
} else {
|
||||
held.push(res); // notifications/initialized: never answered
|
||||
waiting.shift()?.();
|
||||
}
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", () => resolve()));
|
||||
const url = `http://127.0.0.1:${(server.address() as { port: number }).port}/mcp`;
|
||||
try {
|
||||
const b = bindings.create({ session: "dev-impl@lab", hubUrl: url, hubSeat: PARTICIPANT, token: "tok-stall", allowHttp: true });
|
||||
const real = { ...deps, connect: connectMcpHub };
|
||||
expect(await runNetworkBindingPass({ ...real, passTimeoutMs: 300 }, bindings.get(b.id)!)).toMatchObject({ error: "unavailable" });
|
||||
expect(bindings.get(b.id)!.lastError).toBe("This pass took longer than 300 ms, so its connection to the hub was closed. The next pass tries again.");
|
||||
|
||||
const service = new NetworkConnectorService({ ...real, passTimeoutMs: 60_000 }, { intervalMs: 1000 });
|
||||
const stalled = new Promise<void>((resolve) => waiting.push(resolve));
|
||||
service.start();
|
||||
await stalled;
|
||||
const asked = Date.now();
|
||||
await service.stop();
|
||||
expect(Date.now() - asked).toBeLessThan(5_000);
|
||||
expect(bindings.get(b.id)!.lastError).toBe("The connector stopped during this pass and closed its connection to the hub.");
|
||||
} finally {
|
||||
for (const res of held) res.destroy();
|
||||
server.closeAllConnections?.();
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
}
|
||||
});
|
||||
|
||||
it("with no active binding, a round reaches no hub", async () => {
|
||||
const service = new NetworkConnectorService(deps, { intervalMs: 1000 });
|
||||
await service.round();
|
||||
expect(hub.tokens).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -90,7 +90,8 @@ describe("openrig-core plugin — manifest shape (HG-2.2)", () => {
|
||||
// Hook + skills wiring
|
||||
expect(manifest["skills"]).toBe("./skills");
|
||||
expect(manifest["hooks"]).toBe("./hooks/claude.json");
|
||||
expect(manifest["repository"]).toMatch(/github:mvschwarz\/openrig-plugins/);
|
||||
// An https URL: Claude's plugin directory drops a `github:` shorthand from its listing.
|
||||
expect(manifest["repository"]).toBe("https://github.com/mvschwarz/openrig-plugins");
|
||||
expect(manifest["license"]).toBeDefined();
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,424 @@
|
||||
// Hermetic tests for the pane compositor: the bottom live region that keeps the
|
||||
// operator's draft out of the transcript, pins the latest diagnostic as a copy
|
||||
// (never the only one), flushes a held streamed tail when input goes away, and
|
||||
// follows terminal resizes. No live pi.
|
||||
|
||||
import { describe, it, expect, vi } from "vitest";
|
||||
import { PassThrough } from "node:stream";
|
||||
import {
|
||||
composeLayout, createPaneCompositor, createRunnerInput, RunnerCore, type RunnerIo,
|
||||
} from "../src/adapters/pi-runner.js";
|
||||
|
||||
function paneOut(columns: number | null = 40) {
|
||||
let buf = "";
|
||||
const output = Object.assign(new PassThrough(), typeof columns === "number" ? { isTTY: true, columns } : {});
|
||||
output.on("data", (chunk: Buffer) => { buf += chunk.toString(); });
|
||||
return { output: output as unknown as NodeJS.WriteStream, text: () => buf };
|
||||
}
|
||||
|
||||
// ── One physical layout model ────────────────────────────────────────────────
|
||||
|
||||
describe("compose layout", () => {
|
||||
it("counts plain text as one row and places the caret at its end", () => {
|
||||
expect(composeLayout("ab", 40)).toEqual({ rows: 1, caretRow: 0, caretColumn: 2 });
|
||||
});
|
||||
|
||||
it("wraps wide characters whole: a CJK glyph that does not fit starts the next row", () => {
|
||||
// Width 4: "abc" fills columns 0-2, 漢 needs two cells and only one is left, so it
|
||||
// wraps — three terminal rows would be wrong; the terminal uses two.
|
||||
expect(composeLayout("abc漢", 4)).toEqual({ rows: 2, caretRow: 1, caretColumn: 2 });
|
||||
});
|
||||
|
||||
it("lays a 90-glyph CJK draft at width 40 into five rows", () => {
|
||||
expect(composeLayout("寛".repeat(90), 40).rows).toBe(5);
|
||||
});
|
||||
|
||||
it("measures a ZWJ family emoji as one two-cell grapheme at the wrap column", () => {
|
||||
// Width 4: "abc" fills columns 0-2; the family emoji is one grapheme taking two
|
||||
// cells, does not fit the last column, and wraps whole — two rows, not three.
|
||||
expect(composeLayout("abc\u{1F468}\u200D\u{1F469}\u200D\u{1F467}", 4))
|
||||
.toEqual({ rows: 2, caretRow: 1, caretColumn: 2 });
|
||||
});
|
||||
|
||||
it("advances a tab to the next tab stop, not by zero cells", () => {
|
||||
expect(composeLayout("ab\tc", 40)).toEqual({ rows: 1, caretRow: 0, caretColumn: 9 });
|
||||
});
|
||||
|
||||
it("starts a new row at an explicit newline and puts the caret on that row", () => {
|
||||
expect(composeLayout("line1\nab", 40, 6)).toEqual({ rows: 2, caretRow: 1, caretColumn: 0 });
|
||||
expect(composeLayout("line1\nab", 40, 8)).toEqual({ rows: 2, caretRow: 1, caretColumn: 2 });
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
/** A terminal double that models tmux's reflow: soft-wrapped rows join on widening,
|
||||
* paragraphs split on narrowing, and the cursor stays on its character. An ASCII-width
|
||||
* model, which is all these tests need. */
|
||||
export class ReflowScreen {
|
||||
private rows: { text: string; soft: boolean }[] = [{ text: "", soft: false }];
|
||||
private row = 0;
|
||||
private col = 0;
|
||||
columns: number;
|
||||
private handlers = new Set<() => void>();
|
||||
private view: object | null = null;
|
||||
constructor(width: number) { this.columns = width; }
|
||||
terminal(): NodeJS.WriteStream {
|
||||
if (this.view) return this.view as unknown as NodeJS.WriteStream;
|
||||
const self = this;
|
||||
this.view = {
|
||||
isTTY: true,
|
||||
write: (data: string | Buffer) => self.write(typeof data === "string" ? data : data.toString("utf8")),
|
||||
on: (_event: string, handler: () => void) => { self.handlers.add(handler); },
|
||||
get columns() { return self.columns; },
|
||||
};
|
||||
return this.view as unknown as NodeJS.WriteStream;
|
||||
}
|
||||
private ensure(row: number, soft: boolean) {
|
||||
while (this.rows.length <= row) this.rows.push({ text: "", soft: false });
|
||||
if (row > 0) this.rows[row]!.soft = soft;
|
||||
}
|
||||
private place(ch: string) {
|
||||
if (this.col >= this.columns) { this.row += 1; this.col = 0; }
|
||||
// A row reached by wrapping continues the paragraph above; one reached by a
|
||||
// newline was created hard by the \n handler.
|
||||
while (this.rows.length <= this.row) this.rows.push({ text: "", soft: this.row > 0 });
|
||||
const line = this.rows[this.row]!;
|
||||
this.rows[this.row] = { ...line, text: line.text.padEnd(this.col, " ").slice(0, this.col) + ch + line.text.slice(this.col + 1) };
|
||||
this.col += 1;
|
||||
}
|
||||
write(text: string) {
|
||||
for (let i = 0; i < text.length; i += 1) {
|
||||
const ch = text[i]!;
|
||||
if (ch === "\u001b") {
|
||||
const match = /^\u001b\[\??(\d*)([A-Za-z])/.exec(text.slice(i));
|
||||
if (match) {
|
||||
const count = match[1] ? Number(match[1]) : 1;
|
||||
if (match[2] === "A") this.row = Math.max(0, this.row - count);
|
||||
else if (match[2] === "B") this.row += count;
|
||||
else if (match[2] === "C") this.col += count;
|
||||
else if (match[2] === "K" && count === 2) this.rows[this.row] = { ...this.rows[this.row]!, text: "" };
|
||||
i += match[0].length - 1;
|
||||
} else i += 1;
|
||||
continue;
|
||||
}
|
||||
if (ch === "\r") { this.col = 0; continue; }
|
||||
if (ch === "\n") { this.row += 1; this.ensure(this.row, false); this.col = 0; continue; }
|
||||
this.place(ch);
|
||||
}
|
||||
}
|
||||
/** Reflow to a new width the way tmux does, keeping the cursor on its character,
|
||||
* then tell the compositor the terminal resized. */
|
||||
reflowTo(width: number) {
|
||||
let start = this.row;
|
||||
while (start > 0 && this.rows[start]?.soft) start -= 1;
|
||||
let paragraph = 0;
|
||||
for (let i = 0; i < start; i += 1) if (!this.rows[i]?.soft) paragraph += 1;
|
||||
let offset = this.col;
|
||||
for (let i = start; i < this.row; i += 1) offset += this.rows[i]?.text.length ?? 0;
|
||||
const paragraphs: string[] = [];
|
||||
let joined: string | null = null;
|
||||
for (const line of this.rows) {
|
||||
if (line.soft && joined !== null) joined += line.text;
|
||||
else { if (joined !== null) paragraphs.push(joined); joined = line.text; }
|
||||
}
|
||||
if (joined !== null) paragraphs.push(joined);
|
||||
const rendered: { text: string; soft: boolean }[] = [];
|
||||
const startRow: number[] = [];
|
||||
paragraphs.forEach(text => {
|
||||
startRow.push(rendered.length);
|
||||
if (text === "") { rendered.push({ text: "", soft: false }); return; }
|
||||
for (let i = 0; i < text.length; i += width) rendered.push({ text: text.slice(i, i + width), soft: i > 0 });
|
||||
});
|
||||
this.rows = rendered;
|
||||
const text = paragraphs[paragraph] ?? "";
|
||||
const clamped = Math.min(offset, text.length);
|
||||
this.row = (startRow[paragraph] ?? 0) + Math.floor(clamped / width);
|
||||
this.col = clamped % width;
|
||||
this.columns = width;
|
||||
for (const handler of this.handlers) handler();
|
||||
}
|
||||
lines(): string[] { return this.rows.map(line => line.text.replace(/ +$/, "")); }
|
||||
}
|
||||
|
||||
// ── The compositor's transcript contract ─────────────────────────────────────
|
||||
|
||||
describe("pane compositor", () => {
|
||||
it("keeps draftless streaming byte-identical to the bare renderer", () => {
|
||||
const { output, text } = paneOut();
|
||||
const pane = createPaneCompositor(output);
|
||||
pane.transcript("The quick ");
|
||||
pane.transcript("brown fox\n");
|
||||
pane.line("jumps over");
|
||||
expect(text()).toBe("The quick brown fox\njumps over\n");
|
||||
});
|
||||
|
||||
it("writes complete streamed lines above a draft exactly once and holds the unfinished tail", () => {
|
||||
const { output, text } = paneOut();
|
||||
const pane = createPaneCompositor(output);
|
||||
pane.attach(() => "my draft", () => output.write("my draft"), () => 0);
|
||||
pane.transcript("streamed words here\npartial");
|
||||
const raw = text();
|
||||
expect(raw.match(/streamed words here/g)).toHaveLength(1);
|
||||
expect(raw).not.toContain("partial");
|
||||
pane.transcript(" tail\n");
|
||||
expect(text().match(/partial tail\n/g)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("pins diagnostics to a status line, as a copy, and drops the pin on clearStatus", () => {
|
||||
const { output, text } = paneOut(60);
|
||||
const pane = createPaneCompositor(output);
|
||||
pane.attach(() => "draft", () => output.write("draft"), () => 0);
|
||||
pane.line("transcript line");
|
||||
pane.line("[pi-runner] ERROR quota reached, with long trailing advice for the operator");
|
||||
pane.diagnostic("[pi-runner] ERROR quota reached, with long trailing advice for the operator");
|
||||
expect(text()).toContain("[pi-runner] ERROR quota reached");
|
||||
pane.diagnostic("[pi-runner] second notice");
|
||||
expect(text().match(/\[pi-runner\] second notice/g)).toHaveLength(1);
|
||||
pane.detach();
|
||||
pane.clearStatus();
|
||||
const before = text().length;
|
||||
pane.transcript("plain tail\n");
|
||||
expect(text().slice(before)).toBe("plain tail\n");
|
||||
});
|
||||
|
||||
it("prints nothing itself when the width is unknown: the caller's transcript copy stands alone", () => {
|
||||
const { output, text } = paneOut(null);
|
||||
const pane = createPaneCompositor(output);
|
||||
pane.attach(() => "draft", () => output.write("draft"), () => 0);
|
||||
pane.line("[pi-runner] ERROR no width to pin into");
|
||||
pane.diagnostic("[pi-runner] ERROR no width to pin into");
|
||||
expect(text()).toBe("[pi-runner] ERROR no width to pin into\n");
|
||||
});
|
||||
|
||||
it("keeps READY and EXITED markers in the transcript, verbatim", () => {
|
||||
const { output, text } = paneOut();
|
||||
const pane = createPaneCompositor(output);
|
||||
pane.line("[pi-runner] READY session=/state/s.jsonl");
|
||||
pane.line("[pi-runner] EXITED pi exited (code 0)");
|
||||
expect(text()).toBe("[pi-runner] READY session=/state/s.jsonl\n[pi-runner] EXITED pi exited (code 0)\n");
|
||||
});
|
||||
|
||||
it("narrows without leaving a stale wrapped status behind, and keeps the transcript above", () => {
|
||||
const term = new ReflowScreen(120);
|
||||
const pane = createPaneCompositor(term.terminal());
|
||||
pane.line("SENTINEL-ABOVE");
|
||||
pane.diagnostic("S".repeat(110)); // a 110-cell status: one row at 120, two at 80
|
||||
term.reflowTo(80); // tmux reflows; the cursor keeps its character
|
||||
const lines = term.lines();
|
||||
expect(lines.filter(line => line.includes("SENTINEL-ABOVE"))).toHaveLength(1);
|
||||
// The stale first-80-cells copy of the wrapped status must be gone.
|
||||
expect(lines.some(line => line === "S".repeat(80))).toBe(false);
|
||||
// The repainted pin is clipped to the new width instead.
|
||||
expect(lines.some(line => line === "S".repeat(79) + "\u2026")).toBe(true);
|
||||
});
|
||||
|
||||
|
||||
it("keeps the sentinel when a long pinned status is resized twice", () => {
|
||||
const term = new ReflowScreen(120);
|
||||
const pane = createPaneCompositor(term.terminal());
|
||||
pane.line("SENTINEL-ABOVE");
|
||||
pane.diagnostic("S".repeat(110));
|
||||
term.reflowTo(80); // the pin is repainted clipped to 80 cells
|
||||
term.reflowTo(40); // sized from what is on screen (80 cells = 2 rows), not the 110-cell text
|
||||
const lines = term.lines();
|
||||
expect(lines.filter(line => line.includes("SENTINEL-ABOVE"))).toHaveLength(1);
|
||||
expect(lines.some(line => line === "S".repeat(40))).toBe(false);
|
||||
});
|
||||
|
||||
it("widens a wrapped draft without erasing the transcript line above it", () => {
|
||||
const term = new ReflowScreen(40);
|
||||
const input = Object.assign(new PassThrough(), { isTTY: true, isRaw: false,
|
||||
setRawMode: vi.fn(function (this: { isRaw: boolean }, value: boolean) { this.isRaw = value; }) });
|
||||
const output = term.terminal();
|
||||
const pane = createPaneCompositor(output);
|
||||
const editor = createRunnerInput(input as unknown as NodeJS.ReadStream, output, () => {}, pane);
|
||||
try {
|
||||
pane.line("SENTINEL-ABOVE");
|
||||
input.write("d".repeat(90)); // three draft rows at width 40
|
||||
input.write("\u001b[D".repeat(45)); // caret mid-draft
|
||||
term.reflowTo(120); // the draft joins into one row; the caret keeps its character
|
||||
const lines = term.lines();
|
||||
expect(lines.filter(line => line.includes("SENTINEL-ABOVE"))).toHaveLength(1);
|
||||
// One draft row repainted at the new width — not three, and not the sentinel.
|
||||
expect(lines.filter(line => line.replace(/ /g, "").startsWith("dddd")).length).toBe(1);
|
||||
} finally { editor.close(); }
|
||||
});
|
||||
});
|
||||
|
||||
// ── The compose line through the real line editor ────────────────────────────
|
||||
|
||||
describe("compose line through the real editor", () => {
|
||||
function terminal(columns = 80) {
|
||||
const input = Object.assign(new PassThrough(), { isTTY: true, isRaw: false,
|
||||
setRawMode: vi.fn(function (this: { isRaw: boolean }, value: boolean) { this.isRaw = value; }) });
|
||||
const output = Object.assign(new PassThrough(), { isTTY: true, columns });
|
||||
const blocks: string[] = [];
|
||||
let screen = "";
|
||||
output.on("data", chunk => { screen += chunk.toString(); });
|
||||
const pane = createPaneCompositor(output as unknown as NodeJS.WriteStream);
|
||||
const editor = createRunnerInput(input as unknown as NodeJS.ReadStream,
|
||||
output as unknown as NodeJS.WriteStream, s => { blocks.push(s); }, pane);
|
||||
return { input, output, editor, pane, blocks, screen: () => screen };
|
||||
}
|
||||
|
||||
it("submits the full draft once, on readline's decoded line, with no partial copy left", () => {
|
||||
const t = terminal();
|
||||
try {
|
||||
t.input.write("hello compose");
|
||||
t.input.write("\r");
|
||||
expect(t.blocks).toEqual(["hello compose"]);
|
||||
const screen = t.screen();
|
||||
expect(screen).toContain("hello compose");
|
||||
expect(screen).toContain("\u001b[2K");
|
||||
expect(screen).not.toContain("hello compose\n");
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
it("streams around an active draft without fusing", () => {
|
||||
const t = terminal();
|
||||
try {
|
||||
t.input.write("draft line");
|
||||
t.pane.line("streamed model line");
|
||||
const screen = t.screen();
|
||||
expect(screen.match(/streamed model line\n/g)).toHaveLength(1);
|
||||
expect(screen).not.toContain("model linedraft");
|
||||
t.input.write("\r");
|
||||
expect(t.blocks).toEqual(["draft line"]);
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
it("measures CJK drafts in terminal cells, not UTF-16 characters", () => {
|
||||
const t = terminal(40);
|
||||
try {
|
||||
t.input.write("寛".repeat(90));
|
||||
t.input.write("x");
|
||||
expect(t.screen().match(/\u001b\[2K/g)).toHaveLength(5);
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
it("leaves meta-return to readline: no submit, together or split", () => {
|
||||
const together = terminal();
|
||||
try {
|
||||
together.input.write("abc");
|
||||
together.input.write("\u001b\r");
|
||||
expect(together.blocks).toEqual([]);
|
||||
} finally { together.editor.close(); }
|
||||
const split = terminal();
|
||||
try {
|
||||
split.input.write("abc");
|
||||
split.input.write("\u001b");
|
||||
split.input.write("\r");
|
||||
expect(split.blocks).toEqual([]);
|
||||
} finally { split.editor.close(); }
|
||||
});
|
||||
|
||||
it("clears the edit history with the submit: undo does not resurrect the draft", () => {
|
||||
const t = terminal();
|
||||
try {
|
||||
t.input.write("abc\r");
|
||||
expect(t.blocks).toEqual(["abc"]);
|
||||
t.input.write("\u001f"); // Ctrl-_ on the now-empty line
|
||||
t.input.write("\r");
|
||||
expect(t.blocks).toEqual(["abc"]);
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
it("flushes a held streamed tail verbatim when the draft is killed (Ctrl-U)", () => {
|
||||
const t = terminal();
|
||||
try {
|
||||
t.input.write("abc");
|
||||
t.pane.transcript("an unfinished streamed tail");
|
||||
expect(t.screen()).not.toContain("an unfinished streamed tail");
|
||||
t.input.write("\u0015"); // kill line: the draft goes away, the tail is written as-is
|
||||
expect(t.screen()).toContain("an unfinished streamed tail");
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
it("joins a flushed tail with the next delta instead of splitting mid-word", () => {
|
||||
const t = terminal();
|
||||
try {
|
||||
t.input.write("abc");
|
||||
t.pane.transcript("Hello wor");
|
||||
t.input.write("\u0015");
|
||||
t.pane.transcript("ld\n");
|
||||
expect(t.screen()).toContain("Hello world");
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
it("keeps coalesced edit keys and text ordered with the compositor attached", () => {
|
||||
const t = terminal();
|
||||
try {
|
||||
t.input.write("ac\u001b[Db\u00e9\u{1F98A}\r");
|
||||
expect(t.blocks).toEqual(["ab\u00e9\u{1F98A}c"]);
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
|
||||
it("keeps the flushed tail line open so new typing starts on its own line", () => {
|
||||
const t = terminal();
|
||||
try {
|
||||
t.input.write("abc");
|
||||
t.pane.transcript("Hello wor");
|
||||
t.input.write("\u0015"); // flush "Hello wor" verbatim, no newline
|
||||
t.input.write("x"); // new draft, no newline
|
||||
t.input.write("y"); // the next keystroke must not erase the tail line
|
||||
const screen = t.screen();
|
||||
expect(screen).toContain("Hello wor\n");
|
||||
expect(screen).not.toContain("Hello worx");
|
||||
} finally { t.editor.close(); }
|
||||
});
|
||||
|
||||
it("flushes a held streamed tail at EOF", () => {
|
||||
const t = terminal();
|
||||
t.input.write("x");
|
||||
t.pane.transcript("tail at exit");
|
||||
t.editor.close();
|
||||
expect(t.screen()).toContain("tail at exit\n");
|
||||
});
|
||||
});
|
||||
|
||||
// ── Diagnostics reach the transcript, not only the status pin ────────────────
|
||||
|
||||
describe("runner diagnostics", () => {
|
||||
function coreWith() {
|
||||
const lines: string[] = [];
|
||||
const pins: string[] = [];
|
||||
const io: RunnerIo = {
|
||||
sendRpc: () => {},
|
||||
mirrorLine: (line) => lines.push(line),
|
||||
mirrorAppend: () => {},
|
||||
postActivity: () => {},
|
||||
writeSidecar: () => {},
|
||||
now: () => "2026-11-10T00:00:00Z",
|
||||
diagnostic: (text) => pins.push(text),
|
||||
clearStatus: () => pins.push(""),
|
||||
};
|
||||
const core = new RunnerCore(io, { sessionName: "seat-a@some-rig", nodeId: "node-1", launchId: "launch-1" });
|
||||
core.start();
|
||||
return { core, lines, pins };
|
||||
}
|
||||
|
||||
it("writes every diagnostic to the transcript and only pins the latest", () => {
|
||||
const { core, lines, pins } = coreWith();
|
||||
core.handlePiLine(JSON.stringify({
|
||||
type: "response", id: "pi-runner-get-state", success: false,
|
||||
error: "boom: the full provider setup advice follows at length",
|
||||
}));
|
||||
expect(lines.some(line => line.startsWith("[pi-runner] ERROR"))).toBe(true);
|
||||
expect(pins.at(-1)).toContain("the full provider setup advice follows at length");
|
||||
core.handlePiLine(JSON.stringify({ type: "message_end", message: { role: "assistant", stopReason: "error", errorMessage: "second failure with different advice" } }));
|
||||
// The first failure's full text is still in the transcript history.
|
||||
expect(lines.some(line => line.includes("the full provider setup advice follows at length"))).toBe(true);
|
||||
expect(lines.some(line => line.includes("second failure with different advice"))).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps the ERROR marker inside the pane's last 40 lines after later traffic", () => {
|
||||
const { core, lines } = coreWith();
|
||||
core.handlePiLine(JSON.stringify({
|
||||
type: "response", id: "pi-runner-get-state", success: false, error: "quota",
|
||||
}));
|
||||
for (let i = 0; i < 5; i += 1) core.handlePiLine(JSON.stringify({ type: "message_update", delta: "later traffic " }));
|
||||
const tail = lines.slice(-40);
|
||||
expect(tail.some(line => line.includes("[pi-runner] ERROR"))).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -265,7 +265,7 @@ describe("retained authority negative controls", () => {
|
||||
const sliceRevision = readSliceReadiness(f.alpha).revision;
|
||||
const events: Array<{ type: string; revision: string }> = [];
|
||||
let invalidated = 0;
|
||||
const watch = watchProofSources(f.missions, () => { invalidated++; }, {
|
||||
const watch = await watchProofSources(f.missions, () => { invalidated++; }, {
|
||||
emit: (event: { type: string; revision: string }) => events.push(event),
|
||||
} as unknown as EventBus);
|
||||
try {
|
||||
@@ -296,8 +296,11 @@ describe("retained authority negative controls", () => {
|
||||
it("pushes changed source truth, keeps sibling basis, and ignores unchanged source bytes", async () => {
|
||||
const f = fixture(); f.judge(); const sibling = readSliceReadiness(f.beta).revision;
|
||||
const events: Array<{ type: string; revision: string }> = []; let invalidated = 0;
|
||||
const watch = watchProofSources(f.missions, () => { invalidated++; }, { emit: (e: { type: string; revision: string }) => events.push(e) } as unknown as EventBus);
|
||||
const watch = await watchProofSources(f.missions, () => { invalidated++; }, { emit: (e: { type: string; revision: string }) => events.push(e) } as unknown as EventBus);
|
||||
try {
|
||||
// Creation events may interrupt asynchronous setup; count changes only after the initial basis settles.
|
||||
await expect.poll(() => watch.observation().state, { timeout: 5000 }).toBe("watching");
|
||||
events.length = 0; invalidated = 0;
|
||||
const file = join(f.alpha, "proof/evidence.md"); fs.writeFileSync(file, fs.readFileSync(file));
|
||||
await new Promise(r => setTimeout(r, 100)); expect(events).toEqual([]);
|
||||
fs.writeFileSync(file, "Measured outcome changed.\n");
|
||||
@@ -450,7 +453,7 @@ it("notifies scope followers on a node-header change even when proof readiness i
|
||||
const f = fixture(), file = join(f.alpha, "SPEC.md");
|
||||
const original = fs.readFileSync(file, "utf8"), before = readMissionReadiness(f.mission).revision;
|
||||
let scopeChanges = 0;
|
||||
const watch = watchProofSources(f.missions, () => {}, { emit: () => {} } as unknown as EventBus, () => { scopeChanges++; });
|
||||
const watch = await watchProofSources(f.missions, () => {}, { emit: () => {} } as unknown as EventBus, () => { scopeChanges++; });
|
||||
try {
|
||||
const updated = original.replace("id: 01-alpha", "id: renamed-alias");
|
||||
fs.writeFileSync(file, updated);
|
||||
|
||||
@@ -153,8 +153,15 @@ describe("request-local proof-policy reuse", () => {
|
||||
});
|
||||
});
|
||||
|
||||
async function startWatch(...args: Parameters<typeof watchProofSources>) {
|
||||
const opening = watchProofSources(...args);
|
||||
// Startup itself now yields; settle its cooperative turns under the fixture clock.
|
||||
await vi.runAllTimersAsync();
|
||||
return opening;
|
||||
}
|
||||
|
||||
describe("watch-local proof-policy reuse", () => {
|
||||
function observe(missions: string) {
|
||||
async function observe(missions: string) {
|
||||
vi.useFakeTimers();
|
||||
// Each watched directory's listener, by path: "a change" is delivered by the missions root's own watch.
|
||||
const listeners = new Map<string, () => void>();
|
||||
@@ -163,13 +170,13 @@ describe("watch-local proof-policy reuse", () => {
|
||||
return { on: vi.fn(), close: vi.fn() };
|
||||
}) as unknown as typeof fs.watch);
|
||||
const invalidate = vi.fn(), emit = vi.fn();
|
||||
const watch = watchProofSources(missions, invalidate, { emit } as unknown as EventBus);
|
||||
const watch = await startWatch(missions, invalidate, { emit } as unknown as EventBus);
|
||||
const changed = () => listeners.get(missions)!();
|
||||
const oracle = () => createHash("sha256").update(JSON.stringify(readProjectReadiness(missions).missions.map(m => [m.name, m.revision]))).digest("hex");
|
||||
const event = () => {
|
||||
const event = async () => {
|
||||
const previous = watch.observation().revision, calls = invalidate.mock.calls.length;
|
||||
const expected = oracle();
|
||||
changed(); vi.advanceTimersByTime(50);
|
||||
changed(); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(watch.observation()).toEqual({ state: "watching", revision: expected });
|
||||
expect(invalidate).toHaveBeenCalledTimes(calls + Number(previous !== expected));
|
||||
expect(emit).toHaveBeenCalledTimes(invalidate.mock.calls.length);
|
||||
@@ -177,17 +184,17 @@ describe("watch-local proof-policy reuse", () => {
|
||||
return { watch, changed, event, oracle, invalidate };
|
||||
}
|
||||
|
||||
it("reads/parses shared policy once in each real watch callback, preserving the uncached revision", () => {
|
||||
it("reads/parses shared policy once in each real watch callback, preserving the uncached revision", async () => {
|
||||
const f = fixture();
|
||||
fs.cpSync(f.mission, path.join(f.missions, "second"), { recursive: true });
|
||||
const o = observe(f.missions), read = vi.spyOn(fs, "readFileSync"), parse = vi.spyOn(YAML, "parse");
|
||||
const o = await observe(f.missions), read = vi.spyOn(fs, "readFileSync"), parse = vi.spyOn(YAML, "parse");
|
||||
const policy = fs.readFileSync(f.projectFile, "utf8");
|
||||
try {
|
||||
for (const text of ["changed outcome", "second outcome"]) {
|
||||
f.write(path.join(f.dirs[0]!, "SPEC.md"), `# Outcome\n\n## Proof contract\n- [ ] ${text}.\n`);
|
||||
const expected = o.oracle();
|
||||
read.mockClear(); parse.mockClear();
|
||||
o.changed(); vi.advanceTimersByTime(50);
|
||||
o.changed(); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(o.watch.observation()).toEqual({ state: "watching", revision: expected });
|
||||
expect.soft(read.mock.calls.filter(([file]) => file === f.projectFile)).toHaveLength(1);
|
||||
expect.soft(parse.mock.calls.filter(([text]) => text === policy)).toHaveLength(1);
|
||||
@@ -196,37 +203,37 @@ describe("watch-local proof-policy reuse", () => {
|
||||
expect.soft(read.mock.calls.filter(([file]) => file === f.missionFile)).toHaveLength(2);
|
||||
}
|
||||
expect(o.invalidate).toHaveBeenCalledTimes(2);
|
||||
o.event(); expect(o.invalidate).toHaveBeenCalledTimes(2);
|
||||
await o.event(); expect(o.invalidate).toHaveBeenCalledTimes(2);
|
||||
} finally { o.watch.close(); }
|
||||
});
|
||||
|
||||
it("observes changed, missing, overridden, malformed and unreadable policy on the next event", () => {
|
||||
it("observes changed, missing, overridden, malformed and unreadable policy on the next event", async () => {
|
||||
const f = fixture(), sliceFile = path.join(f.dirs[0]!, "slice.yaml");
|
||||
const project = fs.readFileSync(f.projectFile, "utf8"), mission = fs.readFileSync(f.missionFile, "utf8"), slice = fs.readFileSync(sliceFile, "utf8");
|
||||
const o = observe(f.missions);
|
||||
const o = await observe(f.missions);
|
||||
try {
|
||||
for (const [file, base, actor] of [[f.projectFile, project, "new-project"], [f.missionFile, mission, "new-mission"], [sliceFile, slice, "new-slice"]]) {
|
||||
f.write(file!, { ...YAML.parse(base!), proofPolicy: { judges: [actor] } });
|
||||
o.event(); expect(readSliceReadiness(f.dirs[0]!).policy!.judges).toEqual([actor]);
|
||||
await o.event(); expect(readSliceReadiness(f.dirs[0]!).policy!.judges).toEqual([actor]);
|
||||
}
|
||||
f.write(sliceFile, slice); f.write(f.missionFile, mission); o.event();
|
||||
fs.unlinkSync(f.projectFile); o.event(); expect(readSliceReadiness(f.dirs[0]!).state).toBe("legacy");
|
||||
f.write(f.projectFile, project); o.event();
|
||||
f.write(sliceFile, slice); f.write(f.missionFile, mission); await o.event();
|
||||
fs.unlinkSync(f.projectFile); await o.event(); expect(readSliceReadiness(f.dirs[0]!).state).toBe("legacy");
|
||||
f.write(f.projectFile, project); await o.event();
|
||||
for (const bad of ["proofPolicy: [unclosed", { proofPolicy: { judges: [] } }]) {
|
||||
f.write(f.projectFile, bad); o.event(); expect(readSliceReadiness(f.dirs[0]!).state).toBe("unknown");
|
||||
f.write(f.projectFile, project); o.event(); expect(readSliceReadiness(f.dirs[0]!).policy!.judges).toEqual(["judge@trial"]);
|
||||
f.write(f.projectFile, bad); await o.event(); expect(readSliceReadiness(f.dirs[0]!).state).toBe("unknown");
|
||||
f.write(f.projectFile, project); await o.event(); expect(readSliceReadiness(f.dirs[0]!).policy!.judges).toEqual(["judge@trial"]);
|
||||
}
|
||||
const original = fs.readFileSync, read = vi.spyOn(fs, "readFileSync");
|
||||
read.mockImplementation(((file: fs.PathOrFileDescriptor, ...args: any[]) => {
|
||||
if (file === f.projectFile) throw Object.assign(new Error("policy denied"), { code: "EACCES" });
|
||||
return (original as any)(file, ...args);
|
||||
}) as typeof fs.readFileSync);
|
||||
o.event(); expect(readSliceReadiness(f.dirs[0]!).state).toBe("unknown");
|
||||
read.mockImplementation(original); o.event(); expect(readSliceReadiness(f.dirs[0]!).policy!.judges).toEqual(["judge@trial"]);
|
||||
await o.event(); expect(readSliceReadiness(f.dirs[0]!).state).toBe("unknown");
|
||||
read.mockImplementation(original); await o.event(); expect(readSliceReadiness(f.dirs[0]!).policy!.judges).toEqual(["judge@trial"]);
|
||||
} finally { o.watch.close(); }
|
||||
});
|
||||
|
||||
it("observes accepted evidence outside the missions root at its parent, never recursively, through edit and atomic replacement", () => {
|
||||
it("observes accepted evidence outside the missions root at its parent, never recursively, through edit and atomic replacement", async () => {
|
||||
const f = fixture(), dir = f.dirs[0]!, outside = path.join(f.root, "build", "out"), evidence = path.join(outside, "report.md");
|
||||
f.write(evidence, "accepted observation");
|
||||
const item = readSliceReadiness(dir).items[0]!;
|
||||
@@ -238,7 +245,7 @@ describe("watch-local proof-policy reuse", () => {
|
||||
return { on: vi.fn(), close: vi.fn() };
|
||||
}) as unknown as typeof fs.watch);
|
||||
const invalidate = vi.fn();
|
||||
const watch = watchProofSources(f.missions, invalidate, { emit: vi.fn() } as unknown as EventBus);
|
||||
const watch = await startWatch(f.missions, invalidate, { emit: vi.fn() } as unknown as EventBus);
|
||||
const oracle = () => createHash("sha256").update(JSON.stringify(readProjectReadiness(f.missions).missions.map(m => [m.name, m.revision]))).digest("hex");
|
||||
try {
|
||||
expect(watch.observation()).toEqual({ state: "watching", revision: oracle() });
|
||||
@@ -246,34 +253,34 @@ describe("watch-local proof-policy reuse", () => {
|
||||
expect(recursive.get(outside)).toBe(false);
|
||||
expect([...recursive].filter(([root, deep]) => deep && root !== f.missions)).toEqual([]);
|
||||
f.write(evidence, "changed observation");
|
||||
listeners.get(outside)!("change", "report.md"); vi.advanceTimersByTime(50);
|
||||
listeners.get(outside)!("change", "report.md"); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readSliceReadiness(dir).state).toBe("unknown");
|
||||
expect(watch.observation()).toEqual({ state: "watching", revision: oracle() });
|
||||
fs.writeFileSync(`${evidence}.tmp`, "accepted observation"); fs.renameSync(`${evidence}.tmp`, evidence);
|
||||
listeners.get(outside)!("rename", "report.md"); vi.advanceTimersByTime(50);
|
||||
listeners.get(outside)!("rename", "report.md"); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readSliceReadiness(dir).state).toBe("ready");
|
||||
expect(watch.observation()).toEqual({ state: "watching", revision: oracle() });
|
||||
expect(invalidate).toHaveBeenCalledTimes(2);
|
||||
} finally { watch.close(); }
|
||||
});
|
||||
|
||||
it("rechecks accepted evidence between events without reusing judgment inputs", () => {
|
||||
it("rechecks accepted evidence between events without reusing judgment inputs", async () => {
|
||||
const f = fixture(), dir = f.dirs[0]!, evidence = path.join(dir, "proof/evidence.md");
|
||||
f.write(evidence, "accepted observation");
|
||||
const item = readSliceReadiness(dir).items[0]!;
|
||||
recordJudgment(f.missions, { scope: "trial/slices/slice-0", item: item.id, verdict: "accept", reason: "observed", evidence: ["proof/evidence.md"], expectedRevision: item.revision, expectedPrevious: null }, "judge@trial", "fixture");
|
||||
const o = observe(f.missions);
|
||||
const o = await observe(f.missions);
|
||||
try {
|
||||
expect(readSliceReadiness(dir).state).toBe("ready");
|
||||
f.write(evidence, "changed observation"); o.event(); expect(readSliceReadiness(dir).state).toBe("unknown");
|
||||
f.write(evidence, "accepted observation"); o.event(); expect(readSliceReadiness(dir).state).toBe("ready");
|
||||
f.write(evidence, "changed observation"); await o.event(); expect(readSliceReadiness(dir).state).toBe("unknown");
|
||||
f.write(evidence, "accepted observation"); await o.event(); expect(readSliceReadiness(dir).state).toBe("ready");
|
||||
expect(o.invalidate).toHaveBeenCalledTimes(2);
|
||||
} finally { o.watch.close(); }
|
||||
});
|
||||
});
|
||||
|
||||
describe("QA bounded adverse coverage at 8b236e06", () => {
|
||||
function adverseFixture() {
|
||||
async function adverseFixture() {
|
||||
const f = fixture(), dir = f.dirs[0]!, outside = path.join(f.root, "build", "out"), evidence = path.join(outside, "report.md");
|
||||
f.write(evidence, "accepted observation");
|
||||
const item = readSliceReadiness(dir).items[0]!;
|
||||
@@ -284,46 +291,46 @@ describe("QA bounded adverse coverage at 8b236e06", () => {
|
||||
const h = { event: listener, error: undefined as (() => void) | undefined }; handles.set(root, h);
|
||||
return { on: (_kind: string, cb: () => void) => { h.error = cb; }, close: vi.fn() };
|
||||
}) as unknown as typeof fs.watch);
|
||||
const invalidate = vi.fn(), emit = vi.fn(), watch = watchProofSources(f.missions, invalidate, { emit } as unknown as EventBus);
|
||||
const invalidate = vi.fn(), emit = vi.fn(), watch = await startWatch(f.missions, invalidate, { emit } as unknown as EventBus);
|
||||
const oracle = () => createHash("sha256").update(JSON.stringify(readProjectReadiness(f.missions).missions.map(m => [m.name, m.revision]))).digest("hex");
|
||||
return { f, dir, outside, evidence, handles, watch, oracle, invalidate, emit };
|
||||
}
|
||||
it("QA retains unavailable after an async required-handle error and another valid event unless coverage is replaced", () => {
|
||||
const t = adverseFixture();
|
||||
it("QA retains unavailable after an async required-handle error and another valid event unless coverage is replaced", async () => {
|
||||
const t = await adverseFixture();
|
||||
try {
|
||||
expect(t.watch.observation()).toEqual({ state: "watching", revision: t.oracle() });
|
||||
const failed = t.handles.get(t.outside)!; failed.error!();
|
||||
expect(t.watch.observation().state).toBe("unavailable");
|
||||
t.f.write(t.evidence, "changed observation");
|
||||
t.handles.get(t.f.missions)!.event("change", "trial/slices/slice-0/SPEC.md"); vi.advanceTimersByTime(50);
|
||||
t.handles.get(t.f.missions)!.event("change", "trial/slices/slice-0/SPEC.md"); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readSliceReadiness(t.dir).state).toBe("unknown");
|
||||
expect(t.watch.observation().revision).toBe(t.oracle());
|
||||
// A fresh semantic read is not repaired notification coverage for the errored, unreplaced handle.
|
||||
expect(t.watch.observation().state).toBe("unavailable");
|
||||
} finally { t.watch.close(); }
|
||||
});
|
||||
it("rebinds a replaced evidence parent when its structural parent reports a change without a filename", () => {
|
||||
const t = adverseFixture();
|
||||
it("rebinds a replaced evidence parent when its structural parent reports a change without a filename", async () => {
|
||||
const t = await adverseFixture();
|
||||
try {
|
||||
const opened = () => vi.mocked(fs.watch).mock.calls.filter(([root]) => root === t.outside).length;
|
||||
expect(opened()).toBe(1);
|
||||
fs.renameSync(t.outside, `${t.outside}.old`); t.f.write(t.evidence, "changed observation");
|
||||
// The platform names nothing: only the structural parent of the replaced directory reports a change.
|
||||
t.handles.get(path.dirname(t.outside))!.event("rename", null); vi.advanceTimersByTime(50);
|
||||
t.handles.get(path.dirname(t.outside))!.event("rename", null); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readSliceReadiness(t.dir).state).toBe("unknown");
|
||||
expect(t.watch.observation().revision).toBe(t.oracle());
|
||||
// The old same-path handle cannot certify coverage: it is replaced, or the sources are reported unobserved.
|
||||
expect(opened() === 2 || t.watch.observation().state === "unavailable").toBe(true);
|
||||
} finally { t.watch.close(); }
|
||||
});
|
||||
it("watches the real location of evidence reached through a link inside the project", () => {
|
||||
const t = adverseFixture();
|
||||
it("watches the real location of evidence reached through a link inside the project", async () => {
|
||||
const t = await adverseFixture();
|
||||
try {
|
||||
// The evidence directory becomes a link to another directory inside the project, with the same accepted bytes.
|
||||
const real = path.join(t.f.root, "real"), live = path.join(real, "live");
|
||||
t.f.write(path.join(live, "report.md"), "accepted observation");
|
||||
fs.renameSync(t.outside, `${t.outside}.old`); fs.symlinkSync(live, t.outside);
|
||||
t.handles.get(path.dirname(t.outside))!.event("rename", path.basename(t.outside)); vi.advanceTimersByTime(50);
|
||||
t.handles.get(path.dirname(t.outside))!.event("rename", path.basename(t.outside)); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readSliceReadiness(t.dir).state).toBe("ready");
|
||||
// Replacing the real directory is seen at its own parent, or the sources are reported unobserved.
|
||||
const watched = vi.mocked(fs.watch).mock.calls.map(([root]) => root);
|
||||
@@ -331,7 +338,7 @@ describe("QA bounded adverse coverage at 8b236e06", () => {
|
||||
expect(watched.some((root) => !String(root).startsWith(t.f.root) && root !== path.dirname(t.f.root))).toBe(false);
|
||||
} finally { t.watch.close(); }
|
||||
});
|
||||
for (const name of ["missions", null]) it(`rebinds a linked missions root when its real directory is replaced (${name ?? "no filename"})`, () => {
|
||||
for (const name of ["missions", null]) it(`rebinds a linked missions root when its real directory is replaced (${name ?? "no filename"})`, async () => {
|
||||
const f = fixture(), real = path.join(f.root, "real"), linked = path.join(f.root, "linked-missions");
|
||||
fs.mkdirSync(real); fs.renameSync(f.missions, path.join(real, "missions")); fs.symlinkSync(path.join(real, "missions"), linked);
|
||||
vi.useFakeTimers();
|
||||
@@ -342,22 +349,22 @@ describe("QA bounded adverse coverage at 8b236e06", () => {
|
||||
return { on: vi.fn(), close: vi.fn() };
|
||||
}) as unknown as typeof fs.watch);
|
||||
const oracle = () => createHash("sha256").update(JSON.stringify(readProjectReadiness(linked).missions.map(m => [m.name, m.revision]))).digest("hex");
|
||||
const watch = watchProofSources(linked, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
const watch = await startWatch(linked, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
try {
|
||||
expect(watch.observation()).toEqual({ state: "watching", revision: oracle() });
|
||||
fs.renameSync(path.join(real, "missions"), path.join(real, "missions.old"));
|
||||
fs.cpSync(path.join(real, "missions.old"), path.join(real, "missions"), { recursive: true });
|
||||
const before = opened.length;
|
||||
opened.filter((h) => h.root === real).at(-1)!.event("rename", name); vi.advanceTimersByTime(50);
|
||||
opened.filter((h) => h.root === real).at(-1)!.event("rename", name); await vi.advanceTimersByTimeAsync(50);
|
||||
// A new mission in the replacement tree reaches clients through the live watches only.
|
||||
fs.cpSync(path.join(real, "missions", "trial"), path.join(real, "missions", "new-mission"), { recursive: true });
|
||||
for (const h of opened.slice(before)) h.event("rename", h.recursive ? "new-mission/mission.yaml" : "new-mission");
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(watch.observation().state === "unavailable" || watch.observation().revision === oracle()).toBe(true);
|
||||
expect(opened.some((h) => !h.root.startsWith(f.root) && h.root !== path.dirname(f.root))).toBe(false);
|
||||
} finally { watch.close(); }
|
||||
});
|
||||
for (const start of ["outside", "inside"] as const) it(`reports unavailable while a linked missions root's real location is outside the project (starts ${start})`, () => {
|
||||
for (const start of ["outside", "inside"] as const) it(`reports unavailable while a linked missions root's real location is outside the project (starts ${start})`, async () => {
|
||||
const f = fixture(), linked = path.join(f.root, "linked-missions"), inside = path.join(f.root, "real", "missions");
|
||||
const other = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "policy-reuse-other-"))); roots.push(other);
|
||||
const outside = path.join(other, "missions");
|
||||
@@ -370,21 +377,21 @@ describe("QA bounded adverse coverage at 8b236e06", () => {
|
||||
opened.push({ root, event: listener });
|
||||
return { on: vi.fn(), close: vi.fn() };
|
||||
}) as unknown as typeof fs.watch);
|
||||
const emit = vi.fn(), watch = watchProofSources(linked, vi.fn(), { emit } as unknown as EventBus);
|
||||
const emit = vi.fn(), watch = await startWatch(linked, vi.fn(), { emit } as unknown as EventBus);
|
||||
try {
|
||||
if (start === "inside") {
|
||||
expect(watch.observation().state).toBe("watching");
|
||||
// The supported alias is re-pointed outside the project; the link's own parent sees it.
|
||||
fs.unlinkSync(linked); fs.symlinkSync(outside, linked);
|
||||
opened.filter((h) => h.root === f.root).at(-1)!.event("rename", "linked-missions"); vi.advanceTimersByTime(50);
|
||||
opened.filter((h) => h.root === f.root).at(-1)!.event("rename", "linked-missions"); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(emit.mock.calls.map(([e]) => e.revision)).toContain("unavailable");
|
||||
}
|
||||
expect(watch.observation().state).toBe("unavailable");
|
||||
expect(opened.some((h) => h.root.startsWith(other))).toBe(false);
|
||||
} finally { watch.close(); }
|
||||
});
|
||||
it("QA observes replacement of an external evidence parent or honestly reports unavailable", () => {
|
||||
const t = adverseFixture();
|
||||
it("QA observes replacement of an external evidence parent or honestly reports unavailable", async () => {
|
||||
const t = await adverseFixture();
|
||||
try {
|
||||
const previous = t.watch.observation().revision;
|
||||
expect(readSliceReadiness(t.dir).state).toBe("ready");
|
||||
@@ -394,7 +401,7 @@ describe("QA bounded adverse coverage at 8b236e06", () => {
|
||||
if (root === path.dirname(t.outside)) h.event("rename", path.basename(t.outside));
|
||||
if (root === t.f.root) h.event("rename", "build");
|
||||
}
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readSliceReadiness(t.dir).state).toBe("unknown");
|
||||
expect(t.oracle()).not.toBe(previous);
|
||||
expect(t.watch.observation().state === "unavailable" || t.watch.observation().revision === t.oracle()).toBe(true);
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
import { afterEach, expect, it } from "vitest";
|
||||
import * as fs from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { Hono } from "hono";
|
||||
import { proofRoutes } from "../src/routes/proof.js";
|
||||
import { slicesRoutes } from "../src/routes/slices.js";
|
||||
import { readProjectReadiness } from "../src/domain/proof/judgments.js";
|
||||
|
||||
const roots: string[] = [];
|
||||
afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); });
|
||||
|
||||
it.each(["proof", "slices"])("serves another event-loop turn during the %s project read without changing the result", async target => {
|
||||
const root = fs.mkdtempSync(join(tmpdir(), "proof-project-yield-")); roots.push(root);
|
||||
const missions = join(root, "missions");
|
||||
fs.mkdirSync(missions);
|
||||
fs.writeFileSync(join(root, "project.yaml"), "kind: project\nmetadata: { id: trial }\n");
|
||||
for (const name of ["alpha", "beta", "gamma"]) {
|
||||
fs.mkdirSync(join(missions, name));
|
||||
fs.writeFileSync(join(missions, name, "mission.yaml"), "kind: mission\nmetadata: { status: active }\ncomposition: { slices: [] }\n");
|
||||
}
|
||||
const readiness = readProjectReadiness(missions);
|
||||
const expected = target === "proof" ? readiness : {
|
||||
slices: [], totalCount: 0, filter: "all", boundToWorkflow: null,
|
||||
missions: Object.fromEntries(readiness.missions.map(m => [m.name, { authoredStatus: m.historicalStatus, readiness: m }])),
|
||||
};
|
||||
let sourceReady = true;
|
||||
const app = new Hono();
|
||||
app.use("*", async (c, next) => {
|
||||
c.set("sliceIndexer" as never, { isReady: () => true, slicesRoot: missions,
|
||||
withMembershipBatch: (fn: () => unknown) => fn(), list: () => [],
|
||||
observedQueueCounts: () => new Map(), missionAuthoredStatuses: () => ({}),
|
||||
} as never);
|
||||
c.set("sliceDetailProjector" as never, {} as never);
|
||||
c.set("proofSourceWatch" as never, { observation: () => ({ state: sourceReady ? "watching" : "unavailable", revision: "fixture" }) } as never);
|
||||
await next();
|
||||
});
|
||||
app.route("/api/proof", proofRoutes());
|
||||
app.route("/api/slices", slicesRoutes());
|
||||
let completed = false;
|
||||
const read = app.request(`/api/${target}`).then(response => { completed = true; return response; });
|
||||
await new Promise<void>(resolve => setImmediate(resolve));
|
||||
if (completed) {
|
||||
const early = await read; expect(early.status).toBe(200);
|
||||
expect(await early.clone().json()).toMatchObject(expected);
|
||||
}
|
||||
expect(completed, "the proof read must yield before finishing the whole tree").toBe(false);
|
||||
sourceReady = false;
|
||||
const response = await read;
|
||||
expect(response.status).toBe(200);
|
||||
expect(await response.json()).toMatchObject(target === "proof" ? { ...expected, sourceObservation: { state: "unavailable" } } : expected);
|
||||
});
|
||||
@@ -6,7 +6,7 @@ const { watch, createProofPolicyRead, readProjectReadiness, resolveProjectRoot,
|
||||
}));
|
||||
// Real-location checks see every fixture path as itself (no links in these mocked trees).
|
||||
vi.mock("node:fs", () => ({ watch, existsSync: () => true, realpathSync: (p: string) => p }));
|
||||
vi.mock("../src/domain/proof/judgments.js", () => ({ createProofPolicyRead, readProjectReadiness, resolveProjectRoot, proofFs }));
|
||||
vi.mock("../src/domain/proof/judgments.js", () => ({ createProofPolicyRead, readProjectReadiness, readProjectReadinessAsync: async (...args: unknown[]) => readProjectReadiness(...args), resolveProjectRoot, proofFs }));
|
||||
import { watchProofSources } from "../src/domain/proof/source-watch.js";
|
||||
import type { EventBus } from "../src/domain/event-bus.js";
|
||||
afterEach(() => { vi.useRealTimers(); vi.resetAllMocks(); proofFs.exists.mockImplementation(() => true); });
|
||||
@@ -21,14 +21,14 @@ function reads(paths: (missions: string) => Array<[keyof Io, string]>) {
|
||||
}
|
||||
const handle = () => { const close = vi.fn(); watch.mockImplementation(() => ({ on: vi.fn(), close })); return close; };
|
||||
|
||||
it("watches only the missions root recursively; the project and evidence outside it get shallow, filtered watches", () => {
|
||||
it("watches only the missions root recursively; the project and evidence outside it get shallow, filtered watches", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project");
|
||||
const missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
reads(() => [["exists", path.join(project, "project.yaml")], ["readBytes", path.join(project, "build", "out", "report.md")]]);
|
||||
const close = handle();
|
||||
const instance = watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
// Inputs at their parents, plus shallow structural anchors up to the project's parent (for replaced directories).
|
||||
expect(watch.mock.calls.map(([root, options]) => [root, options.recursive])).toEqual([
|
||||
[missions, true], [project, false], [path.join(project, "build", "out"), false],
|
||||
@@ -43,23 +43,23 @@ it("watches only the missions root recursively; the project and evidence outside
|
||||
const notifyProject = watch.mock.calls[1]![2], notifyEvidence = watch.mock.calls[2]![2];
|
||||
notifyProject("change", "unrelated.txt");
|
||||
notifyEvidence("change", Buffer.from("other.md"));
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(2);
|
||||
notifyProject("rename", Buffer.from("project.yaml"));
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(3);
|
||||
notifyEvidence("rename", "report.md");
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(4);
|
||||
// No filename: that directory's watch is rebound (closed, reattached by path) and the read verified again.
|
||||
notifyEvidence("change", null);
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(6);
|
||||
instance.close();
|
||||
expect(close).toHaveBeenCalledTimes(6);
|
||||
});
|
||||
|
||||
it("watches the nearest existing ancestor of a missing input, filtered to the next path segment", () => {
|
||||
it("watches the nearest existing ancestor of a missing input, filtered to the next path segment", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project");
|
||||
const missions = path.join(project, "missions");
|
||||
@@ -67,26 +67,26 @@ it("watches the nearest existing ancestor of a missing input, filtered to the ne
|
||||
proofFs.exists.mockImplementation((p: string) => !p.startsWith(path.join(project, "build")));
|
||||
reads(() => [["readBytes", path.join(project, "build", "out", "report.md")]]);
|
||||
handle();
|
||||
const instance = watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
expect(watch.mock.calls.map(([root, options]) => [root, options.recursive])).toEqual([[missions, true], [project, false], [path.dirname(project), false]]);
|
||||
const notify = watch.mock.calls[1]![2];
|
||||
notify("rename", "unrelated");
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(2);
|
||||
notify("rename", "build");
|
||||
vi.advanceTimersByTime(50);
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(3);
|
||||
instance.close();
|
||||
});
|
||||
|
||||
it("reports unavailable and closes what it opened when a required watch fails, without a wider fallback", () => {
|
||||
it("reports unavailable and closes what it opened when a required watch fails, without a wider fallback", async () => {
|
||||
const project = path.resolve("fixture/project");
|
||||
const missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
reads(() => [["exists", path.join(project, "project.yaml")]]);
|
||||
const close = vi.fn();
|
||||
watch.mockReturnValueOnce({ on: vi.fn(), close }).mockImplementationOnce(() => { throw new Error("ENOSPC"); });
|
||||
const instance = watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
expect(instance.observation().state).toBe("unavailable");
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
expect(watch).toHaveBeenCalledTimes(2);
|
||||
@@ -94,12 +94,12 @@ it("reports unavailable and closes what it opened when a required watch fails, w
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("watches nothing else recursively when the evaluation reads nothing outside the missions root", () => {
|
||||
it("watches nothing else recursively when the evaluation reads nothing outside the missions root", async () => {
|
||||
const missions = path.resolve("fixture/missions");
|
||||
resolveProjectRoot.mockReturnValue(path.dirname(missions));
|
||||
reads((root) => [["exists", path.join(root, "m1", "mission.yaml")]]);
|
||||
const close = handle();
|
||||
const instance = watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
// Only the missions root's structural anchors are added, shallow.
|
||||
expect(watch.mock.calls.map(([root, options]) => [root, options.recursive])).toEqual([
|
||||
[missions, true], [path.dirname(missions), false], [path.dirname(path.dirname(missions)), false],
|
||||
@@ -108,7 +108,7 @@ it("watches nothing else recursively when the evaluation reads nothing outside t
|
||||
expect(close).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it("does not report watching when the inputs keep changing past the reconciliation budget", () => {
|
||||
it("does not report watching when the inputs keep changing past the reconciliation budget", async () => {
|
||||
const project = path.resolve("fixture/project");
|
||||
const missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
@@ -120,7 +120,7 @@ it("does not report watching when the inputs keep changing past the reconciliati
|
||||
return { missions: [] };
|
||||
});
|
||||
handle();
|
||||
const instance = watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit: vi.fn() } as unknown as EventBus);
|
||||
const attached = new Set(watch.mock.calls.map(([root]) => root));
|
||||
// Either the final dependency set is actually watched and verified, or the monitor says it is not watching.
|
||||
const finalDir = path.dirname(sets[2]!);
|
||||
@@ -129,7 +129,7 @@ it("does not report watching when the inputs keep changing past the reconciliati
|
||||
instance.close();
|
||||
});
|
||||
|
||||
it("tells clients when coverage is lost or regained even though the proof result is unchanged", () => {
|
||||
it("tells clients when coverage is lost or regained even though the proof result is unchanged", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project");
|
||||
const missions = path.join(project, "missions");
|
||||
@@ -146,19 +146,19 @@ it("tells clients when coverage is lost or regained even though the proof result
|
||||
});
|
||||
handle();
|
||||
const emit = vi.fn();
|
||||
const instance = watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
expect(instance.observation().state).toBe("watching");
|
||||
const notifyProject = watch.mock.calls[1]![2];
|
||||
notifyProject("change", "project.yaml"); vi.advanceTimersByTime(50);
|
||||
notifyProject("change", "project.yaml"); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(instance.observation().state).toBe("unavailable");
|
||||
expect(emit.mock.calls.map(([e]) => e.revision)).toEqual(["unavailable"]);
|
||||
notifyProject("change", "project.yaml"); vi.advanceTimersByTime(50);
|
||||
notifyProject("change", "project.yaml"); await vi.advanceTimersByTimeAsync(50);
|
||||
expect(instance.observation().state).toBe("watching");
|
||||
expect(emit.mock.calls.map(([e]) => e.revision)).toEqual(["unavailable", instance.observation().revision]);
|
||||
instance.close();
|
||||
});
|
||||
|
||||
it("after close, a queued callback neither re-reads, reopens nor emits", () => {
|
||||
it("after close, a queued callback neither re-reads, reopens nor emits", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project");
|
||||
const missions = path.join(project, "missions");
|
||||
@@ -166,14 +166,141 @@ it("after close, a queued callback neither re-reads, reopens nor emits", () => {
|
||||
reads(() => [["exists", path.join(project, "project.yaml")]]);
|
||||
handle();
|
||||
const emit = vi.fn();
|
||||
const instance = watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
const notifyMissions = watch.mock.calls[0]![2];
|
||||
notifyMissions("change", "x");
|
||||
instance.close();
|
||||
notifyMissions("change", "y");
|
||||
vi.advanceTimersByTime(100);
|
||||
await vi.advanceTimersByTimeAsync(100);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(2);
|
||||
expect(watch).toHaveBeenCalledTimes(3);
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
expect(instance.observation().state).toBe("unavailable");
|
||||
});
|
||||
|
||||
it.each(["changed", "unchanged"])("coalesces mid-read events and publishes only a current %s basis", async mode => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project"), missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
reads(() => [["exists", path.join(project, "project.yaml")]]);
|
||||
handle();
|
||||
const emit = vi.fn();
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
let finish!: (value: { missions: Array<{ name: string; revision: string }> }) => void;
|
||||
readProjectReadiness.mockReturnValueOnce(new Promise(resolve => { finish = resolve; }));
|
||||
const notify = watch.mock.calls[0]![2];
|
||||
notify("change", "SPEC.md");
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
const settledRevision = instance.observation().revision;
|
||||
const calls = readProjectReadiness.mock.calls.length;
|
||||
notify("change", "SPEC.md"); notify("change", "SPEC.md");
|
||||
await vi.advanceTimersByTimeAsync(100);
|
||||
expect(readProjectReadiness).toHaveBeenCalledTimes(calls);
|
||||
finish({ missions: [{ name: "trial", revision: "superseded" }] });
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
expect(instance.observation().revision).toBe(settledRevision);
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
readProjectReadiness.mockReturnValue({ missions: mode === "changed" ? [{ name: "trial", revision: "current" }] : [] });
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(instance.observation().state).toBe("watching");
|
||||
expect(emit).toHaveBeenCalledTimes(mode === "changed" ? 1 : 0);
|
||||
instance.close();
|
||||
});
|
||||
|
||||
it("publishes unchanged-revision coverage recovery only after an interrupted read settles cleanly", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project"), missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
let call = 0;
|
||||
readProjectReadiness.mockImplementation((_m: string, _r: unknown, _root: unknown, io?: Io) => {
|
||||
io?.exists(path.join(project, "project.yaml"));
|
||||
io?.readBytes(path.join(project, `input-${Math.min(call++, 2)}`, "report.md"));
|
||||
return { missions: [] };
|
||||
});
|
||||
handle();
|
||||
const emit = vi.fn();
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
const settled = instance.observation();
|
||||
expect(settled.state).toBe("unavailable"); // all three setup passes attached a previously unwatched dependency
|
||||
const read = readProjectReadiness.getMockImplementation()!;
|
||||
let finish!: (value: { missions: [] }) => void;
|
||||
readProjectReadiness.mockImplementationOnce((...args: unknown[]) => {
|
||||
read(...args);
|
||||
return new Promise(resolve => { finish = resolve; });
|
||||
});
|
||||
const notify = watch.mock.calls[0]![2];
|
||||
notify("change", "report.md"); await vi.advanceTimersByTimeAsync(50);
|
||||
notify("change", "report.md");
|
||||
finish({ missions: [] }); await vi.advanceTimersByTimeAsync(0);
|
||||
expect.soft(instance.observation()).toEqual(settled);
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(instance.observation()).toEqual({ state: "watching", revision: settled.revision });
|
||||
expect.soft(emit.mock.calls.map(([event]) => event.revision)).toEqual([settled.revision]);
|
||||
instance.close();
|
||||
});
|
||||
|
||||
it("reports watch errors immediately while a refresh is still awaiting its read", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project"), missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
reads(() => [["exists", path.join(project, "project.yaml")]]);
|
||||
let error!: () => void;
|
||||
watch.mockImplementation(() => ({ on: (_event: string, callback: () => void) => { error = callback; }, close: vi.fn() }));
|
||||
const emit = vi.fn();
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
let finish!: (value: { missions: [] }) => void;
|
||||
readProjectReadiness.mockReturnValueOnce(new Promise(resolve => { finish = resolve; }));
|
||||
watch.mock.calls[0]; await vi.advanceTimersByTimeAsync(50);
|
||||
error();
|
||||
expect(instance.observation()).toEqual({ state: "unavailable", revision: "unavailable" });
|
||||
expect(emit.mock.calls.at(-1)![0].revision).toBe("unavailable");
|
||||
finish({ missions: [] }); await vi.advanceTimersByTimeAsync(0);
|
||||
expect(instance.observation().state).toBe("unavailable");
|
||||
expect(emit.mock.calls.every(([event]) => event.revision === "unavailable")).toBe(true);
|
||||
instance.close();
|
||||
});
|
||||
|
||||
it("does not publish an interrupted setup candidate before a clean refresh", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project"), missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
reads(() => [["exists", path.join(project, "project.yaml")]]);
|
||||
const read = readProjectReadiness.getMockImplementation()!;
|
||||
let finish!: (value: { missions: [] }) => void;
|
||||
readProjectReadiness.mockImplementationOnce((...args: unknown[]) => {
|
||||
read(...args);
|
||||
return new Promise(resolve => { finish = resolve; });
|
||||
});
|
||||
handle();
|
||||
const emit = vi.fn(), opening = watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
watch.mock.calls[0];
|
||||
finish({ missions: [] });
|
||||
const instance = await opening;
|
||||
expect(instance.observation()).toEqual({ state: "unavailable", revision: "unavailable" });
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
expect(instance.observation().state).toBe("watching");
|
||||
expect(emit).toHaveBeenCalledTimes(1);
|
||||
instance.close();
|
||||
});
|
||||
|
||||
it("does not reattach watches or publish when an in-flight read finishes after close", async () => {
|
||||
vi.useFakeTimers();
|
||||
const project = path.resolve("fixture/project"), missions = path.join(project, "missions");
|
||||
resolveProjectRoot.mockReturnValue(project);
|
||||
reads(() => [["exists", path.join(project, "project.yaml")]]);
|
||||
handle();
|
||||
const emit = vi.fn();
|
||||
const instance = await watchProofSources(missions, vi.fn(), { emit } as unknown as EventBus);
|
||||
let finish!: (value: { missions: [] }) => void;
|
||||
readProjectReadiness.mockReturnValueOnce(new Promise(resolve => { finish = resolve; }));
|
||||
watch.mock.calls[0];
|
||||
await vi.advanceTimersByTimeAsync(50);
|
||||
const watches = watch.mock.calls.length;
|
||||
instance.close(); finish({ missions: [] });
|
||||
await vi.advanceTimersByTimeAsync(100);
|
||||
expect(watch).toHaveBeenCalledTimes(watches);
|
||||
expect(emit).not.toHaveBeenCalled();
|
||||
expect(instance.observation().state).toBe("unavailable");
|
||||
});
|
||||
@@ -25,10 +25,13 @@ it("watches the owning project policy above a nested missions root", async () =>
|
||||
expect(readSliceReadiness(slice).state).toBe("ready");
|
||||
const events: unknown[] = [];
|
||||
let invalidated = 0;
|
||||
const watch = watchProofSources(missions, () => invalidated++, {
|
||||
const watch = await watchProofSources(missions, () => invalidated++, {
|
||||
emit: (event: unknown) => events.push(event),
|
||||
} as unknown as EventBus);
|
||||
try {
|
||||
// A creation event can dirty setup before it returns; start assertions at the settled observed basis.
|
||||
await expect.poll(() => watch.observation().state, { timeout: 5000 }).toBe("watching");
|
||||
events.length = 0; invalidated = 0;
|
||||
await new Promise(resolve => setTimeout(resolve, 250));
|
||||
expect(events).toHaveLength(0);
|
||||
const before = watch.observation().revision;
|
||||
@@ -43,11 +46,11 @@ it("watches the owning project policy above a nested missions root", async () =>
|
||||
} finally { watch.close(); fs.rmSync(root, { recursive: true, force: true }); }
|
||||
});
|
||||
|
||||
it("retains the existing watch boundary for roots without project.yaml", () => {
|
||||
it("retains the existing watch boundary for roots without project.yaml", async () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "proof-legacy-root-"));
|
||||
const missions = path.join(root, "missions");
|
||||
fs.mkdirSync(missions);
|
||||
const watch = watchProofSources(missions, () => {}, { emit() {} } as unknown as EventBus);
|
||||
const watch = await watchProofSources(missions, () => {}, { emit() {} } as unknown as EventBus);
|
||||
try { expect(watch.observation().state).toBe("watching"); }
|
||||
finally { watch.close(); fs.rmSync(root, { recursive: true, force: true }); }
|
||||
});
|
||||
@@ -35,6 +35,62 @@ function run(runtime: string, event: string, overrides: NodeJS.ProcessEnv = {},
|
||||
}
|
||||
const context = (r: ReturnType<typeof run>) => { expect(r.status, r.stderr || String(r.error)).toBe(0); return JSON.parse(r.stdout).hookSpecificOutput.additionalContext as string; };
|
||||
const calls = () => existsSync(env.CALLS!) ? readFileSync(env.CALLS!, "utf8").trim().split("\n") : [];
|
||||
const refocusStatePath = () => join(env.OPENRIG_HOME!, "refocus", `${env.OPENRIG_SESSION_NAME}__test-occupant.json`);
|
||||
|
||||
it.each([6_000, 12_000, 28_000])("Codex refocus output budgets %i standing-content bytes and records any omitted tail", (bytes) => {
|
||||
const content = join(root, "standing.md");
|
||||
writeFileSync(content, `STANDING START\n${"🙂".repeat(bytes / 4)}\nSTANDING END`);
|
||||
run("codex", "PostCompact");
|
||||
const text = context(run("codex", "UserPromptSubmit", { OPENRIG_REFOCUS_CONTENT_FILE: content }));
|
||||
const manifest = JSON.parse(readFileSync(resolve(hook, "../../codex.json"), "utf8"));
|
||||
const handler = manifest.hooks.UserPromptSubmit.flatMap((group: any) => group.hooks)
|
||||
.find((entry: any) => entry.command.includes("refocus.cjs"));
|
||||
expect(Buffer.byteLength(text)).toBeLessThanOrEqual(2_500 * 4);
|
||||
expect(handler.additionalContextLimit).toBeUndefined();
|
||||
const state = JSON.parse(readFileSync(refocusStatePath(), "utf8"));
|
||||
expect(state.firedAt).toEqual(expect.any(String));
|
||||
expect(state.contextOutput.outputBytes).toBe(Buffer.byteLength(text));
|
||||
if (bytes === 6_000) {
|
||||
expect(text).toContain("STANDING END");
|
||||
expect(state.contextOutput).toMatchObject({ status: "full", omittedBytes: 0 });
|
||||
} else {
|
||||
expect(text).toContain("REFOCUS SHORTENED");
|
||||
expect(text).not.toContain("STANDING END");
|
||||
expect(text).not.toContain("\uFFFD");
|
||||
expect(state.contextOutput.status).toBe("shortened");
|
||||
const full = readFileSync(state.contextOutput.fullPath, "utf8");
|
||||
expect(full).toContain("STANDING END");
|
||||
expect(state.contextOutput.originalBytes).toBe(Buffer.byteLength(full));
|
||||
expect(state.contextOutput.omittedBytes).toBe(Buffer.byteLength(full) - state.contextOutput.prefixBytes);
|
||||
expect(text.startsWith(Buffer.from(full).subarray(0, state.contextOutput.prefixBytes).toString("utf8"))).toBe(true);
|
||||
writeFileSync(content, `NEXT STANDING CONTENT\n${"x".repeat(25_000)}`);
|
||||
context(run("codex", "UserPromptSubmit", { OPENRIG_REFOCUS_CONTENT_FILE: content, OPENRIG_REFOCUS_NOW: "1" }));
|
||||
const next = JSON.parse(readFileSync(refocusStatePath(), "utf8"));
|
||||
expect(next.contextOutput.fullPath).not.toBe(state.contextOutput.fullPath);
|
||||
expect(readFileSync(state.contextOutput.fullPath, "utf8")).toBe(full);
|
||||
}
|
||||
expect(run("codex", "UserPromptSubmit").stdout).toBe("");
|
||||
});
|
||||
|
||||
it("Codex refocus output still fires and records shortening when the full-text file cannot be saved", () => {
|
||||
const content = join(root, "standing.md");
|
||||
writeFileSync(content, "x".repeat(25_000));
|
||||
run("codex", "PostCompact");
|
||||
const stateFile = refocusStatePath();
|
||||
const preload = join(root, "fail-context-save.cjs");
|
||||
writeFileSync(preload, `const fs = require("node:fs"), write = fs.writeFileSync;
|
||||
fs.writeFileSync = (file, ...args) => {
|
||||
if (String(file).endsWith(".context.txt")) throw new Error("fixture context save refused");
|
||||
return write(file, ...args);
|
||||
};`);
|
||||
const text = context(run("codex", "UserPromptSubmit", { OPENRIG_REFOCUS_CONTENT_FILE: content }, preload));
|
||||
expect(text).toContain("REFOCUS SHORTENED");
|
||||
expect(text).toContain("Full text could not be saved");
|
||||
const state = JSON.parse(readFileSync(stateFile, "utf8"));
|
||||
expect(state.firedAt).toEqual(expect.any(String));
|
||||
expect(state.contextOutput).toMatchObject({ status: "shortened", fullPath: null });
|
||||
expect(state.contextOutput.omittedBytes).toBeGreaterThan(0);
|
||||
});
|
||||
it.each(["claude", "codex"])("%s carries only A's role after PostCompact and reuses the work response", (runtime) => {
|
||||
expect(run(runtime, "PostCompact").stdout).toBe("");
|
||||
const text = context(run(runtime, "UserPromptSubmit"));
|
||||
|
||||
@@ -719,7 +719,7 @@ describe("an answered request goes to the receiving agent: the pair's round-0 co
|
||||
const askId = String((await f.submit()).body.askId);
|
||||
// The result says what became of the answer (the TUI prints it), never that the request closed.
|
||||
expect(await f.act(askId, "approve")).toMatchObject({ status: 200,
|
||||
body: { answer: { message: `Your answer on ${askId} went to ${DECIDER}, the receiving agent, which decides. The runner waits until then; nothing has run.` } } });
|
||||
body: { answer: { message: `Your answer on ${askId} went to ${DECIDER}, the receiving agent, which decides. The runner waits until then; nothing has run. A note to you says the agent's recorded state.` } } });
|
||||
expect(f.queueRepo.getById(askId)).toMatchObject({ state: "handed-off", handedOffTo: DECIDER });
|
||||
expect(await f.read(askId)).toMatchObject({ body: { state: "pending" } });
|
||||
expect(handedFrom(f, askId)!.body).toContain(`they wrote "yes"`);
|
||||
@@ -1023,3 +1023,58 @@ describe("a direct answer with no notify envelope (the Slack fallback)", () => {
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
// A person answering from the TUI answers as the host's human seat, and a person-addressed row to a human seat must
|
||||
// carry an evidence ref (convention C3). Without one, both notes to the person were refused and never written.
|
||||
describe("the person's note about a runner answer is written under the queue's own conventions", () => {
|
||||
const HUMAN_SEAT = "human@host";
|
||||
const answerAsHost = (f: ReturnType<typeof fixture>, askId: string, choice: { optionId?: string; decision?: string }) =>
|
||||
f.contract.act({ verb: "answer", qitemId: askId, actorSession: HUMAN_SEAT, authority: "primary-user",
|
||||
...(choice.optionId ? { questionId: "permission", optionId: choice.optionId } : {}), ...(choice.decision ? { decision: choice.decision } : {}) } as never);
|
||||
const notesTo = (f: ReturnType<typeof fixture>) => f.queueRepo.list({ limit: 100 })
|
||||
.filter((row) => row.destinationSession === HUMAN_SEAT && row.humanIntent === "update");
|
||||
|
||||
it("with no agent that can decide, the kept-answer note is written once, linked to the request, with the direct route", async () => {
|
||||
const f = fixture({ decider: null });
|
||||
const askId = String((await f.submit()).body.askId);
|
||||
await answerAsHost(f, askId, { optionId: "allow" });
|
||||
await vi.waitFor(() => expect(notesTo(f)).toHaveLength(1));
|
||||
const [note] = notesTo(f);
|
||||
expect(note).toMatchObject({ qitemId: `qitem-runner-permission-kept-${askId.replace(/^qitem-runner-permission-/, "")}`,
|
||||
evidenceRef: `rig queue show ${askId}`, sourceSession: SEAT });
|
||||
expect(note!.body).toContain("The tool hasn't run, and nothing was permitted or refused.");
|
||||
expect(note!.body).toContain(`rig seat decide-permission ${askId} --option <id>`);
|
||||
// The answer itself is kept on the request, and the request still waits.
|
||||
expect(f.queueRepo.getById(askId)).toMatchObject({ state: "pending", humanAnswers: { permission: "allow" } });
|
||||
// Answering again keeps the new answer and writes no second note.
|
||||
await answerAsHost(f, askId, { decision: "only the test file" });
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(notesTo(f)).toHaveLength(1);
|
||||
expect(f.permits()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("when the answer goes to an agent, the who-decides note is written once, linked to the request, with the agent's state", async () => {
|
||||
const f = fixture({ receiver: "dev-other@perm-rig" });
|
||||
const askId = String((await f.submit()).body.askId);
|
||||
await answerAsHost(f, askId, { decision: "only the test file" });
|
||||
await vi.waitFor(() => expect(notesTo(f)).toHaveLength(1));
|
||||
const [note] = notesTo(f);
|
||||
expect(note).toMatchObject({ evidenceRef: `rig queue show ${askId}`, sourceSession: SEAT });
|
||||
expect(note!.body).toContain(`went to dev-other@perm-rig, which decides it while ${SEAT} waits. Nothing runs until it decides.`);
|
||||
expect(note!.body).toContain("Receiving agent: dev-other@perm-rig, chosen in agents.operator_session; its session is recorded as running.");
|
||||
expect(note!.body).toContain(`rig seat decide-permission ${askId} --option <id>`);
|
||||
expect(f.queueRepo.getById(askId)).toMatchObject({ state: "handed-off", handedOffTo: "dev-other@perm-rig" });
|
||||
});
|
||||
|
||||
it("a runner already waiting when the answer is handed to the agent reads the agent's decision at once, not at its read's end", async () => {
|
||||
const f = fixture({ decider: DECIDER });
|
||||
const askId = String((await f.submit()).body.askId);
|
||||
const started = Date.now();
|
||||
const waiting = f.read(askId, 5);
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(await f.answer(askId, { optionId: "allow" })).toMatchObject({ status: 200 });
|
||||
expect(await f.decide(askId, "allow")).toMatchObject({ status: 200, body: { state: "answered" } });
|
||||
expect(await waiting).toMatchObject({ body: { state: "answered", optionId: "allow" } });
|
||||
expect(Date.now() - started).toBeLessThan(2500);
|
||||
});
|
||||
});
|
||||
@@ -499,7 +499,8 @@ describe("SeatLifecycleService.launchFresh", () => {
|
||||
code: "claimed_session",
|
||||
message: `Seat "${seat.sessionName}" has an adopted/operator-owned occupant; fresh launch refuses even with --stop.`,
|
||||
// B5: rig unclaim releases it (the process keeps running); rig down leaves a session OpenRig didn't start.
|
||||
guidance: "Release it with rig unclaim 'dev-impl@fresh-rig' (its process keeps running), end that process yourself, then launch fresh. "
|
||||
guidance: "Release it with rig unclaim 'dev-impl@fresh-rig' (its process keeps running), end that process and the underlying tmux session, "
|
||||
+ "including any remaining shell, then launch fresh. Closing only its viewing window leaves the tmux session running. "
|
||||
+ "rig down 'fresh-rig' leaves a session OpenRig didn't start running.",
|
||||
};
|
||||
const before = db.serialize();
|
||||
@@ -686,6 +687,11 @@ describe("SeatLifecycleService.launchFresh", () => {
|
||||
alive.add(seat.sessionName);
|
||||
const result = await service.launchFresh({ seatRef: "dev.impl", fresh: true, stop: true, reason: "x" });
|
||||
expect(result).toMatchObject({ ok: false, code: "unmanaged_session_collision" });
|
||||
// A session someone released with rig unclaim and left open is the usual cause: say how to clear it.
|
||||
expect(result).toMatchObject({
|
||||
guidance: "If you released this session with rig unclaim, end that underlying tmux session, including any remaining shell, "
|
||||
+ "then launch again. Closing only its viewing window doesn't end the session.",
|
||||
});
|
||||
expect(killed).toEqual([]);
|
||||
expect(alive.has(seat.sessionName)).toBe(true);
|
||||
});
|
||||
|
||||
@@ -446,6 +446,11 @@ describe("SeatHandoverService source execution (OPR.0.5.5.5)", () => {
|
||||
expect(sessionRegistry.getBindingForNode(node.id)?.tmuxSession).toBe("dev-impl@seat-rig");
|
||||
expect(nodeRow(node.id).handover_result).not.toBe("complete");
|
||||
expect(result.guidance).toMatch(/re-wakeable|binding is unchanged/);
|
||||
// #2985: the harness launch itself failed, so nothing was sent into the pane: the failure keeps the generic
|
||||
// guidance and doesn't claim a successor was launched.
|
||||
expect(result).not.toHaveProperty("replacementStarted");
|
||||
expect(result).not.toHaveProperty("successorPane");
|
||||
expect(result.guidance).not.toContain("a successor was launched");
|
||||
});
|
||||
|
||||
it("rebuild: an induced priming-delivery failure unwinds the successor candidate and reports the step — binding unchanged", async () => {
|
||||
@@ -519,9 +524,143 @@ describe("SeatHandoverService source execution (OPR.0.5.5.5)", () => {
|
||||
expect(sendKeys).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("a seat-pane discovery row claimed by another node still refuses the handover (#237 holds)", async () => {
|
||||
function seedForeignPaneClaim(opts: { archived: boolean; status: "running" | "superseded" | "detached" | "exited" }) {
|
||||
const rig = rigRepo.createRig("seat-rig");
|
||||
const node = rigRepo.addNode(rig.id, "dev.impl", { runtime: "codex", cwd: "/project" });
|
||||
const session = sessionRegistry.registerSession(node.id, "dev-impl@seat-rig");
|
||||
sessionRegistry.updateStatus(session.id, opts.status);
|
||||
sessionRegistry.updateBinding(node.id, { tmuxSession: "dev-impl@seat-rig", tmuxPane: "%9" });
|
||||
const discovered = discoveryRepo.upsertDiscoveredSession({ tmuxSession: "dev-impl@seat-rig", tmuxPane: "%9", runtimeHint: "codex", confidence: "high" });
|
||||
discoveryRepo.markClaimed(discovered.id, node.id);
|
||||
if (opts.archived) rigRepo.archiveRig(rig.id);
|
||||
return { rig, node, session, discovered };
|
||||
}
|
||||
|
||||
it.each(["fresh", "fork:dev-impl@seat-rig", "rebuild"])("%s reclaims the pane from an exited owner in an archived rig (#2861)", async (source) => {
|
||||
const previous = seedForeignPaneClaim({ archived: true, status: "exited" });
|
||||
const { node, sessionId } = seedSeat({ resumeToken: "native-abc" });
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source });
|
||||
|
||||
expect(result).toMatchObject({ ok: true, result: { mutated: true, currentStatus: { handoverResult: "complete" } } });
|
||||
expect(liveSessionId(node.id)).not.toBe(sessionId);
|
||||
expect(sessionRegistry.getBindingForNode(node.id)).toMatchObject({ tmuxSession: "dev-impl@seat-rig", tmuxPane: "%9" });
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)).toMatchObject({ status: "claimed", claimedNodeId: node.id });
|
||||
});
|
||||
|
||||
it.each(["superseded", "detached"] as const)("reclaims the pane from a %s owner in an archived rig (#2861)", async (status) => {
|
||||
const previous = seedForeignPaneClaim({ archived: true, status });
|
||||
const { node } = seedSeat();
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source: "rebuild" });
|
||||
|
||||
expect(result).toMatchObject({ ok: true, result: { mutated: true } });
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)).toMatchObject({ status: "claimed", claimedNodeId: node.id });
|
||||
});
|
||||
|
||||
it("reclaims a pane claim whose owner node was deleted (#2861)", async () => {
|
||||
const previous = seedForeignPaneClaim({ archived: true, status: "exited" });
|
||||
db.prepare("DELETE FROM nodes WHERE id = ?").run(previous.node.id);
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)).toMatchObject({ status: "claimed", claimedNodeId: null });
|
||||
const { node } = seedSeat();
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source: "rebuild" });
|
||||
|
||||
expect(result).toMatchObject({ ok: true, result: { mutated: true } });
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)).toMatchObject({ status: "claimed", claimedNodeId: node.id });
|
||||
});
|
||||
|
||||
it("reclaims a pane recorded as claimed without a claimant (#2861)", async () => {
|
||||
const { node } = seedSeat();
|
||||
const row = discoveryRepo.upsertDiscoveredSession({ tmuxSession: "dev-impl@seat-rig", tmuxPane: "%9", runtimeHint: "codex", confidence: "high" });
|
||||
db.prepare("UPDATE discovered_sessions SET status = 'claimed' WHERE id = ?").run(row.id);
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source: "rebuild" });
|
||||
|
||||
expect(result).toMatchObject({ ok: true, result: { mutated: true } });
|
||||
expect(discoveryRepo.getDiscoveredSession(row.id)).toMatchObject({ status: "claimed", claimedNodeId: node.id });
|
||||
});
|
||||
|
||||
it("an archived claim owner with a newer running session still refuses (#2861)", async () => {
|
||||
const previous = seedForeignPaneClaim({ archived: true, status: "exited" });
|
||||
const running = sessionRegistry.registerSession(previous.node.id, "dev-impl@seat-rig");
|
||||
sessionRegistry.updateStatus(running.id, "running");
|
||||
const { node, sessionId } = seedSeat();
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source: "rebuild" });
|
||||
|
||||
expect(result).toMatchObject({ ok: false, code: "discovered_not_active" });
|
||||
expect(liveSessionId(node.id)).toBe(sessionId);
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)?.claimedNodeId).toBe(previous.node.id);
|
||||
});
|
||||
|
||||
it("an exited owner outside an archived rig still refuses (#2861)", async () => {
|
||||
const { rig, node, sessionId } = seedSeat();
|
||||
const other = rigRepo.addNode(rig.id, "dev.other", { runtime: "codex" });
|
||||
const exited = sessionRegistry.registerSession(other.id, "dev-other@seat-rig");
|
||||
sessionRegistry.updateStatus(exited.id, "exited");
|
||||
const row = discoveryRepo.upsertDiscoveredSession({ tmuxSession: "dev-impl@seat-rig", tmuxPane: "%9", runtimeHint: "codex", confidence: "high" });
|
||||
discoveryRepo.markClaimed(row.id, other.id);
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source: "rebuild" });
|
||||
|
||||
expect(result).toMatchObject({ ok: false, code: "discovered_not_active" });
|
||||
expect(liveSessionId(node.id)).toBe(sessionId);
|
||||
expect(discoveryRepo.getDiscoveredSession(row.id)?.claimedNodeId).toBe(other.id);
|
||||
});
|
||||
|
||||
it("discovered-source adoption does not reclaim an archived owner's pane (#2861)", async () => {
|
||||
const previous = seedForeignPaneClaim({ archived: true, status: "exited" });
|
||||
const { node, sessionId } = seedSeat();
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "adopt", source: `discovered:${previous.discovered.id}` });
|
||||
|
||||
expect(result).toMatchObject({ ok: false, code: "discovered_not_active" });
|
||||
expect(liveSessionId(node.id)).toBe(sessionId);
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)?.claimedNodeId).toBe(previous.node.id);
|
||||
});
|
||||
|
||||
it("does not reclaim when the archived owner resumes during the presence probe (#2861)", async () => {
|
||||
const previous = seedForeignPaneClaim({ archived: true, status: "exited" });
|
||||
const { node, sessionId } = seedSeat();
|
||||
hasSession.mockImplementation(async () => {
|
||||
sessionRegistry.updateStatus(previous.session.id, "running");
|
||||
return true;
|
||||
});
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source: "rebuild" });
|
||||
|
||||
expect(result).toMatchObject({ ok: false, code: "discovered_not_active" });
|
||||
expect(liveSessionId(node.id)).toBe(sessionId);
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)?.claimedNodeId).toBe(previous.node.id);
|
||||
});
|
||||
|
||||
it.each([false, true])("does not overwrite a new claim during the presence probe; original owner deleted=%s (#2861)", async (deleted) => {
|
||||
const previous = seedForeignPaneClaim({ archived: true, status: "exited" });
|
||||
if (deleted) db.prepare("DELETE FROM nodes WHERE id = ?").run(previous.node.id);
|
||||
const { rig, node, sessionId } = seedSeat();
|
||||
const other = rigRepo.addNode(rig.id, "dev.other", { runtime: "codex" });
|
||||
const running = sessionRegistry.registerSession(other.id, "dev-other@seat-rig");
|
||||
sessionRegistry.updateStatus(running.id, "running");
|
||||
hasSession.mockImplementation(async () => {
|
||||
discoveryRepo.markClaimed(previous.discovered.id, other.id);
|
||||
return true;
|
||||
});
|
||||
|
||||
const result = await service.handover({ seatRef: "dev-impl@seat-rig", reason: "context-wall", source: "rebuild" });
|
||||
|
||||
expect(result).toMatchObject({ ok: false, code: "discovered_not_active" });
|
||||
expect(liveSessionId(node.id)).toBe(sessionId);
|
||||
expect(discoveryRepo.getDiscoveredSession(previous.discovered.id)?.claimedNodeId).toBe(other.id);
|
||||
});
|
||||
|
||||
it.each([false, true])("a foreign pane claim refuses with a running session present=%s (#237 holds)", async (runningSession) => {
|
||||
const { rig, node, sessionId } = seedSeat();
|
||||
const other = rigRepo.addNode(rig.id, "dev.other", { runtime: "codex", cwd: "/project" });
|
||||
if (runningSession) {
|
||||
const running = sessionRegistry.registerSession(other.id, "dev-other@seat-rig");
|
||||
sessionRegistry.updateStatus(running.id, "running");
|
||||
}
|
||||
const row = discoveryRepo.upsertDiscoveredSession({ tmuxSession: "dev-impl@seat-rig", tmuxPane: "%9", runtimeHint: "codex", confidence: "high" });
|
||||
discoveryRepo.markClaimed(row.id, other.id);
|
||||
|
||||
|
||||
@@ -447,7 +447,7 @@ describe("SeatLifecycleService", () => {
|
||||
// the refusal names the supported route, rig unclaim, and says rig down doesn't stop a session OpenRig didn't start,
|
||||
// even in a one-seat rig.
|
||||
expect(claimedResult.guidance).toBe("To end this session, run rig unclaim 'dev-adopted@s5-rig' (it releases the session from OpenRig "
|
||||
+ "and leaves its process running), then end that process in its own terminal. rig down 's5-rig' leaves a session OpenRig didn't start running.");
|
||||
+ "and leaves its process running), then end that process and the underlying tmux session, including any remaining shell, so no session keeps the seat's name. Closing only its viewing window leaves the tmux session running. rig down 's5-rig' leaves a session OpenRig didn't start running.");
|
||||
|
||||
const probed = seatFixture("s5-rig", "dev.flaky");
|
||||
tmux.failProbeFor(probed.sessionName);
|
||||
@@ -471,7 +471,7 @@ describe("SeatLifecycleService", () => {
|
||||
const shared = await service.stopSeat({ seatRef: claimed.sessionName, reason: "x" });
|
||||
if (shared.ok) throw new Error("expected refusal");
|
||||
expect(shared.guidance).toBe("To end this session, run rig unclaim 'dev-adopted@adopted-rig' (it releases the session from OpenRig "
|
||||
+ "and leaves its process running), then end that process in its own terminal. rig down 'adopted-rig' leaves a session OpenRig didn't start running.");
|
||||
+ "and leaves its process running), then end that process and the underlying tmux session, including any remaining shell, so no session keeps the seat's name. Closing only its viewing window leaves the tmux session running. rig down 'adopted-rig' leaves a session OpenRig didn't start running.");
|
||||
|
||||
sessionRegistry.updateBinding(claimed.node.id, { attachmentType: "external_cli", externalSessionName: claimed.sessionName });
|
||||
const external = await service.stopSeat({ seatRef: claimed.sessionName, reason: "x" });
|
||||
@@ -485,7 +485,7 @@ describe("SeatLifecycleService", () => {
|
||||
|
||||
it("claimedSessionRoutes routes a tmux session through rig unclaim, never rig down", () => {
|
||||
expect(claimedSessionRoutes("r", "a@r")).toBe("To end this session, run rig unclaim 'a@r' (it releases the session from OpenRig "
|
||||
+ "and leaves its process running), then end that process in its own terminal. rig down 'r' leaves a session OpenRig didn't start running.");
|
||||
+ "and leaves its process running), then end that process and the underlying tmux session, including any remaining shell, so no session keeps the seat's name. Closing only its viewing window leaves the tmux session running. rig down 'r' leaves a session OpenRig didn't start running.");
|
||||
});
|
||||
|
||||
// ---- P5 / P6 — clean ----
|
||||
|
||||
@@ -636,10 +636,10 @@ describe("rig seat restart --keep-conversation: gates before any signal", () =>
|
||||
w.db.prepare("UPDATE sessions SET origin = 'claimed' WHERE id = ?").run(w.session.id);
|
||||
const claimed = await w.service.restartInPlace({ seatRef: SEAT, plan: true });
|
||||
expect(claimed).toMatchObject({ ok: false, code: "claimed_session" });
|
||||
// B5: the refusal's route is rig unclaim, then ending the process in its own terminal; rig down leaves a session
|
||||
// B5: the refusal's route is rig unclaim, then ending the process and its tmux session; rig down leaves a session
|
||||
// OpenRig didn't start running, so it isn't offered as the way to end this one.
|
||||
const guidance = (claimed as { guidance?: string }).guidance ?? "";
|
||||
expect(guidance).toMatch(/^To end this session, run rig unclaim '[^']+' \(it releases the session from OpenRig and leaves its process running\), then end that process in its own terminal\. /);
|
||||
expect(guidance).toMatch(/^To end this session, run rig unclaim '[^']+' \(it releases the session from OpenRig and leaves its process running\), then end that process and the underlying tmux session, including any remaining shell, so no session keeps the seat's name\. Closing only its viewing window leaves the tmux session running\. /);
|
||||
expect(guidance).toMatch(/rig down '[^']+' leaves a session OpenRig didn't start running\.$/);
|
||||
expect(guidance).not.toMatch(/run rig down/);
|
||||
});
|
||||
|
||||
@@ -320,6 +320,8 @@ describe("SuccessorSessionLauncher", () => {
|
||||
|
||||
expect(res).toMatchObject({ ok: false, step: "start_agent", code: "successor_launch_failed" });
|
||||
expect((res as { message: string }).message).toContain("codex binary not found");
|
||||
// #2985: the launch itself failed, so no successor was sent into the pane.
|
||||
expect(res).not.toHaveProperty("successorLaunched");
|
||||
// The preserved seat's session is NEVER killed on unwind (retiree state stays recoverable).
|
||||
expect(killSession).not.toHaveBeenCalled();
|
||||
// No discovery candidate was committed for a failed successor.
|
||||
@@ -330,6 +332,8 @@ describe("SuccessorSessionLauncher", () => {
|
||||
checkReady.mockResolvedValue({ ready: false, reason: "harness not interactive" });
|
||||
const res = await launcher().createSuccessor({ node: { id: "n", runtime: "codex", cwd: "/w" }, departingSessionName: "a@r" });
|
||||
expect(res).toMatchObject({ ok: false, step: "start_agent", code: "successor_not_ready" });
|
||||
// #2985: the harness launch was sent and only readiness failed: the successor may still come up in this pane.
|
||||
expect(res).toMatchObject({ replacementStarted: true, successorLaunched: true, tmuxPane: "%7" });
|
||||
expect(killSession).not.toHaveBeenCalled();
|
||||
expect(discoveryRepo.listDiscovered()).toHaveLength(0);
|
||||
});
|
||||
@@ -463,6 +467,8 @@ describe("SuccessorSessionLauncher", () => {
|
||||
checkReady.mockRejectedValue(new Error("tmux socket closed"));
|
||||
const res = await launcher().createSuccessor({ node: { id: "n", runtime: "codex", cwd: "/w" }, departingSessionName: "a@r" });
|
||||
expect(res).toMatchObject({ ok: false, step: "start_agent", code: "successor_readiness_failed" });
|
||||
// #2985: the launch was sent before the probe threw, so the successor may still be in the pane.
|
||||
expect(res).toMatchObject({ replacementStarted: true, successorLaunched: true, tmuxPane: "%7" });
|
||||
expect((res as { message: string }).message).toContain("tmux socket closed");
|
||||
expect(checkReady).toHaveBeenCalledTimes(1);
|
||||
expect(killSession).not.toHaveBeenCalled();
|
||||
@@ -473,6 +479,7 @@ describe("SuccessorSessionLauncher", () => {
|
||||
checkReady.mockResolvedValue({ ready: false, code: "trust_gate", reason: "trust prompt" });
|
||||
const res = await launcher().createSuccessor({ node: { id: "n", runtime: "codex", cwd: "/w" }, departingSessionName: "a@r" });
|
||||
expect(res).toMatchObject({ ok: false, step: "start_agent", code: "successor_attention_required" });
|
||||
expect(res).toMatchObject({ replacementStarted: true, successorLaunched: true, tmuxPane: "%7" });
|
||||
expect(checkReady).toHaveBeenCalledTimes(1);
|
||||
expect(killSession).not.toHaveBeenCalled();
|
||||
expect(discoveryRepo.listDiscovered()).toHaveLength(0);
|
||||
|
||||
@@ -11,6 +11,46 @@ import { SeatLaunchEnvironment } from "../src/domain/seat-launch-environment.js"
|
||||
import { shellQuote } from "../src/adapters/shell-quote.js";
|
||||
import { NO_SESSION_BUS } from "../src/domain/seat-runtime-dir.js";
|
||||
const exec = promisify(execFile);
|
||||
async function respawnExitedFixture(tmux: (args: string[]) => Promise<string>): Promise<string> {
|
||||
// macOS CI sometimes refuses forkpty for this disposable setup command. The real adapter respawn below is never
|
||||
// retried: preserve all #1178 assertions, and retry only this exact pre-fork refusal, at most twice (50 ms total).
|
||||
for (let attempt = 0; ; attempt++) {
|
||||
try { return await tmux(["respawn-pane", "-k", "-t", "=seat:", "/bin/sh", "-c", "exit"]); }
|
||||
catch (error) {
|
||||
const stderr = (error as { stderr?: string }).stderr?.trim();
|
||||
if (process.platform !== "darwin" || attempt >= 2 || stderr !== "respawn pane failed: fork failed: Device not configured") throw error;
|
||||
console.warn(`macOS fixture PTY allocation refused; retrying owned setup respawn (${attempt + 1}/2).`);
|
||||
await new Promise(resolve => setTimeout(resolve, 25));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
describe.skipIf(process.platform !== "darwin")("macOS fixture PTY allocation", () => {
|
||||
const noPty = () => Object.assign(new Error("fixture command failed"), { stderr: "respawn pane failed: fork failed: Device not configured\n" });
|
||||
it("retries only the owned setup respawn's allocation error, then still performs the setup", async () => {
|
||||
const calls: string[][] = [];
|
||||
await respawnExitedFixture(async args => {
|
||||
calls.push(args);
|
||||
if (calls.length < 3) throw noPty();
|
||||
return "";
|
||||
});
|
||||
expect(calls).toEqual(Array(3).fill(["respawn-pane", "-k", "-t", "=seat:", "/bin/sh", "-c", "exit"]));
|
||||
});
|
||||
it("stops after three refused allocations and preserves the last error", async () => {
|
||||
const failure = noPty();
|
||||
let calls = 0;
|
||||
await expect(respawnExitedFixture(async () => { calls++; throw failure; })).rejects.toBe(failure);
|
||||
expect(calls).toBe(3);
|
||||
});
|
||||
it("does not retry a different error or an allocation that already succeeded", async () => {
|
||||
const failure = Object.assign(new Error("fixture command failed"), { stderr: "can't find pane: seat" });
|
||||
let calls = 0;
|
||||
await expect(respawnExitedFixture(async () => { calls++; throw failure; })).rejects.toBe(failure);
|
||||
expect(calls).toBe(1);
|
||||
await respawnExitedFixture(async () => { calls++; return ""; });
|
||||
expect(calls).toBe(2);
|
||||
});
|
||||
});
|
||||
const waitFile = async (file: string) => {
|
||||
// Redirection creates the file before printf writes it; the final newline marks completion.
|
||||
for (let i = 0; i < 100; i++) {
|
||||
@@ -65,7 +105,7 @@ describe.skipIf(process.platform === "win32")("#1178 initial-shell environment p
|
||||
if (successor) {
|
||||
await tmux(["new-session","-d","-s","seat","sleep 60"]);
|
||||
await tmux(["set-window-option","-t","=seat:","remain-on-exit","on"]);
|
||||
await tmux(["respawn-pane","-k","-t","=seat:","/bin/sh","-c","exit"]);
|
||||
await respawnExitedFixture(tmux);
|
||||
for(let i=0;i<100 && await tmux(["display-message","-p","-t","=seat:","#{pane_dead}"])!=="1";i++) await new Promise(r=>setTimeout(r,25));
|
||||
expect((await adapter.respawnPane(await tmux(["display-message","-p","-t","=seat:","#{pane_id}"]),custom,{cwd:home,env:selected})).ok).toBe(true);
|
||||
} else expect((await adapter.createSession("seat",home,selected)).ok).toBe(true);
|
||||
@@ -194,7 +234,7 @@ describe.skipIf(process.platform === "win32")("#1178 initial-shell environment p
|
||||
if (mode==="respawn") {
|
||||
await tmux(["new-session","-d","-s","seat","sleep 60"]);
|
||||
await tmux(["set-window-option","-t","=seat:","remain-on-exit","on"]);
|
||||
await tmux(["respawn-pane","-k","-t","=seat:","/bin/sh","-c","exit"]);
|
||||
await respawnExitedFixture(tmux);
|
||||
for(let i=0;i<100 && await tmux(["display-message","-p","-t","=seat:","#{pane_dead}"])!=="1";i++) await new Promise(r=>setTimeout(r,25));
|
||||
expect((await adapter.respawnPane(await tmux(["display-message","-p","-t","=seat:","#{pane_id}"]),undefined,{cwd:home,env:selected})).ok).toBe(true);
|
||||
} else expect((await adapter.createSession("seat",home,selected)).ok).toBe(true);
|
||||
|
||||
@@ -688,7 +688,7 @@ export async function hydrateSnapshot(
|
||||
// answer where there is one, else the registry's probe. Only a drilled-into remote host's rigs are read, through this
|
||||
// daemon's read-only forwarding: their names and states, never their seats. A registry entry for this host is this host.
|
||||
const registered = (hostsRead?.hosts ?? []).flatMap((h) => typeof h.id === "string" && h.id && h.id !== "local"
|
||||
? [{ id: h.id, reachable: h.status === "reachable" }] : []);
|
||||
? [{ id: h.id, reachable: h.status === "reachable" ? true : h.status === "unreachable" ? false : null }] : []);
|
||||
const aggStatus = new Map(aggHosts.filter((h) => h.hostId !== "local").map((h) => [h.hostId, h.status]));
|
||||
const remoteIds = [...new Set([...aggStatus.keys(), ...registered.map((h) => h.id)])].filter((id) => id !== localHost.name);
|
||||
const drilledRemote = remoteDrill && drilledHost && remoteIds.includes(drilledHost) ? drilledHost : null;
|
||||
@@ -698,7 +698,9 @@ export async function hydrateSnapshot(
|
||||
const remoteHosts: HostNode[] = remoteIds.map((id) => ({
|
||||
id,
|
||||
name: id,
|
||||
reachable: aggStatus.has(id) ? aggStatus.get(id) === "ok" : registered.some((h) => h.id === id && h.reachable),
|
||||
reachable: aggStatus.has(id)
|
||||
? aggStatus.get(id) === "ok" ? true : aggStatus.get(id) === "unreachable" ? false : null
|
||||
: registered.find((h) => h.id === id)?.reachable ?? null,
|
||||
rigs: id === drilledRemote
|
||||
? (remoteRigs ?? []).map((rig) => ({ id: rig.id, name: rig.name, lifecycleState: rig.lifecycleState, hasLiveAgents: rig.hasLiveAgents ?? null, pods: [], remoteHost: id })).sort(byRigName)
|
||||
: [],
|
||||
|
||||
@@ -103,7 +103,7 @@ function agentOf(parsed: KeyParts, snap: FleetSnapshot): AgentRow | null {
|
||||
/** Use structured identity/presence, never the possibly clipped status suffix. */
|
||||
function rigPresence(parsed: KeyParts, snap: FleetSnapshot): boolean | null {
|
||||
const host = snap.hosts.find(h => h.name === parsed.parts[0]);
|
||||
if (!host?.reachable) return null;
|
||||
if (host?.reachable !== true) return null;
|
||||
return host.rigs.find(r => r.name === parsed.parts[1])?.hasLiveAgents ?? null;
|
||||
}
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ import { buildLedgerExplorer } from "./crash-cart/ledger-explorer.js";
|
||||
import type { CrashCartModel } from "./crash-cart/crash-cart-model.js";
|
||||
import type { DaemonState, DaemonUnverifiedEvidence } from "./crash-cart/contract.js";
|
||||
import { runtimeMarkSegs } from "./topology/runtime-marks.js";
|
||||
import { hostLabel } from "./topology/host-label.js";
|
||||
import { hostLabel, hostReachabilityLabel } from "./topology/host-label.js";
|
||||
import { barCells, flashActive, reducedMotion, spinnerFrame } from "./motion.js";
|
||||
import { explorerWidth, MOTION_FRAME_MS } from "./visual-layout.js";
|
||||
import type { ColorMode, Token } from "./theme.js";
|
||||
@@ -788,7 +788,7 @@ function instanceContentLines(
|
||||
...lines,
|
||||
healthSummaryLine(snap, scope, contentWidth),
|
||||
...wrapDetailLines([
|
||||
{ text: `host ${host.name} · ${host.reachable ? "reachable" : "not reachable"}` },
|
||||
{ text: `host ${host.name} · ${hostReachabilityLabel(host)}` },
|
||||
{ text: `Its seats aren't read in this view: rig ps --host ${host.name} --nodes -A` },
|
||||
], contentWidth),
|
||||
{ text: "" },
|
||||
|
||||
Loaded 100 of 107 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user