diff --git a/.agents/skills/developing-openrig/SKILL.md b/.agents/skills/developing-openrig/SKILL.md new file mode 100644 index 000000000..b75de62e5 --- /dev/null +++ b/.agents/skills/developing-openrig/SKILL.md @@ -0,0 +1,66 @@ +--- +name: developing-openrig +description: "Use when changing OpenRig's own source in a clone of the openrig repository: finding which package or file owns a behaviour, choosing which tests to run, testing a change without disturbing the OpenRig daemon your own session runs on, judging whether a diff touches a high-risk area, or preparing a pull request. Not for operating rigs (openrig-skills) or designing rig topologies." +--- + +# Developing OpenRig + +This skill is a map to the other maps. Each one is incomplete and may be stale: where a map and the code disagree, the +code is right, and fixing the map is a welcome contribution. + +## First: which OpenRig are you in? + +If you are an agent running inside OpenRig, two different OpenRigs are in play: + +- **The installed one runs your session.** `rig --version` and `rig daemon status` describe it. +- **The checkout is what you're changing.** `git rev-parse HEAD` describes it. + +Restarting the installed daemon does not run your checkout, and stopping it stops the agents running on it, including +you. Test your change with the repo's own harnesses, which start a private daemon and tmux server (see "Test it" below), +not by restarting the daemon you live in. + +## Find your way + +| You want to know | Read | How much to trust it | +|---|---|---| +| How the packages fit, the request path, where to add a command, route, migration, adapter, skill, context pack or scenario | [`ARCHITECTURE.md`](../../../ARCHITECTURE.md) | Checked against the commit it names; counts come with the command to refresh them | +| Whether your change touches a high-risk area, what depends on it, what broke there before | [`docs/as-built/arteries.md`](../../../docs/as-built/arteries.md) | Incomplete by design: absence from it doesn't make a change safe | +| What to run before you push, and what each layer can and can't prove | [`docs/as-built/test-layers.md`](../../../docs/as-built/test-layers.md) | Checked against the commit it names | +| The exact behaviour of a `rig` command | `rig --help` on the running binary, then `packages/cli/src/commands/` | The binary and the code win over any document | +| A subsystem in depth | [`docs/as-built/`](../../../docs/as-built/README.md) | Most modules predate 0.6; check each one's `last-verified-against-source` marker against `git log` before relying on it | +| Rig spec, agent spec and other user-facing reference | [`docs/reference/`](../../../docs/reference/) | Ships to every user; if you find drift, fix it there | +| How to open a good pull request, and what review looks like | [`CONTRIBUTING.md`](../../../CONTRIBUTING.md) | Current | + +If none of these answers your question, search the code before inventing an answer. A missing map is worth an issue. + +## Before you change an artery + +If your diff touches anything in `arteries.md` (message delivery, launch and resume, the queue, rig identity, skill +projection, process observation, migrations, restore): + +1. Read the artery row: what depends on it, and the past regressions listed there. +2. Describe the downstream effect in your pull request, not just the diff. +3. Add or extend a stub-agent scenario, or say precisely why the stub can't exercise it and what you ran instead. + +## Test it + +The short version is `npm run build`, `npm run lint`, `npm test`. `test-layers.md` has the full ladder, including the +stub-agent scenarios, which run the real CLI, daemon, tmux and SQLite with scripted agents and no model cost. + +- The scenario runner starts its own daemon and tmux server and drops your session's `TMUX` and daemon variables, so + it won't touch the daemon your session runs on. +- A stub proves OpenRig's own plumbing, not how Claude Code or Codex behave. Say which you exercised. + +## How we judge changes + +OpenRig is a coordination layer for a trusted environment: it should help agents and people keep work flowing. + +- A change that removes friction, or makes state more truthful, is usually welcome. +- A change that adds a refusal, a prompt or a required step needs the concrete case in CONTRIBUTING.md: who is + harmed, how, and what it costs everyone else. +- One concern per pull request. Keep the diff reviewable in one sitting. + +## When the map is wrong + +Fix it in the same pull request, or open an issue. Update a document's `last-verified-against-source` marker only when +you have actually checked it against that commit. diff --git a/.claude/skills/developing-openrig/SKILL.md b/.claude/skills/developing-openrig/SKILL.md new file mode 100644 index 000000000..b75de62e5 --- /dev/null +++ b/.claude/skills/developing-openrig/SKILL.md @@ -0,0 +1,66 @@ +--- +name: developing-openrig +description: "Use when changing OpenRig's own source in a clone of the openrig repository: finding which package or file owns a behaviour, choosing which tests to run, testing a change without disturbing the OpenRig daemon your own session runs on, judging whether a diff touches a high-risk area, or preparing a pull request. Not for operating rigs (openrig-skills) or designing rig topologies." +--- + +# Developing OpenRig + +This skill is a map to the other maps. Each one is incomplete and may be stale: where a map and the code disagree, the +code is right, and fixing the map is a welcome contribution. + +## First: which OpenRig are you in? + +If you are an agent running inside OpenRig, two different OpenRigs are in play: + +- **The installed one runs your session.** `rig --version` and `rig daemon status` describe it. +- **The checkout is what you're changing.** `git rev-parse HEAD` describes it. + +Restarting the installed daemon does not run your checkout, and stopping it stops the agents running on it, including +you. Test your change with the repo's own harnesses, which start a private daemon and tmux server (see "Test it" below), +not by restarting the daemon you live in. + +## Find your way + +| You want to know | Read | How much to trust it | +|---|---|---| +| How the packages fit, the request path, where to add a command, route, migration, adapter, skill, context pack or scenario | [`ARCHITECTURE.md`](../../../ARCHITECTURE.md) | Checked against the commit it names; counts come with the command to refresh them | +| Whether your change touches a high-risk area, what depends on it, what broke there before | [`docs/as-built/arteries.md`](../../../docs/as-built/arteries.md) | Incomplete by design: absence from it doesn't make a change safe | +| What to run before you push, and what each layer can and can't prove | [`docs/as-built/test-layers.md`](../../../docs/as-built/test-layers.md) | Checked against the commit it names | +| The exact behaviour of a `rig` command | `rig --help` on the running binary, then `packages/cli/src/commands/` | The binary and the code win over any document | +| A subsystem in depth | [`docs/as-built/`](../../../docs/as-built/README.md) | Most modules predate 0.6; check each one's `last-verified-against-source` marker against `git log` before relying on it | +| Rig spec, agent spec and other user-facing reference | [`docs/reference/`](../../../docs/reference/) | Ships to every user; if you find drift, fix it there | +| How to open a good pull request, and what review looks like | [`CONTRIBUTING.md`](../../../CONTRIBUTING.md) | Current | + +If none of these answers your question, search the code before inventing an answer. A missing map is worth an issue. + +## Before you change an artery + +If your diff touches anything in `arteries.md` (message delivery, launch and resume, the queue, rig identity, skill +projection, process observation, migrations, restore): + +1. Read the artery row: what depends on it, and the past regressions listed there. +2. Describe the downstream effect in your pull request, not just the diff. +3. Add or extend a stub-agent scenario, or say precisely why the stub can't exercise it and what you ran instead. + +## Test it + +The short version is `npm run build`, `npm run lint`, `npm test`. `test-layers.md` has the full ladder, including the +stub-agent scenarios, which run the real CLI, daemon, tmux and SQLite with scripted agents and no model cost. + +- The scenario runner starts its own daemon and tmux server and drops your session's `TMUX` and daemon variables, so + it won't touch the daemon your session runs on. +- A stub proves OpenRig's own plumbing, not how Claude Code or Codex behave. Say which you exercised. + +## How we judge changes + +OpenRig is a coordination layer for a trusted environment: it should help agents and people keep work flowing. + +- A change that removes friction, or makes state more truthful, is usually welcome. +- A change that adds a refusal, a prompt or a required step needs the concrete case in CONTRIBUTING.md: who is + harmed, how, and what it costs everyone else. +- One concern per pull request. Keep the diff reviewable in one sitting. + +## When the map is wrong + +Fix it in the same pull request, or open an issue. Update a document's `last-verified-against-source` marker only when +you have actually checked it against that commit. diff --git a/.gitignore b/.gitignore index ab7d34366..4e37f94f1 100644 --- a/.gitignore +++ b/.gitignore @@ -22,8 +22,14 @@ notes/* *.sqlite-wal *.sqlite.pre-reboot-backup .openrig/ -.claude -.agents +.claude/* +!.claude/skills/ +.claude/skills/* +!.claude/skills/developing-openrig/ +.agents/* +!.agents/skills/ +.agents/skills/* +!.agents/skills/developing-openrig/ .openrig .worktrees/ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 000000000..e3c8ab1b0 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,327 @@ +# OpenRig architecture + +> **This map is not the territory.** It is a zoomed-out guide for finding your way around the +> code. It is incomplete by design and it may be stale. Where it disagrees with the code, the code +> is right; please fix the page. Every path, symbol, count and command below was checked against +> commit `1347d825` (`v0.6.3-33-g1347d825c`). Each count shows the command that produced it, so you +> can re-run it. + +## What OpenRig is + +OpenRig runs a team of coding agents on your machine as one system. A local daemon keeps the team's +state in SQLite and runs each agent as an ordinary Claude Code or Codex session (or a plain +terminal, a Pi runner, or a scripted test stub) inside tmux. People and agents drive it through the +`rig` CLI, a terminal UI (TUI) and an MCP server, all of which talk to the daemon over local HTTP. +The agents themselves are unmodified. What OpenRig adds is the coordination around them: launching +and restoring agents, delivering messages, keeping durable work in a queue, and giving each agent +the right skills and context. + +## Packages + +| Directory | Package | What it is | How it ships | +|---|---|---|---| +| `packages/daemon` | `@openrig/daemon` (private) | The daemon: HTTP routes, domain services, SQLite migrations, runtime adapters, built-in specs, skills, plugins, static context-pack sources | Copied into `@openrig/cli` | +| `packages/cli` | `@openrig/cli` | The `rig` command, the MCP server, daemon start and stop | The only published package | +| `packages/tui` | `@openrig/tui` (private) | The terminal UI | Copied into `@openrig/cli`; opened by `rig tui`, by bare `rig` in a terminal, or by the `openrig-tui` bin | +| `packages/ui` | `@openrig/ui` (private) | The React web UI, in **maintenance mode**: it still ships and runs, but gets no new feature work. The CLI and TUI are the supported surfaces | Built files copied into `@openrig/cli` and served by the daemon | +| `packages/test-system` | none (not an npm workspace) | The stub-agent scenario library, the CI scenario harness, live-model eval cases | Not shipped | + +**What bundles what.** `packages/cli/package.json` publishes `LICENSE`, `dist`, `daemon`, `ui`, +`tui` and `scripts`. `scripts/build-package.sh` (the CLI's `prepublishOnly`) builds the daemon, UI, +TUI and CLI, then assembles them inside `packages/cli/`: the daemon's `dist`, `assets`, `specs` and +`policies`, the generated context packs and a copy of `docs/reference/` go under `daemon/`; the UI +build goes to `ui/dist`; the TUI build goes to `tui/dist`. So `docs/reference/` ships to every +user; `docs/as-built/` and this file do not. + +**Sharing code between packages.** The CLI and TUI import daemon code only through the subpaths in +the `exports` field of `packages/daemon/package.json` (18 at this commit), for example +`@openrig/daemon/attention`, which points at `packages/daemon/src/attention-surface.ts`. The CLI +uses 16 of them and the TUI uses 2. In development they resolve through the npm workspace. At +package time `scripts/rewrite-daemon-imports.mjs` rewrites them to the shipped `daemon/dist` copy +and fails the build on a specifier with no `exports` entry. To share new daemon code, add a +`*-surface.ts` file and an `exports` entry. + +## The request path + +``` + rig (CLI) TUI MCP server (rig mcp serve, stdio) + \ | / + +---- HTTP to the daemon (port 7433 by default) --------------+ + | + packages/daemon/src/server.ts createApp(): context middleware, /healthz, app.route("/api/...") + | + packages/daemon/src/routes/ one Hono router per area; reads services with c.get(...) + | + packages/daemon/src/domain/ services, repositories, orchestrators, event bus + / | \ + SQLite (db/) tmux (adapters/tmux.ts) runtime adapters (adapters/*-adapter.ts) + claude-code, codex, pi, terminal, stub +``` + +**Inside the daemon** (`packages/daemon/src/`): + +- **Startup.** `index.ts` is the process entry: it resolves where to listen, starts the server and + runs the periodic queue sweeps. It calls `createDaemon()` in `startup.ts`, which opens the + database (`db/connection.ts`: WAL mode, foreign keys on), runs `migrate(db, ALL_MIGRATIONS)`, + constructs every service and adapter, and passes them to `createAppWithWebSocket(deps)` in + `server.ts`. +- **`server.ts`.** In `createApp(deps)`, the first `app.use("*", ...)` middleware puts each + service on the request context (`c.set("" as never, deps.)`), serves `/healthz`, and + mounts each router with `app.route("/api/", ...)`. Unknown `/api/*` paths get a JSON 404; + other GET requests serve the web UI's built files. +- **`routes/`.** Thin handlers. For example, `routes/ps.ts` reads `psProjectionService` from the + context and returns its entries as JSON. `routes/terminal-ws.ts` registers the terminal + WebSocket, and `routes/require-sender-identity.ts` is a shared helper rather than a router. +- **`domain/`.** The behaviour: repositories over SQLite (`rig-repository.ts`, + `queue-repository.ts`, `session-registry.ts`), orchestration (`startup-orchestrator.ts`, + `node-launcher.ts`, `rigspec-instantiator.ts`), message delivery (`session-transport.ts`, + `seat-delivery-guard.ts`) and events (`event-bus.ts`). Larger areas have their own + subdirectories, such as `gateway/`, `provider/`, `policies/`, `scope/`, `context-packs/` and + `mission-control/`. +- **`adapters/`.** The outside world. `tmux.ts` wraps tmux. The five `RuntimeAdapter` + implementations launch each runtime and project skills and files into it. `cmux.ts` and + `compose-services-adapter.ts` cover other integrations. +- **`db/`.** `connection.ts`, `migrate.ts`, `all-migrations.ts` and `migrations/`. +- **`middleware/auth-bearer-token.ts`.** The bearer-token check for operator write routes, plus + the loopback and Tailscale bind detection used at startup. +- **Live updates** go out as server-sent events: `routes/events.ts` uses Hono's `streamSSE`, and + the TUI subscribes to `/api/activity/events`. +- **State** lives in `$OPENRIG_HOME` (default `~/.openrig`). The database is `openrig.sqlite` + there (`daemon-db-path.ts`). + +**The clients:** + +- **CLI.** `createProgram()` in `packages/cli/src/index.ts` registers every top-level command; + the commands live in `packages/cli/src/commands/`. Most are thin clients: they call a daemon route + through `DaemonClient` (`packages/cli/src/client.ts`) and render the result. A few lifecycle + helpers open the database file directly. Bare `rig` in a terminal opens the TUI + (`packages/cli/src/front-door.ts`). +- **MCP.** `createMcpServer(client)` in `packages/cli/src/mcp-server.ts` registers the tools over + the same `DaemonClient`. `rig mcp serve` (`packages/cli/src/commands/mcp.ts`) runs it on stdio. +- **TUI.** `packages/tui/src/daemon-client.ts` is its only HTTP module. It reads existing daemon + projections, for example `/api/ps`, `/api/rigs/summary` and `/api/queue/list`, and the + `/api/activity/events` stream when the daemon offers it. The daemon URL comes from `--url`, + `OPENRIG_URL`, or `http://127.0.0.1:7433`. Agents can drive a running TUI through its control + socket (`packages/tui/src/socket-server.ts`). +- **Web UI.** Served by the daemon from the bundled `ui/dist`. + +**In a development checkout,** `rig daemon start` prefers `packages/daemon/dist` over the bundled +copy (`resolveDaemonPath` in `packages/cli/src/daemon-lifecycle.ts`), and the TUI launcher prefers +`packages/tui/dist` (`packages/cli/src/front-door.ts`). Rebuild the package you changed. A daemon +that is already running keeps running the code it started with. + +## Key counts + +At commit `1347d825`. Run these from the repository root to refresh them. + +| What | Count | Command | +|---|---|---| +| Database migrations | 89 (latest: `089_classification_identity_provenance.ts`) | `git ls-files packages/daemon/src/db/migrations \| wc -l` | +| Files in `routes/` | 67 | `git ls-files packages/daemon/src/routes \| wc -l` | +| ... of which create a Hono router | 65 | `git grep -l 'new Hono' -- packages/daemon/src/routes \| wc -l` | +| `app.route(...)` mounts in `server.ts` | 69 | `grep -c 'app.route(' packages/daemon/src/server.ts` | +| Top-level `rig` commands | 85 | `grep -c 'program.addCommand(' packages/cli/src/index.ts` | +| Runtime adapters | 5 (`claude-code`, `codex`, `pi`, `terminal`, `stub`) | `git grep -l 'implements RuntimeAdapter' packages/daemon/src \| wc -l` | +| MCP tools | 18 | `grep -c 'server.tool(' packages/cli/src/mcp-server.ts` | +| Daemon `exports` subpaths | 18 | `node -p 'Object.keys(require("./packages/daemon/package.json").exports).length'` | +| Library scenarios | 11 | `grep -l '^scenario:' packages/test-system/scenarios/*.yaml \| wc -l` | + +## Where to add things + +Each recipe names the files to touch. Read a neighbouring example before you start; the code is +the authority. + +### A CLI command + +1. Create `packages/cli/src/commands/.ts` exporting a function that returns a commander + `Command` and takes optional injected dependencies. `packages/cli/src/commands/unarchive.ts` + is a short example. +2. In the action, get the daemon with `getDaemonStatus(...)` and `daemonStatusGuard(status)` + (`packages/cli/src/daemon-lifecycle.ts`), then call it with + `new DaemonClient(getDaemonUrl(status))` (`packages/cli/src/client.ts`). Offer `--json` for + agents, and set `process.exitCode` on failure. +3. Register it in `createProgram()` in `packages/cli/src/index.ts` with + `program.addCommand(...)`. If it takes injected dependencies, add them to `ProgramDeps` in the + same file. +4. Add `packages/cli/test/.test.ts`. `packages/cli/test/archive.test.ts` shows how tests + inject mocked lifecycle dependencies. +5. If the command needs new daemon behaviour, add a route as well. + +### A daemon route + +1. Create `packages/daemon/src/routes/.ts` with a Hono router. Both styles exist: a + constant (`export const psRoutes = new Hono()` in `routes/ps.ts`) and a factory + (`whoamiRoutes()` in `routes/whoami.ts`). +2. Read services from the request context with `c.get("" as never)`. Keep the behaviour in + `packages/daemon/src/domain/`. +3. Mount it in `createApp()` in `packages/daemon/src/server.ts` with + `app.route("/api/", ...)`, above the `/api/*` 404 catch-all. +4. A new service needs three more edits: a field on `AppDeps` and a `c.set(...)` line in + `server.ts`, and its construction in `createDaemon()` in `packages/daemon/src/startup.ts`. +5. Test it the way `packages/daemon/test/whoami-routes.test.ts` does: build a small Hono app, + `c.set` the service, mount the router and call `app.request(...)`. `createFullTestDb()` in + `packages/daemon/test/helpers/test-app.ts` gives you an in-memory database migrated with the + test fixture list (`migrationsForFullTestDb`). + +### A database migration + +1. Create `packages/daemon/src/db/migrations/_.ts` exporting a `Migration` + (`{ name: "_.sql", sql: "..." }`; the type is in `packages/daemon/src/db/migrate.ts`). + `089_classification_identity_provenance.ts` is a one-line example. +2. Take the next number after the highest on `main` (089 at this commit). Parallel pull requests + often pick the same number, so check again and renumber if `main` moved before you merge. +3. Import it in `packages/daemon/src/db/all-migrations.ts` and append it to `ALL_MIGRATIONS`. + Startup runs exactly that list (`startup-migrations-mirror.test.ts` pins this). +4. Add it to `migrationsForFullTestDb` in `packages/daemon/test/helpers/test-app.ts`, or list it in + `migrationsForFullTestDbExclusions` with a reason. `migration-fixture-parity.test.ts` fails + otherwise. +5. `migrate()` sorts migrations by name and records each applied name in `schema_migrations`, so + an install that already applied a migration never runs it again. Editing a migration that has + shipped changes nothing for those installs; add a new one instead. A long-lived install applies + every new migration on its next start, against real data. + +### A runtime adapter + +1. Implement `RuntimeAdapter` from `packages/daemon/src/domain/runtime-adapter.ts`: `runtime`, + `listInstalled`, `project`, `deliverStartup`, `launchHarness` and `checkReady`. The smallest + implementation is `packages/daemon/src/adapters/terminal-adapter.ts`; the scripted test double + is `stub-runtime-adapter.ts`. +2. Register it in `packages/daemon/src/startup.ts`. The adapter maps are keyed by runtime name: + `adapters:` (for the pod instantiator) and `runtimeAdapters:`. The context monitor has its own + map for the runtimes it samples. +3. Accept the runtime name in rig spec validation: `SUPPORTED_RUNTIMES` in + `packages/daemon/src/domain/rigspec-preflight.ts`. Other runtime lists, such as `RuntimeHint` in + `domain/discovery-types.ts` and `LEGACY_KNOWN_RUNTIMES` in `domain/rigspec-schema.ts`, may need + it too; search for an existing runtime name such as `"pi"` to find them. +4. Launch, projection and readiness are arteries. Read + [docs/as-built/arteries.md](docs/as-built/arteries.md) first. + +### A shipped skill + +Skills ship from three copies in this repository (the "edges"), listed in +`scripts/skill-edge-layout.generated.json`: + +| Edge | Path | Layout | +|---|---|---| +| `spec` | `packages/daemon/specs/agents/shared/skills/` | by category (`core`, `pm`, `pods`, `process`) | +| `plugin` | `packages/daemon/assets/plugins/openrig-core/skills/` | flat | +| `canonical` | `skills/_canonical/` | public mirror of the `spec` copy | + +Each skill in the layout file lists which edges carry it. At this commit, 33 skills are in `spec` +and `canonical`, 17 only in `plugin`, and 2 in all three. + +- `npm run mirror-skills:check` (part of `npm test`) hashes every file in each edge and compares the + hashes with `scripts/skill-edge-digests.generated.json`. It also fails on a `SKILL.md` the layout + does not list. +- The full apply, `npm run mirror-skills`, regenerates all three edges from the maintainers' skill + source, which lives outside this repository. It needs `OPENRIG_SKILL_CANON_ROOT` and three + authority-file environment variables, so an outside contributor cannot run it. +- **To fix a skill:** edit the file in every edge that carries it (same bytes), run + `node scripts/regen-edge-digests.mjs` to refresh the hashes from disk, then run + `npm run mirror-skills:check`. Say in the pull request that you edited mirrored skill files: the + next full apply regenerates these copies from the maintainers' source, so the change has to land + there too. +- **To add a skill:** open an issue first. Its layout entry is generated from files that are not + in this repository. +- Every shipped skill also becomes a context pack at package time (next recipe). + +### A context pack + +- **Static packs** are hand-written: a directory under `packages/daemon/context-packs-src//` + with a `manifest.yaml` and its files. Keep `version: "0"` in the manifest; the generator stamps + the package version. A pack file may be a symlink to a file under `docs/reference/`, so a + reference document has one source (for example `help/help.md` points to + `docs/reference/help.md`). Any other symlink fails the build. There are 5 static packs at this + commit. +- **Generated packs** go to `packages/daemon/context-packs/`, which is gitignored and never edited + by hand. `scripts/generate-context-packs.mjs` builds it at package time from every shipped skill + and every static pack. +- `npm run generate-context-packs:check` (part of `npm test`) validates every pack through the + daemon's own manifest parser, so build the daemon first. It also runs the leak scan over the + static sources, including the `docs/reference/` files they link to. +- The daemon serves the shipped packs through `rig context get`; `startup.ts` registers the bundled + directory as its built-in root. + +### A scenario + +Scenarios run the real CLI, daemon, tmux and SQLite with scripted stub agents (`runtime: stub`) +and no model cost. + +1. Add `packages/test-system/scenarios/.yaml`, named for the class of bug it + catches. Topology fixtures sit beside it (`*-stub.yaml`). +2. The format is checked by `validateScenario` in + `packages/daemon/test/helpers/scenario-schema.ts`. Step verbs: `up`, `down`, `send`, `restart`, + `restore`, `emit`, `mutate`, `policy`, `seed_regression`, `daemon`, plus `expect`. `expect` may + only read shipped surfaces: `ps`, `queue`, `stream`, `scope`, `pane`, `transcript`, + `tui_socket`, `policy_provenance`. +3. Today the runner binds `up`, `down`, `send`, `restart` and `daemon`. `restore`, `emit`, + `mutate` and `policy` fail with `UnboundActionError`, and `seed_regression` needs a fault + controller (`packages/daemon/test/helpers/scenario-real-deps.ts`). +4. Write a header comment naming the defect class and the seed: what a seeded regression plants, + and which `expect` must catch it. A scenario counts only as a pair: it passes on healthy code + and fails on the seeded run (`packages/test-system/README.md`). +5. **Run it on your machine** after `npm run build`, from a shell that is not inside tmux: + `node --import tsx packages/daemon/scripts/run-scenarios.mjs `. It uses a private + daemon and a private tmux server, and refuses to start if `TMUX` is set. It supplies no fault + controller, so a `seed_regression` step fails there; the seeded pair runs in the container + path. +6. **CI.** The `installed-scenario` job runs `scripts/run-pr-scenarios.sh` in disposable, + network-less containers built from the packed package. It runs two cases, both built around + queue durability (`library` is `queue-baton-survives-restart.yaml`). Adding a case today means + changing three files: the `--case` list in `scripts/run-pr-scenarios.sh`, `CASES` in + `packages/test-system/ci/result.mjs`, and the fault controller in + `packages/test-system/ci/run.mjs`. [docs/as-built/test-layers.md](docs/as-built/test-layers.md) + covers both run modes, running a case on your own Docker host, and what a stub can and cannot + prove. + +## Guards you will meet + +`npm test` runs `test:repo` (builds the daemon, runs `node --test scripts/*.test.mjs`, the docs +guard, `mirror-skills --check` and `generate-context-packs --check`) and then the daemon, CLI and +TUI suites. `npm run test:ui` runs the web UI suite separately. `npm run lint` typechecks all four +packages. + +- **Docs guard** (`scripts/check-docs-guard.mjs`). Under `docs/`, git may track only + `docs/as-built/`, `docs/reference/`, `docs/releases/` and exactly `docs/DESIGN.md`; + `.gitignore` ignores the rest of `docs/`. Remember that `docs/reference/` ships inside the npm + package, and the daemon copies its Markdown files to `$OPENRIG_HOME/reference/` when it + starts. Files in `docs/as-built/` carry a `last-verified-against-source` commit in their + frontmatter. +- **Internal-leak guard** (`scripts/internal-leak-scanner.mjs`, with rules in + `scripts/internal-tokens.generated.json`). It refuses maintainer-internal names and paths in + content that ships. It runs in `generate-context-packs.mjs` (static pack sources, on every + `npm test`), in the full skill mirror apply (each public skill), in `scripts/build-package.sh` + (the daemon `specs/` tree before it is copied into the package), and in the release-time + substance gate (`scripts/check-substance-gate.mjs`, `npm run gate:substance`) over the packed + CLI. No workflow runs it over a pull request's whole diff. +- **Portability report** (`.github/workflows/portability-report.yml`, running + `scripts/portability-report.mjs`). On each pull request it lists added lines containing + machine-specific values: credentials, home and temporary paths, network addresses, email + addresses. It never fails the job; you read it and decide. +- **Generated files.** Do not hand-edit `scripts/*.generated.json` or anything under + `packages/daemon/context-packs/`. The skill-edge digests are the one generated file a contributor + refreshes, with `node scripts/regen-edge-digests.mjs` (see "A shipped skill"). +- **Hosted CI** (`.github/workflows/tests.yml`): `build-and-package`, `typecheck`, `repo-checks`, + `package-tests` (macOS; daemon, CLI, TUI and UI suites with no credentials and no external + network) and `installed-scenario`. + +## Pointers + +- [CONTRIBUTING.md](CONTRIBUTING.md): setup, what a pull request needs, what to expect from review. +- [docs/reference/developing.md](docs/reference/developing.md): which checks block and which are + advisory. It predates the hosted Tests workflow; `.github/workflows/tests.yml` is current. +- [docs/reference/worktree-builds.md](docs/reference/worktree-builds.md): building in a git + worktree, with its own `npm install` rather than a symlinked `node_modules`. +- [docs/reference/](docs/reference/): user and operator reference; it ships with the package. +- [docs/as-built/](docs/as-built/README.md): module-by-module descriptions of the system. **These + are being re-verified.** Most modules were last verified against `7eaf524c` (2026-05-16, around + v0.3.1); the two pages this change adds, `arteries.md` and `test-layers.md`, are verified against + `1347d825`. The older modules' counts are historical: `architecture/daemon-core.md` + describes 40 migrations, and there are 89 today. Use them for orientation, then check the code. +- [docs/as-built/test-layers.md](docs/as-built/test-layers.md): what each test layer covers and + what it cannot catch. +- [docs/as-built/arteries.md](docs/as-built/arteries.md): the areas where a small change has a + large effect, what depends on them, and what broke there before. +- [packages/test-system/ci/README.md](packages/test-system/ci/README.md): the CI scenario job. +- [docs/releases/](docs/releases/) and [CHANGELOG.md](CHANGELOG.md): release history, written by + the maintainers at release time. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 44611f749..36040335e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -52,10 +52,13 @@ npm run lint # typecheck every package ``` `npm test` builds the daemon and runs repository checks before the package suites. Read the -specific failure: `npm run mirror-skills` updates skill mirrors, and -`npm run generate-context-packs` updates generated packs. Use them when their source changed and -review the generated diff; neither command fixes every documentation failure. The UI unit-test -suite is advisory and separate: `npm run test:ui`. +specific failure. A shipped skill may exist in more than one copy (see ARCHITECTURE.md): edit every copy, then run +`node scripts/regen-edge-digests.mjs` (the full `npm run mirror-skills` apply needs maintainer-only +inputs; see "A shipped skill" in [ARCHITECTURE.md](ARCHITECTURE.md#a-shipped-skill)). +`npm run generate-context-packs` updates generated packs. Review the generated diff; neither +fixes every documentation failure. The UI unit-test suite is advisory and separate: +`npm run test:ui`. [docs/as-built/test-layers.md](docs/as-built/test-layers.md) lists every test +layer, what CI runs, and what each layer can and cannot prove. For hands-on development, read the [check requirements](docs/reference/developing.md), [worktree setup](docs/reference/worktree-builds.md), and @@ -103,6 +106,13 @@ clarifying question or run the reproduction; a maintainer makes the merge decisi ## Where things live +- **Start here:** [ARCHITECTURE.md](ARCHITECTURE.md) maps the packages, the request path, and where to + add a command, route, migration, adapter, skill, context pack or scenario. + [docs/as-built/arteries.md](docs/as-built/arteries.md) lists the areas where a small change has a + large effect; if your change touches one, read it first. +- **Using a coding agent?** Claude Code and Codex load the repository's `developing-openrig` skill + when they work in this checkout (`.claude/skills/` and `.agents/skills/`). It points at the maps + above and at what to run before you push. - Repository reference: `docs/reference/`; user documentation: [openrig.dev/docs](https://openrig.dev/docs). - Skills: `packages/daemon/specs/agents/shared/skills/` and plugin skills under `packages/daemon/assets/plugins/`; static context-pack sources: `packages/daemon/context-packs-src/`. diff --git a/docs/as-built/arteries.md b/docs/as-built/arteries.md new file mode 100644 index 000000000..872f0a35e --- /dev/null +++ b/docs/as-built/arteries.md @@ -0,0 +1,103 @@ +--- +kind: as-built +title: Arteries — where a small change has a large effect +status: active +applies-when: | + You are about to change, or review a change to, an area that many other paths depend on: + message delivery, launch and resume, the queue, rig identity, skill projection, native + process observation, migrations, or restore. Read this to see what depends on the area and + what broke there before, then go and read the code. +siblings: [README.md, test-layers.md] +prerequisite-reads: [../../ARCHITECTURE.md] +last-verified-against-source: 1347d825 +last-updated: 2026-10-01 +--- + +# Arteries: where a small change has a large effect + +## Read this first: this map is not the territory + +- **This page is a zoomed-out view, not a description of OpenRig.** It names a few areas where + small changes caused large regressions before. OpenRig is much larger than this page. Code, + behaviour and risk that are not listed here exist, and some of it is riskier than anything below. +- **It is incomplete by construction.** A pull request is not safe because its files are missing + from this list. +- **It may be stale.** It was checked against commit `1347d825`. Files and dependencies move. If + this page and the code disagree, the code is right; fix the page. +- **Use it to change altitude, then go to the ground:** read the code, run the commands, check the + behaviour. When you are lost, go and look rather than leaning on this page. + +## The arteries + +Paths are relative to `packages/`. The key files are entry points, not the full extent of each +area. + +| Artery | Key files | What depends on it | Past regressions and lessons | +|---|---|---|---| +| **Message delivery to seats** | `daemon/src/domain/session-transport.ts`, `daemon/src/domain/seat-delivery-guard.ts`, `daemon/src/adapters/tmux.ts` | Every `rig send`, queue wake, nudge, watchdog wake and handoff | #150 (the fix for #142) stopped typing into a bare shell. It then refused managed Codex seats (fixed by #171 before release) and every Claude seat launched with an explicit permission mode, which shipped in 0.6.2 as #197 (fixed by #220). The delivery guard's rule that a name must resolve to exactly one target, added in #109 (0.6.0), was involved in #141 (fixed by #151), the delivery half of #174 (fixed by #181) and #188 (fixed by #189). | +| **Launch, relaunch, resume** | `daemon/src/adapters/claude-code-adapter.ts`, `daemon/src/domain/claude-managed-launch.ts`, `daemon/src/adapters/codex-runtime-adapter.ts`, `daemon/src/domain/codex-daemon-support.ts`, `daemon/src/domain/native-resume-probe.ts`, `daemon/src/domain/startup-orchestrator.ts` | Every seat start, restore and permission mode | Always setting `CLAUDE_CONFIG_DIR` gave explicit-mode Claude seats a blank config (#154; #225 fixes its config-selection part). Codex seats launch with `--no-daemon` only when `codex --help` shows support (#69, #77); when support is unknown, OpenRig refuses to launch, so a failing or slow `codex --help` blocks the seat. The fixed 30-second readiness window times out under load (#182, open). A git worktree's `.git` file broke the Codex sandbox (#121; see #126). | +| **Queue claim, pickup, wakes** | `daemon/src/domain/queue-repository.ts`, `daemon/src/domain/queue-pickup.ts`, `daemon/src/domain/queue-wake-ladder.ts`, `daemon/src/routes/require-sender-identity.ts`, `cli/src/client.ts` | All durable work between seats | After a slow `/healthz`, the CLI host-qualified a local sender and claims failed (#131, fixed by #135). Done-and-closed items read as stalled forever (#164, fixed by #176). | +| **Rig identity: same names, archived and deleted rigs** | `daemon/src/domain/rig-repository.ts`, `daemon/src/domain/rig-lifecycle-service.ts`, `daemon/src/domain/rig-teardown.ts`, `daemon/src/domain/seat-handover-service.ts`, `daemon/src/domain/running-name-guard.ts` | Seat resolution, `rig remove`, restore, importing a rig from YAML | Archived rigs still matched seat references, and `rig remove` killed another rig's live seat with the same name (#174, fixed by #181). Replacing stopped same-name rig generations on YAML import followed in #196. | +| **Projection of skills and files** | `daemon/src/domain/rigspec-instantiator.ts`, `daemon/src/domain/projection-planner.ts`, and each runtime adapter's `project()` | What every agent is given at start | Codex seats skipped skills that a Claude sibling had already projected (#159, fixed by #162). File-shaped entries failed with ENOTDIR (fixed by #185). | +| **Native process observation** | `daemon/src/domain/native-process-lineage.ts` (parses `ps -Ao pid,ppid,pgid,tpgid,ucomm,lstart,command`) | Liveness, identity verdicts, delivery checks | Processes whose command name contains a space were dropped (#134, fixed by #82). Non-English locales broke the date parsing (fixed by #239, which runs `ps` with `LC_ALL=C`). #220 also changed this file to recognise managed Claude seats behind launch wrappers. | +| **Schema and migrations** | `daemon/src/db/migrations/`, `daemon/src/db/all-migrations.ts`, `daemon/src/db/migrate.ts` | Every install, and every upgrade of a long-lived install | Parallel pull requests pick the same next number: open #112 adds `086_rig_kernel_variant` while `main` already has `086_classification_fields_and_attempts`, and open #155 and #195 take 090 and 091. A long-lived install upgrading across several releases applies many migrations in one start, against real data. | +| **Restore and recovery** | `daemon/src/domain/reconciler.ts`, `daemon/src/domain/restore-check-service.ts`, `daemon/src/domain/startup-orchestrator.ts`, `daemon/src/adapters/tmux.ts` | Reboots, power loss, daemon restarts | Detached seats kept stale tmux pane ids after a reboot (#141, fixed by #151 in `tmux.ts`). `restore-check` ignores the configured shared-docs root (#130, open). After a fresh relaunch, a stale Codex restore attention flag never clears (#116, open). | + +## How reviews use this page + +- **Find where the effect lands.** A reviewer notes which artery a pull request touches, if any, + and what depends on it. +- **At an artery, review the effect, not only the diff.** Whatever the diff size, the reviewer + states the downstream effects and checks behaviour with a scratch reproduction or a test. +- **Away from the arteries,** small pull requests keep a normal review. Scrutiny goes where the + effect is. + +## The scenario rule + +**A pull request that touches an artery adds or extends a stub-agent scenario, or says in its +description why it cannot.** Reviewers check for one or the other. Scenarios live in +`packages/test-system/scenarios/`; [ARCHITECTURE.md](../../ARCHITECTURE.md#a-scenario) says how to +write and run one, and [test-layers.md](test-layers.md) says what each test layer can and cannot +catch. + +The goal is that every artery has at least one scenario whose seeded regression fails, so a small +change to an artery meets a real end-to-end check before it merges, not only a reviewer's reading. + +### Scenario coverage today + +At commit `1347d825`. For why the stub cannot reach some of these yet, see "What a stub can and +cannot prove" in [test-layers.md](test-layers.md). + +- **Queue durability across a daemon restart: covered.** CI runs + `queue-baton-survives-restart.yaml` healthy, then with a seeded `baton-drop` fault that must + fail, then healthy again (#243). It runs `rig queue claim` and checks the destination after the + restart, but pickup, wakes and claim-identity edge cases (such as the #131 sender case) have no + scenario yet. +- **Delivery (the #197 class): not covered.** The stub runtime has no input behaviour, so it cannot + exercise delivery to a managed wrapper; `send-verify-means-rendered.yaml` is not admitted to CI + for that reason (`packages/test-system/ci/README.md`). +- **Restore (the #141 class): not covered.** The scenario runner does not bind the `restore` verb + yet; it fails with `UnboundActionError`. +- **Rig identity (the #174 class): not covered.** No library scenario sets up a same-name archived + or removed rig. +- **Projection (the #159 class): not coverable with `runtime: stub`,** which projects skills into its + own `.openrig/stub/skills/` directory. Use adapter-level unit tests. +- **Native process observation: unit tests only** (for example + `packages/daemon/test/codex-process-observation.test.ts`, `native-ps-locale.test.ts` and + `send-runtime-not-running.test.ts`). +- **Migrations: unit guards only** (`migration-fixture-parity.test.ts`, + `startup-migrations-mirror.test.ts`). Every scenario starts from an empty database, so upgrading + an existing database is not covered. +- **Launch:** every scenario's `up` launches stub seats through the real launch path. Native Claude + Code and Codex launches are not exercised; the CI containers have no provider CLIs or + credentials. + +## Regression log + +When a regression reaches a release, add one line: date, issue, artery, and which layer missed it +(the layers are named in [test-layers.md](test-layers.md)). Each line should end up as a scenario. + +- 2026-09-30, #197 (0.6.2), delivery: review saw a small diff in isolation; CI had no managed + explicit-mode Claude seat; the project's own day-to-day seats use the default launch path. +- 2026-09-29/30, #141, #174, #188 (after 0.6.0), delivery and rig identity: reviews of #109 did not + see the downstream effect of the exactly-one-target rule. diff --git a/docs/as-built/test-layers.md b/docs/as-built/test-layers.md new file mode 100644 index 000000000..7c1b25fa6 --- /dev/null +++ b/docs/as-built/test-layers.md @@ -0,0 +1,460 @@ +--- +kind: as-built +title: Test layers — what to run before you push, and what each layer proves +status: active +applies-when: | + You changed OpenRig and want to know which checks to run before opening a pull + request, what each check actually proves, and what CI will run for you. Also + read it before claiming a stub-agent scenario covers a behaviour. +siblings: [arteries.md, README.md, codemap.md] +last-verified-against-source: 1347d825 +last-updated: 2026-10-01 +--- + +# Test layers: what to run before you push, and what each layer proves + +> **This page is a map, not the territory.** It was checked against source at `1347d825` +> and it will drift. It is also incomplete on purpose. Before you rely on a command or a +> claim here, read the script it names or run it. If a behaviour is missing from this page +> (or from [arteries.md](arteries.md)), that tells you nothing about whether a change to it +> is safe. + +## Short version + +For every change, run what the pull-request template asks for: + +```sh +npm run build +npm run lint # typecheck only; there is no formatter or style linter +npm test # = npm run test:repo && npm run test:workspaces +``` + +Then add the layers your change calls for, using the ladder below. CI also runs a +network-denied macOS test pass and a containerised installed-package scenario on every PR +to `main`. In the PR description, say which layers you ran and which you could not run. + +## The ladder + +| Layer | Command | What it proves | What it cannot prove | When to run it | +|---|---|---|---|---| +| Typecheck | `npm run lint` | Builds the daemon, then `tsc --noEmit` passes for daemon, ui, cli and tui | Any runtime behaviour. Formatting. | Every change | +| Build | `npm run build` | All four workspaces compile (the UI also runs `vite build`) | That the *published* package is complete (see Packaging) | Every change | +| Repo checks | `npm run test:repo` | Builds the daemon, then runs `node --test --test-concurrency=1 scripts/*.test.mjs`, `scripts/check-docs-guard.mjs`, `mirror-skills --check` and `generate-context-packs --check` | Package behaviour | Every change. It matters most when you touch `scripts/`, `docs/` or skills. | +| Package suites | `npm run test:workspaces`. One package: `npm run test -w packages/cli`. One file, from the repo root: `npx vitest run packages/cli/test/.test.ts` | Vitest unit and integration tests for daemon, cli and tui. Some of them start real scenario daemons, tmux sessions and stub runners. | The installed package. Real Claude Code or Codex. CI's sandbox conditions (see CI). | Every change (`npm test` covers it) | +| Web UI suite | `npm run test:ui` | jsdom unit tests for `packages/ui` | Real browser rendering | When you touch `packages/ui`, or a daemon API the UI reads. The web UI is in maintenance mode, so this suite is advisory locally and is not part of `npm test`. | +| Generated files | `npm run mirror-skills:check`, `npm run generate-context-packs:check` (both part of `npm test`) | Every copy of each shipped skill matches its recorded hash, and every static context pack passes the daemon's own manifest parser and the leak scan | Whether the skill content is any good | After you edit a shipped skill: edit the same file in every copy that carries it, run `node scripts/regen-edge-digests.mjs`, then the check. The full `npm run mirror-skills` apply needs maintainer-only inputs. See "A shipped skill" in [ARCHITECTURE.md](../../ARCHITECTURE.md#a-shipped-skill). | +| Local gate | `npm run gate` | Lint plus `npm test` on a clean tree, tied to `HEAD`, one run per machine at a time, with a JSON receipt | Anything that lint and `npm test` don't already prove | Optional. Useful when several agents share one machine, or when you want a receipt tied to a commit. | +| Packaging | `npm run build:package`, `npm run test:tui-package`, `bash scripts/smoke-fresh-install.sh` | The publishable `@openrig/cli` assembles, the packed TUI launches, and a fresh install starts a daemon | Behaviour beyond startup | When you touch dependencies, bin entries, build scripts, or files a package ships | +| Stub scenarios, host mode | `node --import tsx packages/daemon/scripts/run-scenarios.mjs [scenario.yaml …]` | A scenario's steps and assertions pass against a scratch daemon, real tmux and `runtime: stub` seats | The installed package. Seeded-regression pairing. Everything under [What a stub can and cannot prove](#what-a-stub-can-and-cannot-prove). | While you write or debug a scenario | +| Stub scenarios, installed package in a container | CI job `installed-scenario`. Before pushing: `DOCKER_HOST=ssh:// bash scripts/run-pr-scenarios.sh --remote …` | A clean Linux install starts. A claimed queue item survives a daemon restart. A planted fault is caught at the intended assertion. | Native runtimes. Most command families (see [Help wanted](#help-wanted-command-families-without-a-behavioural-scenario)). | Changes that can affect daemon startup, packaging, queue durability or the scenario harness | +| Live-model evals | `npm run eval -w packages/daemon -- --provider rig …` | Whether a real Claude Code seat runs the expected `rig context get ` for natural prompts | Determinism. Codex. Anything without a case. | Changes to shipped skills or context entries meant to steer agents. **Uses provider credits.** | +| Testbed runbooks | `docker/testbed/runbooks/` (manual) | The testbed image itself works as a test environment | Product behaviour | Only when you change the testbed image or the container plumbing | + +### Gotchas that cost people time + +- **Stale vendored daemon.** `npm run build:package` leaves an assembled copy of the daemon at + `packages/cli/daemon/` (gitignored). If you change daemon source afterwards, + `scripts/check-cli-daemon-freshness.test.mjs` inside `test:repo` can fail on the stale copy. + Re-run `npm run build:package` or delete `packages/cli/daemon/`. `npm run gate` deletes it for you. +- **Local passes, CI fails.** CI runs the package suites with a scratch `HOME`, no + credentials, and outbound network blocked except to localhost (see below). A test that reads + your real home directory, a running daemon, the network or a credential can pass on your + machine and fail in CI. +- **Docs location.** `check-docs-guard` fails if a tracked file under `docs/` sits outside + `docs/as-built/`, `docs/reference/` or `docs/releases/`. `docs/DESIGN.md` is the one allowed + exception. + +## What CI runs on your pull request + +`.github/workflows/tests.yml` runs on pull requests to `main` and on manual dispatch, all on +Node 22. Each suite is its own job, so one failure doesn't hide the others. No provider +credentials are supplied. + +| Job | Runner | What it does | +|---|---|---| +| `build-and-package` | ubuntu | `npm ci` and `npm run build`. Then `npm pack --dry-run` for cli, daemon and tui, with each file list uploaded as the `pack-lists` artifact. The job only records the lists; reviewers read them. | +| `typecheck` | ubuntu | `npm run lint` | +| `repo-checks` | ubuntu | `npm run test:repo` | +| `package-tests` | macOS, matrix `daemon`, `cli`, `tui`, `ui` (fail-fast off) | Installs tmux, runs `npm ci` and `npm run build`. Then runs `npm run test -w packages/` under `env -i` with a scratch `HOME`/`TMPDIR`/`TMUX_TMPDIR` and a `sandbox-exec` profile that blocks outbound network except localhost and the job's own socket directory. `scripts/ci-macos-process-probe.mjs` runs first. | +| `installed-scenario` | ubuntu | `bash scripts/run-pr-scenarios.sh` (six container runs, described below). Uploads `dist/pr-scenarios/` as `installed-scenario-evidence`, including on failure. | + +A second workflow, `.github/workflows/portability-report.yml` (job `portability-report`), +lists lines your PR adds that contain machine-specific values: credentials, home or temp +paths, network addresses, email addresses. It never fails because of a finding. To see the +matched text locally (credentials show only their first characters), run +`node scripts/portability-report.mjs`. By default it compares the +merge-base with `origin/main` against `HEAD`; pass `--staged` to check staged changes instead. + +The UI suite is a CI matrix leg even though it is advisory locally. Check your PR's status to +see whether a red UI leg is treated as blocking. Branch-protection settings are not in the +repository. + +## The local gate: `npm run gate` + +`scripts/gate-lane.mjs` runs two legs: `npm run lint` and `npm run test`. It does not run +`test:ui`, the scenarios or the packaging checks. On top of those legs it: + +- holds a machine-wide lock on localhost port 40404 (override with `OPENRIG_GATE_LANE_PORT`). + A second gate exits 2 straight away instead of waiting. +- refuses a dirty worktree. Refuses a `node_modules` that is a symlink, or `@openrig/*` + workspace links that resolve outside this worktree. Both are common with shared git worktrees. +- deletes the stale vendored daemon bundle at `packages/cli/daemon/` before the legs run. +- writes `gate-lane-verdict.json` at the repo root (or wherever `OPENRIG_GATE_VERDICT` + points). The file holds the `HEAD` SHA, per-leg results and durations, and a note of other + `node`/`vitest`/`tsc` processes running at the time. It fails if `HEAD` changes during the + run. The file is not gitignored, so don't commit it. + +Exit codes: 0 pass, 1 a leg failed, 2 lane busy, 3 the runner itself errored. +`OPENRIG_GATE_LANE_SMOKE=1` skips the legs to test the wiring only, and the verdict records +`smoke: true`. + +## Stub-agent scenarios + +### What's in the box + +| Piece | Where | +|---|---| +| `runtime: stub` adapter | `packages/daemon/src/adapters/stub-runtime-adapter.ts`, with `stub-runner.ts` (the process that runs in the pane), `stub-script.ts` (the launch-script format) and `stub-compaction.ts` / `stub-restore.ts` | +| Scenario format and runner | `packages/daemon/test/helpers/scenario-*.ts`. Step verbs: `up`, `down`, `send`, `restart`, `daemon`, `seed_regression` are bound. `restore`, `emit`, `mutate`, `policy` parse but are unbound. Assertion surfaces: `ps`, `queue`, `stream`, `scope`, `pane`, `transcript`, `tui_socket`, `policy_provenance`. | +| Scenario library | `packages/test-system/scenarios/`: 11 scenarios, 6 stub topologies, `agents/`, `culture.md` | +| Runner fixtures | `packages/daemon/test/fixtures/scenarios/` (`scenario-01-per-seat-scripts`, `scenario-02-baton`, `scenario-10-one-view-state`) | +| Host runner | `packages/daemon/scripts/run-scenarios.mjs` | +| In-container entry and result check | `packages/test-system/ci/run.mjs`, `packages/test-system/ci/result.mjs` | +| PR orchestrator | `scripts/run-pr-scenarios.sh` | +| Image | `docker/testbed/Dockerfile`, built by `scripts/build-testbed-image.sh`; `docker/testbed/Dockerfile.scenarios` is layered on top by `scripts/run-pr-scenarios.sh` | + +### Host mode: while you're writing a scenario + +```sh +npm run build # the runner drives packages/cli/dist/bin-wrapper.js +node --import tsx packages/daemon/scripts/run-scenarios.mjs # the fixture scenario-*.yaml files +node --import tsx packages/daemon/scripts/run-scenarios.mjs packages/daemon/test/fixtures/scenarios/scenario-02-baton.yaml +``` + +- The runner creates a scratch `HOME`/`OPENRIG_HOME`, its own tmux server and its own + daemon. It builds the scenario environment from `HOME`, `PATH` and `TERM` only, so your + shell's daemon-target variables (`OPENRIG_URL`, `OPENRIG_PORT` and similar), `TMUX` and + `OPENRIG_TEST_CLOCK_NOW` are dropped rather than inherited. It does not touch the daemon or + tmux server your own session uses. +- It accepts YAML paths only. Any flag, including `--container`, is refused before anything runs. +- It supplies no fault controller, so any scenario with a `seed_regression` step fails loudly + at that step. That includes the library's `queue-baton-survives-restart`. Use the container + path to run the seeded pair. +- `scenario-10-one-view-state` declares its own normaliser for its cross-surface `equals` + check. Whether it passes in host mode has not been verified. + +### Container mode: the scenario pack CI runs + +In CI the `installed-scenario` job runs `scripts/run-pr-scenarios.sh` with no arguments: +both cases (`fixture` = `scenario-02-baton.yaml`, `library` = `queue-baton-survives-restart.yaml`), +each run three times as healthy, then lost-baton, then healthy. That is six fresh containers. + +To run a case before you push, point `DOCKER_HOST` at a Docker engine you control over SSH: + +```sh +npm ci +DOCKER_HOST=ssh:// bash scripts/run-pr-scenarios.sh \ + --remote --case library --mode healthy --out dist/scenario-check +``` + +- Options: `--case fixture|library` (default: both), `--mode healthy|lost-baton` (default: + healthy, lost-baton, healthy) and `--out ` (default `dist/pr-scenarios`). Use a fresh + `--out` directory for each run you want to keep. +- `--remote` accepts only `ssh://` values for `DOCKER_HOST`, and it refuses if + `DOCKER_CONTEXT` is set. The engine must be Linux on amd64 or arm64. The script reads the + *server's* platform, so a macOS/arm64 laptop driving a Linux/amd64 engine works. +- Without `--remote`, the script refuses to run anywhere except GitHub Actions on Linux. +- The source build and pack run on your machine. The image build and the containers run on + the engine. Building the image needs network access for the pinned base image, Node and + dependencies. The scenario containers have no network. +- The image is built from your working tree but tagged with `git rev-parse HEAD`. Commit + first, so the tag names what you actually tested. +- It never pushes images. It removes only the containers it named. If the SSH connection + drops, it prints the name of any container it could not remove. + +What happens, in order: + +1. `scripts/build-testbed-image.sh` packs `@openrig/cli` from your tree. It installs the + package into a digest-pinned Debian image with Node, checks `rig --version`, and checks + that the installed daemon loads. The result is tagged `openrig-testbed:`. +2. `docker/testbed/Dockerfile.scenarios` adds a bundled copy of + `packages/test-system/ci/run.mjs` plus the fixture and library scenarios. +3. Each run uses a fresh container: `--network none`, read-only root plus a 512 MB `/tmp` + tmpfs, `--cap-drop ALL`, `no-new-privileges`, non-root, 2 CPUs, 2 GiB, 256 PIDs and a + 300-second deadline. +4. In lost-baton mode, the runner stops the daemon, changes exactly one queue row in the + scratch database from `in-progress` to `pending`, and restarts the daemon. + `packages/test-system/ci/result.mjs` then accepts only one outcome: a failure at the + post-restart queue assertion, with `baton-1` observed as `pending` at the expected + destination. A startup error, a timeout or any other failure does not count as a caught + regression. Healthy runs must pass with no fault armed. + +A single healthy run proves that one leg only. The paired fault run is what shows the +assertion can catch the failure. + +Adding a new case to this pack today means changing three files: +`scripts/run-pr-scenarios.sh` (the `--case` list), `CASES` in +`packages/test-system/ci/result.mjs`, and the fault controller in +`packages/test-system/ci/run.mjs`, which is currently specific to the baton row. + +### What a green stub scenario proves + +- The packaged CLI installs into a clean Linux container and starts a daemon. +- `rig up` and `rig down` of a stub topology go through the real daemon, tmux, SQLite and CLI. +- Queue state, read through the shipped `rig queue list` path, survives a daemon restart. +- With the paired seeded run: the assertion fails when that state is broken. + +### What a stub can and cannot prove + +The stub does not fabricate product outputs. It runs the real daemon, tmux and CLI. But it is +not Claude Code or Codex, and it differs from them in ways that decide what a stub green +means: + +| Area | What the stub does at `1347d825` | What that means for your test | +|---|---|---| +| Consuming a message | `stub-runner.ts` runs its launch script once and then idles. It has no stdin reader, socket or other input channel. The default script prints `[stub] scripted reply: acknowledged` at boot, before anything has been sent. | A pane showing a reply, or your echoed text, does not prove the message was consumed. No stub scenario can currently prove "delivered and answered". Separately, `rig send --verify` means "appeared in the pane", not acknowledgement (see `rig send --help`), and the scenario `send` step doesn't pass `--verify` at all. | +| Launch path | `StubRuntimeAdapter` types `node …` into the pane (`tmux.sendText`, then Enter). Claude Code with an explicit permission mode launches through a managed launch (`ClaudeManagedLaunch.prepare`, `tmux.sendShellCommand`); without one it uses `sendText`. The shell-foreground check in `session-transport.ts` (`unverifiedShellForeground`) runs for every runtime except `terminal`, but its native-process proof is limited to `codex` and `claude-code`. | A stub seat doesn't exercise wrapper, managed-launch or native-process-identity behaviour (the class behind #197). Wrapping the stub in a shell wouldn't change that. | +| Permissions | `validateNativePermissionSelection` accepts only `codex` and `claude-code`. The stub models only the `floor` / `full_bypass` launch posture. | Testing Claude permission modes or per-seat permission selection needs a real runtime. | +| Queue pickup and wake | No stub worker reacts to a nudge by claiming an item, working on it and handing it back. | The baton scenarios prove the claim *survives a restart*. They don't prove delivery, pickup or wake. | +| Reboot / tmux reset | `daemon: {op: restart}` restarts only the scenario daemon, and the tmux server keeps running. No step resets tmux. | Recycled pane IDs and stale bindings after a real reboot (the class behind #141) are not covered by a daemon restart. | +| Multi-rig and same-name identities | The `up` step always uses the scenario's top-level `topology` and ignores a per-step override. No step archives or removes a rig. | Cross-rig scope, same-name generations and archive/remove routing (the class behind #174) need runner work before they can be tested. | +| Step-time restore, compaction, mutation, policy | `restore`, `emit`, `mutate` and `policy` parse, but the runner throws `UnboundActionError` when it reaches them. Behaviours run only from a per-seat launch script (`env.stub_scripts`). A seat `restart` step maps to `rig launch`, which is not native resume or fork. | Step-time compaction and restore, and native resume and fork, are not covered. | +| Skill projection | The stub projects skills into `/.openrig/stub/skills/`. Claude Code projects into `/.claude/skills/`. | A stub green says nothing about provider-specific projection (the class behind #159). | +| Model and provider | No model, no tokens. Stub scripts refuse `usage_limit` because it only exists on real runtimes. | Model behaviour, provider usage limits and real context usage need live evals or a real runtime. | +| Platform | CI's scenario job runs in a Linux container. | macOS process behaviour is covered by the macOS `package-tests` jobs, not by scenarios. | + +The most useful next pieces of harness work, roughly in dependency order: + +1. A stub that consumes input and answers a fresh nonce. +2. A model-free test entry point shaped like a provider, that goes through the real native + adapter and managed launch. +3. Lifecycle and identity steps (tmux reset, archive/remove). +4. Then pickup, wake and restore scenarios. + +### Library status + +Only `queue-baton-survives-restart` runs in CI (since #243). The other ten scenarios are +authored but not yet runnable as meaningful checks: + +| Scenario | Why it doesn't run as a check yet | Smallest useful next step | +|---|---|---| +| `queue-baton-survives-restart` | Runs in CI: healthy / seeded `baton-drop` / healthy | Extend it to handoff, block, resolve and unclaim | +| `clean-lifecycle-no-residue` | The post-`down` "nothing left over" assertion is commented out, because the format has no way to say "empty" | Add a real absence assertion | +| `ps-scope-honesty` | The second `up` reuses the first topology, and the expected match tolerates the other rig | Make `up` honour the per-step topology, then assert the exclusion | +| `send-verify-means-rendered` | The `send` step drops `verify`, and the stub has no input loop | Build the input-consuming stub and assert a response derived from the nonce | +| `home-divergence-preseed-visible` | The pipeline ignores `env.pre_existing_tmux`, and the default script never prints the expected `trust reached` | Recreate the real HOME split and whatever consumes it | +| `compaction-restore-resumes-role` | Step-time `emit` is unbound | Input-triggered hooks plus evidence specific to one seat | +| `stream-emit-durable-replay-live` | Step-time `emit` is unbound | Assert the actual stream contract | +| `policy-posture-survives-structural-edit` | `policy` and `mutate` are unbound | Rewrite around current per-seat permission selection and an observed launch | +| `locked-restore-never-permissive` | `policy` and `restore` are unbound | Keep the invariant (the selection survives a restore) and use current modes | +| `one-view-state-after-mutation-storm` | The validator rejects its bare-list `equals` (`EQUALS_NOT_DECLARATIVE`), and `mutate` is unbound | Declare the comparable fields and real mutations | +| `kill-daemon-mid-handoff` | A pane `send` doesn't create the queue item the next assertion expects, and later assertions use a shape the queue reader doesn't return | Rebuild the setup before treating a red result as meaningful | + +In most of these files, `seed_regression` steps that come *after* the assertions are +historical markers, not executable fault injections. + +## If your change touches an artery, add or extend a scenario (or say why not) + +[arteries.md](arteries.md) lists the behaviours that everything else depends on. If your +diff can affect one, your PR description should do one of these: + +- link the scenario that covers it, and say which layer you ran it in; +- add or extend a scenario. For a regression scenario, show that the fixed code passes and + that the planted old defect fails *at the intended assertion*. A setup crash or a timeout + doesn't count; +- or say precisely why the stub can't exercise it. Use the table above, which covers + message consumption, native launch and permissions, tmux reset and so on. Then say what + you ran instead, for example a manual run with the real runtime. + +This is part of normal review, not an extra approval step. It also isn't a claim that +changes outside the arteries map are safe. + +## Live-model evals + +`packages/test-system/evals/` is a separate check from scenarios. A real seat receives a +natural prompt and decides what to do. A case passes when the seat's transcript shows the +expected `rig context get ` command, in the expected order. Cases live in +`cases/selection.yaml` (10) and `cases/loading.yaml` (4). + +```sh +# Harness check: a fake provider fed a JSON map of prompt -> transcript. +# Any prompt without a transcript counts as an error, never as a pass. +npm run eval -w packages/daemon -- --transcripts --out + +# Live run: drives a real Claude Code seat +npm run eval -w packages/daemon -- --provider rig --seat --out +npm run eval -w packages/daemon -- --provider rig --seat-spec "$PWD/" --out +``` + +- **Live runs use provider credits, and the results vary from run to run.** CI never runs them. +- `--provider rig` needs `OPENRIG_HOME` set, and it supports Claude Code seats only. It + refuses a Codex seat. +- `--seat-spec` runs `rig up` of a scratch rig against **your current daemon**, then + `rig down` at the end. `--seat` attaches to an existing seat and leaves it running. +- `evals/rig-seat/example-scratch-rig.yaml` sets `permission_policy: builtin:yolo` and + `cwd: "."`. It also references `local:agents/blank`, and no such agent directory ships + beside it. Read the file and expect to adapt it before you use it. +- Before any run, the runner builds the production context package in a temp directory and + refuses if any case's entry doesn't resolve in it. +- `npm test` covers the harness without a model: the case schema, the grader, the fixtures + and production entry resolution (`packages/daemon/test/eval-*.test.ts`). + +## Docker testbed runbooks + +`docker/testbed/runbooks/` holds manual procedures, L0 to L6, that you run on a machine with +Docker. They check the testbed image itself: + +- L0: resolve the pinned base image and the stub asset list +- L1: PTY allocation +- L2: tmux server lifecycle +- L3: the daemon inside a container, settling a stub rig +- L4: the scenario environment still refuses a foreign daemon target inside a container +- L5: several containers acting as named hosts on one Docker network +- L6: the scenario runner driving a scenario through a container + +Use them when you change `docker/testbed/`, `scripts/build-testbed-image.sh` or the +container staging helpers. Ordinary PRs don't need them. They were written before the CI +scenario job and contain internal planning references, so treat them as procedures to adapt, +not as current status. + +## Help wanted: command families without a behavioural scenario + +> This list may be stale. It was taken from the `rig --help` tree of 0.6.3: 85 visible +> top-level families. At `1347d825`, `packages/cli/src/index.ts` registers 85 top-level +> commands. Before picking one up, check `packages/test-system/scenarios/`, the cases in +> `scripts/run-pr-scenarios.sh`, and the current `rig --help`. +> One scenario doesn't cover a family's every option, sequence or platform. A `--help` +> check only proves the command is discoverable. Existing unit tests are not counted here. +> They exist, but this list is about behavioural scenarios. + +Status today: `queue` has partial coverage, namely create, claim and list, plus the claim +surviving a restart. `up`, `down` and `daemon` are used by the existing scenarios but have no +scenario of their own. Everything else has none. + +Before you start, two honest constraints: + +- Today's step verbs can't run arbitrary `rig` commands. Many rows below need a runner + binding first. One proposal (not implemented) is a single structured-argv step that runs + the installed CLI through the existing `rigBin`, instead of one new verb per command. +- Rows that need a seat to *answer* (anything with "nonce") depend on the input-consuming + stub described above. + +| Group | Family | Proposed first check (observable result) | +|---|---|---| +| lifecycle | `start` | Restore two stopped stub seats. Both answer distinct fresh nonces, and no second daemon starts. | +| lifecycle | `daemon` | start/status/stop/restart. Port, process and persisted queue identity agree through public commands. | +| lifecycle | `bootstrap` | Bootstrap a minimal stub spec. Seats come up ready and the selected spec is the one used. | +| lifecycle | `up` | Launch two stub seats. Each consumes its own send. Relaunching doesn't duplicate identities. | +| lifecycle | `down` | Tear down one rig. Its panes are gone, a sibling rig still answers, retained data follows the docs. | +| lifecycle | `create` | Create a one-seat rig through the public path. It appears and answers without a hand-written topology. | +| lifecycle | `launch` | Fresh, relaunch, resume and fork on provider-shaped fixtures. Correct restored token, one live process. | +| reads | `status` | Two rigs in different states. Status matches their exact identities and transitions. | +| reads | `ps` | Two rigs with overlapping seat names. Rig and node filters return exactly the intended members. | +| reads | `usage` | Deterministic stub telemetry. top and series respect seat and time bounds. Missing data isn't reported as zero. | +| reads | `health` | Known fresh, stale and cleared findings. Scope, explain and diagnosis values match the fixture. | +| continuity | `snapshot` | Snapshot two seats. Rig, membership and snapshot bytes survive a daemon restart. | +| continuity | `restore` | Restore after a daemon and tmux reset. Seat identity and nonce answers survive. | +| continuity | `agent-image` | create/show/pin/unpin/delete for a provider-shaped fixture. References and identity match. | +| continuity | `fork` | Fork at a known history token. The child keeps the prefix with its own identity, and the parent still answers. | +| continuity | `restore-check` | One restorable and one missing transcript. Readiness tells them apart without relaunching. | +| continuity | `restore-packet` | write/read/validate round trip. A malformed packet fails clearly. | +| continuity | `compact-plan` | Scripted history and usage fixtures. Candidates and exclusions match thresholds, and nothing is compacted. | +| continuity | `compact` | Input-driven compact and restore keep the role and token. Messages before and after reach the same seat. | +| continuity | `handover` | Scripted predecessor and successor. Context is delivered before the rebind, and the successor answers. | +| recovery | `crash-cart` | Stop only the scratch daemon. The output names paths and state, and the command changes nothing. | +| recovery | `preflight` | Ready and missing-dependency fixtures produce named checks without launching seats. | +| recovery | `doctor` | Healthy and damaged scratch installs. Exact diagnosis, no automatic repair. | +| recovery | `destroy` | In a disposable scenario HOME only. Removes the intended state and keeps explicitly retained markers. | +| registry | `gateway` | Add, show, update and remove a synthetic human. Reads match exact fields, and removal sticks. | +| registry | `host` | add/list/select/rename isolated entries that resolve to fixture endpoints. Pairing needs two container daemons. | +| coordination | `parked` | An owed item with a stopped worker is reported. A held worker with a live wake is distinguished. | +| coordination | `stream` | emit/list/show/archive an exact item. A restart and a bounded watch keep the order with no duplicates. | +| coordination | `queue` | (partial) Next: handoff, block, resolve and unclaim, each with exact items and destinations. | +| coordination | `project` | Acquire a lease, classify one item twice, read one projection. The wrong holder cannot act. | +| coordination | `view` | Register a view over known items. list/show return the exact filtered result after a mutation. | +| coordination | `watchdog` | Register one short wake, see one response, stop it, prove no further deliveries. | +| coordination | `workflow` | validate/compile/instantiate a tiny flow. It transitions and resumes after a restart with no duplicate queue work. | +| coordination | `chatroom` | send/history/topic/clear in a scratch room. A bounded watcher sees exactly the ordered messages. | +| coordination | `heartbeat` | Known execution-proof and missing-closure fixtures report the right status. Empty output isn't completion. | +| authoring | `export` | Export a stub rig, then validate and re-import it. Membership and configuration are equivalent. | +| authoring | `import` | Import a minimal spec, and the export matches it. Malformed input leaves state unchanged. | +| authoring | `package` | validate/plan/install/rollback in scratch. Installed files and prior state match. | +| authoring | `requirements` | Complete and missing capability fixtures produce the right named requirements. | +| authoring | `bundle` | create/inspect/install a local bundle and round-trip it. A missing asset gives a named failure. | +| authoring | `agent` | Validate a good and a malformed stub AgentSpec. The diagnostic names the bad field. | +| authoring | `spec` | validate/audit/preflight/show a minimal spec, with predictable diagnostics for a malformed field. | +| authoring | `specs` | add/show/preview/rename/remove a library fixture. A missing reference fails explicitly. | +| authoring | `workspace` | validate/doctor known roots plus one missing-frontmatter case. Reports the exact affected path. | +| authoring | `config` | set/get/reset a scratch key and init a workspace. Readback and files match the requested root. | +| presentation | `ui` | Open the URL through a fake browser executable and read a local UI route. Rendering is out of scope. | +| presentation | `tui` | Drive the TUI command bar and socket over two stub rigs. select, mutate and back show exact identities. | +| presentation | `terminal` | open/status/views against a fake terminal-app endpoint. Correct target and layout arguments. | +| adoption | `discover` | One unmanaged and one owned scratch tmux session. Only the intended candidate is discovered. | +| adoption | `attach` | Attach to the intended seat. `whoami` reports it without duplicating the process. | +| adoption | `bind` | Bind one discovered pane. A different, already-occupied identity is refused. | +| adoption | `adopt` | Adopt two scratch sessions. PIDs and history are preserved, and the identities become addressable. | +| adoption | `reconcile-session` | Rebind a hand-resumed provider-shaped fixture without starting or typing into another process. | +| adoption | `unclaim` | Release an adopted seat. The tmux process and its history stay alive. | +| adoption | `release` | Release one rig's claimed sessions. Processes survive, and a sibling rig is unchanged. | +| identity | `archive` | Archive an old same-name generation. The current one stays routable, and the old data is kept. | +| identity | `unarchive` | Unarchive a retained generation without taking over another live seat. | +| identity | `remove` | Remove an archived generation or one member. Only that identity goes, and a reused name still routes. | +| identity | `whoami` | Inside each stub seat, it returns that seat, not a sibling or the caller. | +| identity | `seat` | Status, held-message guard, stop/launch and resume token keep the right process and permissions. | +| integration | `mcp` | Serve against a scratch daemon. One listed tool's value matches CLI state. | +| integration | `slack` | Manifest and config round trip, plus deterministic inbound and outbound fixture requests. No real Slack account. | +| integration | `ask` | Search fixture transcripts via a fake model endpoint. Citations resolve to the stored lines. | +| integration | `file` | Copy a fixture payload through isolated transport endpoints. Verify bytes and paths. | +| integration | `env` | A fixture service-backed rig. status/logs/down reflect that service and stop only it. | +| delivery | `transcript` | Ordered, unique stub output. The selected seat's transcript contains no other seat's token. | +| delivery | `send` | Send a fresh nonce and require an answer derived from it. Boot text or echo can't pass. | +| delivery | `capture` | A unique token in the pane. Wrong-seat, tail and not-found cases are told apart. | +| delivery | `broadcast` | Broadcast a nonce to a rig or pod. Each intended seat answers once, and an excluded seat gets nothing. | +| delivery | `walk` | Walk two pieces through an input-consuming stub. Order is kept, and a bad piece isn't reported as delivered. | +| context | `context` | add/get/compose/trace/profile over a fixture project. Output bytes, order and roots match. | +| context | `plugin` | list/show/used-by/validate a fixture plugin. A missing resource is not the same as a valid empty selection. | +| context | `skill` | Resolve and audit a fixture skill, then inspect the projection the consumer actually gets. | +| context | `mode` | set/effective/cite/unset at nested scopes. Precedence holds, and the inherited value returns after unset. | +| context | `startup-proof` | Submit a proof from a real stub identity. Check the stored attribution and the duplicate behaviour. | +| permissions | `policy` | Apply and read an explicit permission selection using a provider-shaped fixture. It survives a restore, and the default is unchanged. | +| accounts | `auth` | Synthetic auth files in scratch: save, list, validate, switch. No provider contact, no real credentials. | +| accounts | `provider` | Synthetic account, binding and signal state. The precheck refuses where it should, and a switch keeps seat history. | +| topology | `expand` | Add a pod to a live stub rig. Old seats survive, and new members answer nonces. | +| topology | `add` | Add one seat. Exact binding, siblings preserved, and the duplicate-name behaviour is explicit. | +| topology | `grow` | Grow two seats. Identities are unique, config is inherited, and existing history is kept. | +| topology | `shrink` | Remove a pod. Unrelated seats still answer, and the removed membership is gone. | +| onboarding | `setup` | Scripted newcomer answers against synthetic providers. No install or login for an unused provider. | +| work | `scope` | Create a mission and a slice. move/progress/notes/graph readback keep identity and paths. | +| work | `proof` | add/judge/show evidence in a fixture scope. Readiness updates without overwriting earlier proof. | + +How to make a case useful: + +- Assert actual state or content, not just exit code 0. Where ordinary use includes bad + input, add an invalid or missing input case too. +- Reuse the existing runner and the installed CLI. Don't add a second scenario engine. +- If a scenario exposes an existing product bug, keep the failing case and open an issue. + Include the exact source revision, the shortest failing sequence, and the intended versus + observed result. Don't weaken the assertion to make it pass. Keep "the harness setup + failed" separate from "the product misbehaved". +- A check that only covers the success path is still useful. Just don't call it a + regression detector until you have seen its negative case fail. + +## Existing docs in this area that are out of date + +Read these with care. This page notes them and proposes no changes. + +- `packages/test-system/README.md`: says the scenarios "DO NOT run" until a runner lands. + The runner exists, and one library scenario runs in CI. Its layout lists `fixtures/*.yaml`, + a directory that no longer exists (the topologies live in `scenarios/`). It also says the + emit behaviours don't exist yet. Launch-script `emit` now exists, but step-time `emit` is + still unbound. `packages/test-system/ci/README.md` is the current description of the CI + scenario job. +- `packages/test-system/scripts/README.md` and `packages/test-system/ROUTING.md` are + dependency-routing notes from when the scenarios were first written. Several of the items + they list have since landed. Treat them as history. +- `packages/test-system/evals/README.md`: says cases are `cases/*.ts`. They are + `cases/selection.yaml` and `cases/loading.yaml`. Its "build in progress" status line is old. +- `docs/reference/developing.md`: says "There is no external CI at this tip". In fact + `.github/workflows/tests.yml` runs on every PR to `main`, and its `package-tests` matrix + includes `ui`. Its description of `test:repo` leaves out the `generate-context-packs --check` step. +- `docker/testbed/runbooks/`: see above. They are manual procedures that predate the CI job. diff --git a/packages/daemon/assets/plugins/openrig-core/skills/openrig-skills/SKILL.md b/packages/daemon/assets/plugins/openrig-core/skills/openrig-skills/SKILL.md index 51a9f4d9b..15010c65e 100644 --- a/packages/daemon/assets/plugins/openrig-core/skills/openrig-skills/SKILL.md +++ b/packages/daemon/assets/plugins/openrig-core/skills/openrig-skills/SKILL.md @@ -120,6 +120,13 @@ and a useful report to the OpenRig team if you are still stuck. If `rig` won't r `daemon/docs/reference/help.md` inside the installed `@openrig/cli` package, or https://www.openrig.dev/help/agents (the same text). +## Developing OpenRig itself + +If you're changing OpenRig's own source in a clone of the openrig repository, use that repository's +`developing-openrig` skill (in the checkout's `.claude/skills/` or `.agents/skills/`). It maps the architecture, the +areas where a small change has a large effect, and which tests to run before a pull request. + ## Need more than what ships here? -This index covers the **shipped** surface. A dev host carries far more (factory, architecture, PM-craft, studio skills) reached through the host's own routers/codemaps — if you're on a builder host and need something not listed above, that deeper routing is the next hop, not a wall. (Host-scale routing is the subject of the context-routing architecture doc; at product scale, this one file is the whole map.) +This index covers the **shipped** surface. Your project, rig or machine may add its own skills and routers. Check the +skills your working directory loads before concluding that something doesn't exist. diff --git a/packages/daemon/specs/agents/shared/skills/core/openrig-skills/SKILL.md b/packages/daemon/specs/agents/shared/skills/core/openrig-skills/SKILL.md index 51a9f4d9b..15010c65e 100644 --- a/packages/daemon/specs/agents/shared/skills/core/openrig-skills/SKILL.md +++ b/packages/daemon/specs/agents/shared/skills/core/openrig-skills/SKILL.md @@ -120,6 +120,13 @@ and a useful report to the OpenRig team if you are still stuck. If `rig` won't r `daemon/docs/reference/help.md` inside the installed `@openrig/cli` package, or https://www.openrig.dev/help/agents (the same text). +## Developing OpenRig itself + +If you're changing OpenRig's own source in a clone of the openrig repository, use that repository's +`developing-openrig` skill (in the checkout's `.claude/skills/` or `.agents/skills/`). It maps the architecture, the +areas where a small change has a large effect, and which tests to run before a pull request. + ## Need more than what ships here? -This index covers the **shipped** surface. A dev host carries far more (factory, architecture, PM-craft, studio skills) reached through the host's own routers/codemaps — if you're on a builder host and need something not listed above, that deeper routing is the next hop, not a wall. (Host-scale routing is the subject of the context-routing architecture doc; at product scale, this one file is the whole map.) +This index covers the **shipped** surface. Your project, rig or machine may add its own skills and routers. Check the +skills your working directory loads before concluding that something doesn't exist. diff --git a/scripts/skill-edge-digests.generated.json b/scripts/skill-edge-digests.generated.json index 3e107cb04..eb08ffb89 100644 --- a/scripts/skill-edge-digests.generated.json +++ b/scripts/skill-edge-digests.generated.json @@ -10,7 +10,7 @@ "core/openrig-architect/SKILL.md": "0c3f1b7ff5bf185c9da963905229e951ffe1fb6d45a3cc9247cb747fc1d9a59d", "core/openrig-cmux/SKILL.md": "73a86eac45d68b16a38fc0a4438713c634d86061073dac96a4d9e656ea5d34dc", "core/openrig-herdr/SKILL.md": "37508a02f50bb69ee97c22c37ccf70aa8e37dc0784061eb2a5ad813571eebd1a", - "core/openrig-skills/SKILL.md": "fee88b59cf454156ab7d04bae9ae01ce6bba01a89643f3c39fea7425df247457", + "core/openrig-skills/SKILL.md": "e5f47e24a4c3cb1afceb1a3a93647313e9c30830aff58faabb5fa57838d1206c", "core/openrig-software-factory/SKILL.md": "33930e72341661eaaa9d3c5ac945297926f430d643b6f279fcb7f693ac30b7f7", "core/openrig-software-factory/references/worked-example.md": "ff5485a0e84de43188cf791cc96258abe038d035d439997f21865e33a1916bc5", "core/openrig-upgrade/SKILL.md": "c3555c66abe0426e2990aec024849f644cf43320a65b8f1211e96d3b234f6e83", @@ -80,7 +80,7 @@ "openrig-operating-model/templates/PLAYBOOK.md": "ff6f9dc8caab06dc7b9ae21977310ddefa23e925453bd54629ce0a669403bfb0", "openrig-operating-model/templates/SOP.md": "1eaf0daf4f9bbf0715e6a5aaf0f29f030af5fe8b8a10232488ee97c507afa6dc", "openrig-operating-model/templates/SPEC.md": "f126536daa59cea3f1cfb0898d83c7cdf99f6392f6f155ea64fe55187dd0f5f1", - "openrig-skills/SKILL.md": "fee88b59cf454156ab7d04bae9ae01ce6bba01a89643f3c39fea7425df247457", + "openrig-skills/SKILL.md": "e5f47e24a4c3cb1afceb1a3a93647313e9c30830aff58faabb5fa57838d1206c", "openrig-user/SKILL.md": "ed43062826294ad6879c125197b7d5752396372ac80b25a2f36e89afd617ae33", "orienting-to-an-inherited-seat/CHANGELOG.md": "4d0970e75e5c15b00a48d6a7150376d646257777264fd1b9a191165a8d5a6e84", "orienting-to-an-inherited-seat/SKILL.md": "4d2bb1f0dd522b0b956bf2191ce2121c36590936e217121f781525f77f8229a9", @@ -106,7 +106,7 @@ "core/openrig-architect/SKILL.md": "0c3f1b7ff5bf185c9da963905229e951ffe1fb6d45a3cc9247cb747fc1d9a59d", "core/openrig-cmux/SKILL.md": "73a86eac45d68b16a38fc0a4438713c634d86061073dac96a4d9e656ea5d34dc", "core/openrig-herdr/SKILL.md": "37508a02f50bb69ee97c22c37ccf70aa8e37dc0784061eb2a5ad813571eebd1a", - "core/openrig-skills/SKILL.md": "fee88b59cf454156ab7d04bae9ae01ce6bba01a89643f3c39fea7425df247457", + "core/openrig-skills/SKILL.md": "e5f47e24a4c3cb1afceb1a3a93647313e9c30830aff58faabb5fa57838d1206c", "core/openrig-software-factory/SKILL.md": "33930e72341661eaaa9d3c5ac945297926f430d643b6f279fcb7f693ac30b7f7", "core/openrig-software-factory/references/worked-example.md": "ff5485a0e84de43188cf791cc96258abe038d035d439997f21865e33a1916bc5", "core/openrig-upgrade/SKILL.md": "c3555c66abe0426e2990aec024849f644cf43320a65b8f1211e96d3b234f6e83", diff --git a/skills/_canonical/core/openrig-skills/SKILL.md b/skills/_canonical/core/openrig-skills/SKILL.md index 51a9f4d9b..15010c65e 100644 --- a/skills/_canonical/core/openrig-skills/SKILL.md +++ b/skills/_canonical/core/openrig-skills/SKILL.md @@ -120,6 +120,13 @@ and a useful report to the OpenRig team if you are still stuck. If `rig` won't r `daemon/docs/reference/help.md` inside the installed `@openrig/cli` package, or https://www.openrig.dev/help/agents (the same text). +## Developing OpenRig itself + +If you're changing OpenRig's own source in a clone of the openrig repository, use that repository's +`developing-openrig` skill (in the checkout's `.claude/skills/` or `.agents/skills/`). It maps the architecture, the +areas where a small change has a large effect, and which tests to run before a pull request. + ## Need more than what ships here? -This index covers the **shipped** surface. A dev host carries far more (factory, architecture, PM-craft, studio skills) reached through the host's own routers/codemaps — if you're on a builder host and need something not listed above, that deeper routing is the next hop, not a wall. (Host-scale routing is the subject of the context-routing architecture doc; at product scale, this one file is the whole map.) +This index covers the **shipped** surface. Your project, rig or machine may add its own skills and routers. Check the +skills your working directory loads before concluding that something doesn't exist.