Compare commits

..
Author SHA1 Message Date
CodeWhale BotandClaude Sonnet 5.5 c5f449df11 execpolicy,engine,docs: align shell spelling classification, safe-command guards and ANSI-C expansion
Tighten shell classification, safe-command admission, flag filtering, and ANSI-C
escape decoding so command classification matches POSIX shell tokenization and
auto-review boundaries:

- Treat any `$` expansion (including zsh `$=v`, `$~v`, `$^v`), non-plain
  command words (`=name`, glued globs `/bin/(r)m`, `rm#`), glued parentheses
  (outside `name()` and `name=(`), shell builtins (`alias`, `enable`, `setopt`,
  `unsetopt`, `emulate`, `shopt`, `autoload`, `zmodload`, `hash` with operands),
  `$'`...'` words, and PowerShell `iex`/`Invoke-Expression` as dynamic; refine
  quoted fallback parsing and document deny-rule coverage in
  AUTHORIZATION_ORDER.md.
- Require no shell composition and a leading primary command token (no env
  assignments or redirection prefixes) before admitting codewhale read-only
  invocations, and parse readonly tokens with strict shlex quoting.
- Normalize Windows trailing backslashes once before lexing and splitting in
  is_safe_command so both paths inspect the same command form.
- Guard rg --hostname-bin, ag --pager, info -o/--output, less/more output and
  log-file flags, and git stash list writable options under literal-word rules.
- Decode ANSI-C escapes (including control/newline escapes and nested escaped
  quotes) in shell expansion and multi-word shell-c destroyer-floor walking.
- Add CHANGELOG.md entry synced to crates/tui/CHANGELOG.md and regression unit
  tests across codewhale-execpolicy and codewhale-engine.

Checks (local, hermetic):
- cargo fmt --all -- --check: exit 0
- sh scripts/preflight.sh --check: preflight: OK
- nextest (-p codewhale-execpolicy -p codewhale-engine -E 'package(codewhale-execpolicy) | test(execpolicy) | test(auto_review) | test(command_safety) | test(shell_expand)'): 338 passed, 7164 skipped
- clippy (-p codewhale-execpolicy -p codewhale-engine --lib --tests -- -D warnings): exit 0
- Independent pro review (a4abbeec-c415-488b-b351-74cbbc98b3d3, 3d745acf-f658-4c11-a4ff-d6a67e9705e0): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 19:00:00 -08:00
CodeWhale BotandClaude Sonnet 5.5 dee906d48a Mask armored PEM preambles, spaced JSON containers, escaped assignments, and plural token keys
Unify key sensitivity classification in codewhale-sanitize::redact (shared KeySensitivity and TOKEN_METRIC_NAMES) so both sanitize and workflow treat plural token keys and credential-shaped tokens consistently while keeping tokenizer and token-count metrics visible. Extend PEM block detection to longer PGP armor preambles with Comment URLs and 16-character wrapped lines, handle spaced nested JSON objects and arrays across workflow disclosure and routing normalization, and scan multi-backslash, escaped-quote, and \u00XX-delimited assignments while preserving O(n) scaling and PR 7318 compatibility.

Checks:
- nextest: codewhale-sanitize, codewhale-workflow, codewhale-engine test(redact): 333 passed, 7221 skipped
- nextest --lib: codewhale-config, codewhale-secrets, codewhale-telemetry: 942 passed, 2 skipped
- clippy -D warnings: codewhale-sanitize/workflow (--all-targets), codewhale-engine (--lib --tests), codewhale-config/secrets/telemetry (--lib): 0 warnings
- cargo fmt --all -- --check, sh scripts/preflight.sh --check, git apply --check pr7318.patch: passed
- release scaling probe (5k vs 20k): 1.34x-4.20x across 8 pathological shapes (max 26.221ms)
- Independent pro review (10a7e23c-db69-4cad-b4d9-0999dba98d90, 79c55068-69c2-4b3c-b533-d94e4b4b26d0): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:58:40 -08:00
CodeWhale Bot e2336e9790 Refine PEM lookahead, escaped assignments, plural token metrics, and spaced arming guards
Sanitizer
- Replace single-next-line PEM body lookahead with bounded multi-line
  preamble lookahead (up to 8 lines across blanks, line gutters, diff
  markers, armor metadata headers, and comment/concat wrappers) and lower
  the PEM body base64-run threshold to 24 chars, while allowing unclosed
  same-line bodies to open a multi-line block and recognizing JS comments
  and `\u000a`/`\u000d` escapes after headers.
- Handle escaped quotes (`\"`, `\'`) and embedded newline/tab escapes
  carrying nested `NAME=value` assignments inside JSON string values via
  shared `split_value_quotes`, `redact_single_keyed_assignment`,
  `redact_backslash_assignments`, and `credential_word_core`.
- Guard `SpacedAssignment::advance` against arming on tokens containing
  `=` or `:` (after stripping any trailing escape chunk) so complete
  inline `--flag=value` options do not arm the following argument; expose
  `normalize_sensitive_key` for reuse in `codewhale-workflow`.

Workflow redactor
- Replace bare `tokens` in `TOKEN_METRIC_NAMES` with an explicit closed
  list of model/tokenizer metric keys and normalize keys via
  `codewhale_sanitize::redact::normalize_sensitive_key` in
  `is_token_metric_name`, ensuring plural credential keys (`API_TOKENS`,
  `ACCESS_TOKENS`, `SLACK_BOT_TOKENS`) always match while metric keys
  stay visible and bare `tokens` remains shape-checked.
- Strip `{` and `}` in `unwrap_token`, preserve armed state across spaced
  opening brackets (`[`, `{`, `(`) while clearing on closing punctuation,
  split on all ASCII whitespace via `split_whitespace_preserving` to
  preserve newlines and clear pending assignment names across lines, and
  guard `pending_name` against arming on tokens containing `=` or `:`.

Checks: codewhale-sanitize 25 passed; codewhale-workflow 260 passed (244
lib, 16 integration); codewhale-config --lib 832 passed (1 ignored);
codewhale-secrets --lib 55 passed (1 ignored); codewhale-telemetry --lib
53 passed; clippy (`codewhale-sanitize` and `codewhale-workflow`
`--all-targets -- -D warnings`) clean; `cargo fmt --all -- --check`
clean; `git diff --check` clean; `git apply --check pr7318.patch` clean.
2026-11-13 18:57:33 -08:00
CodeWhale Bot 0e7e102653 Handle plural token names, prefixed PEM blocks, JSON escapes, and compact members
Sanitizer
- Allow multi-line private-key PEM blocks to open after another PEM
  boundary or after a line prefix (such as line-number gutters, diff
  markers, quotes, comments, or string wrappers) when nothing
  alphanumeric follows the header and the next line carries a base64
  run of at least 40 characters; close the block on any line containing
  the PEM footer.
- Recognize `\n`, `\r`, `\t`, `\b`, `\f`, and `\uXXXX` escape sequences
  ahead of bare credential prefixes in `mask_bare_secret_tokens` and
  `redact_structured_word`.
- Remove the doc comment on private `SpacedAssignment::FlagValue`.

Workflow redactor
- Demote `token`-only matches that overlap `TOKEN_METRIC_NAMES` (such as
  plural `tokens`) to shape-checked matching instead of discarding them,
  across both inline assignments and pending/armed multi-token forms,
  keeping numeric counts and labels visible while masking opaque values.
- Split compact JSON and query-string tokens on `, ( ) [ ] { } & ?` in
  `redact_token` so non-first sensitive members and path values are
  checked per segment on both disclosure and routing paths.

Checks: codewhale-sanitize 24 passed, codewhale-workflow 260 passed (244
lib and 16 integration), codewhale-config 832 passed (1 ignored),
codewhale-secrets 55 passed (1 ignored), codewhale-telemetry 53 passed,
codewhale-engine filtered (`tools::github::report fleet redact`) 551
passed (6672 filtered out); clippy on the two changed crates clean;
cargo fmt clean.
2026-11-13 18:57:32 -08:00
CodeWhale BotandClaude Sonnet 5.5 880ed1c100 Scan the rest of a key line, URL userinfo and flag values for credentials
Sanitizer
- A line holding a private-key PEM has the key body masked and the rest of
  the line goes through the normal line scan, before and after the key and
  after the footer of a multi-line key. PEMs embedded in JSON or written
  after NAME= are covered; several PEMs on one line are each masked.
- PGP private key blocks and headers written without the trailing dashes
  open a multi-line block again.
- Bare credential scanning also splits on @ / + # and %, so user:token@host
  URLs, vault/KEY paths, +KEY diff lines and %-suffixed tokens are masked.
  It is the last fallback for every word and runs for a wrapped value that
  follows a sensitive name under the model-bound policy, so quoted,
  bracketed and tab-indented credentials are masked there too.
- A sensitive flag (--api-key, --password, --token) masks the word that
  follows it; --max-tokens style flags stay visible.
- Rename redact::is_sensitive_key to names_a_credential so it no longer
  shares a name with sanitize::is_sensitive_key.

Workflow redactor
- One-line and embedded private-key PEMs are masked by the shared scanner
  wherever the header sits, including a PGP block, instead of requiring a
  token that begins with the header.
- Names written without separators (SLACKTOKEN, apitoken, OPENAIAPIKEY)
  match by substring again, except token-metric names such as max_tokens.
- :=, => and == after a secret name are assignment operators: the value that
  follows is masked and the operator is kept.
- A secret flag masks its value word.

Checks: codewhale-sanitize 22 passed, codewhale-workflow 260 passed (244
lib and 16 integration), codewhale-config 832, codewhale-net 72,
codewhale-secrets 55, codewhale-telemetry 53; clippy on the two changed
crates clean.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:57:31 -08:00
CodeWhale BotandClaude Sonnet 5.5 c20ef4ebdb Share one credential detector between the sanitizer and the workflow redactor
Credentials are now masked wherever they sit in a word: after NAME= or
NAME: under any name, inside quotes, brackets or JSON, next to non-ASCII
text, and in a single-line PEM private key. Hyphenated names such as
X-Api-Key, names without separators such as DBPASSWORD, and spaced
assignments such as KEY = value mask their value. Token counts such as
max_tokens=8192 stay visible.

The workflow disclosure redactor takes its credential and name decisions
from codewhale-sanitize instead of its own prefix and marker lists; path
rules, receipt kinds and the Authorization/Bearer handling are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:57:29 -08:00
CodeWhale BotandClaude Sonnet 5.5 71fa01314c skills: update existing installs and support local folders/archives (CW-8184)
PF-4 one-step skill mutation controller:
- install_remote updates matching .installed-from installs via update_with_registry or install_with_registry(..., update=true), keeps skills_state.toml activation state intact, reports replaced provenance, and still refuses unmarked directories.
- execute_one_step accepts local path forms and .tar.gz/.tgz archives, unpacks into a TempDir with bounded_tar limits, refuses links/special entries/unsafe paths, unwraps single wrapper directories, strips shipped reserved markers, preserves Unix executable bits, and installs/updates through copy_skill_package and write_installed_from_v2.
- Person-started global installs and updates write write_trust_v2 for verified digests; project, agent, CLI, and unset initiators leave trust unchanged.
- Adds SkillOneStepRequest, SkillOneStepReceipt, and SkillInitiator non-breakingly alongside existing SkillMutationRequest / SkillMutationReceipt callers.

Checks:
- Local macOS APFS unit tests (`mutation::onestep_tests`): 18 passed, 0 failed (`macos-onestep-tests.log`)
- Boat nextest (`-p codewhale-skills`): 207 passed, 0 skipped (`20261114T025254Z-pf4-nextest-124098.log`)
- Boat clippy (`-p codewhale-skills --all-targets`): exit 0 (`20261114T025305Z-pf4-clippy-43bdaf.log`)
- Boat downstream check (`-p codewhale-tui -p codewhale-app-server --all-targets` on base `371eafad0c72` before test-only change): exit 0 (`20261114T021127Z-pf4-wscheck-ef210b`)
- cargo fmt --all -- --check && sh scripts/preflight.sh --check: preflight: OK
- Independent pro review (8ec27dc9-2d95-4287-9bfa-abdcbcb39b7d, a195c281-2537-4359-9db7-ed023d7fc07d): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:57:03 -08:00
CodeWhale BotandClaude Sonnet 5.5 6a9f3b40b4 net,cli,engine,docs: harden private-network rules, IPv6 embedded forms, redirect hops and bundle guards
`/network allow` admits a loopback or private address only for a host that is
visibly local and explicitly typed: a `localhost` name (loopback resolution only)
or a local/private IP literal. Non-localhost DNS names resolving to private
addresses stay refused even when listed, IPv6 `deny` rules canonicalize across
bracketed and uncompressed spellings and beat `allow` rules, and wildcard rules
never match IP address literals. Callers pass `PrivateTargets` explicitly;
`skill-install` and `extension_host::dsh_network` pass `Refuse`.

Cloud metadata (`169.254.0.0/16`, `fe80::/10`, provider endpoints, and
IPv4-mapped/compatible/translated, `64:ff9b::/96`, all four RFC 6052 layouts in
`64:ff9b:1::/48`, 6to4 and Teredo embedded forms) and `0.0.0.0/8` are refused on
every hop, while multi-trailing-dot `localhost` names and literals fail fast
before DNS or ambient proxy routing.

`GuardedGet` validates every redirect hop and pins resolved addresses. Portable
config bundles reject on import (without echoing the entry value) and omit on
export visibly local `network.allow` entries including `localhost` names and
local-domain wildcards (`.localdomain`, `*.localdomain`, `.localhost`,
`*.localhost`, `*.dev.localhost`), and `config import` refuses cross-host
visibly local redirects. Docs, `CHANGELOG.md`, `crates/tui/CHANGELOG.md` and
`web/data/agent-docs.generated.json` are updated to match.

Checks (macOS arm64, Rust 1.100.0, hermetic HOME):
- nextest -p codewhale-net -p codewhale-skill-install: 142 passed, 1 skipped (ignored subprocess helper)
- cargo test -p codewhale-cli --lib config_bundles: 64 passed, 0 failed, 319 filtered out
- nextest -p codewhale-cli --lib config_bundles: 64 passed, 319 skipped
- nextest -p codewhale-engine --lib tools::web extension_host::dsh_network: 248 passed, 7027 skipped
- nextest -p codewhale-tui --lib commands::groups::utility::network: 7 passed, 5763 skipped
- clippy --locked -p codewhale-net -p codewhale-skill-install -p codewhale-cli -p codewhale-engine --all-targets --all-features -- -D warnings: exit 0
- cargo fmt --all -- --check, scripts/preflight.sh --check, web check-facts/check-cloud-facts/check-release-docs/check-runtime-routes: OK
- Independent pro review (00152352-6903-4f7e-bf70-ca77c4efc665, 1dc64bac-559e-404b-9ee8-9bd220b3ba8a): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:48:52 -08:00
CodeWhale BotandClaude Sonnet 5.5 473a71e29f feat(plugins): refresh the first-party catalog snapshot per release; replace only the embedded copy (CW-8185)
scripts/refresh-first-party-marketplace.sh rewrites the embedded snapshot from
the public marketplace head (through sync-marketplace.py) after refusing a head
whose remote entries lack a sha256, a version or platforms, or follow a floating
release URL. With --check it fails when the files the catalog installs from
(marketplace.json, plugins/, skills/) differ between the snapshot revision and
the head; commits that touch none of them do not make it stale.
release-candidate.yml runs the check in its resolve job, before the long
parity and artifact builds.

MarketplaceStore now supersedes a saved "codewhale" catalog only when its source
is a 40-hex revision of the embedded snapshot's repository other than the
embedded one. A catalog the desktop added from its bundled file, one a person
added from their own document or URL, and a moving-branch URL are kept. The
replacement also applies before a write, so a stale embedded copy cannot block
re-adding the embedded source.

The computer-use pin assertion no longer hard-codes a release URL; it reads the
snapshot, so the pin can move without editing the test. New tests cover the
sha256 rule, the replacement and the kept catalogs.

The snapshot value itself is unchanged here: the Computer Use pin follows the
marketplace bump being merged.

Checks:
- Boat nextest: 44 passed, 195 skipped (`20261114T010310Z-pf5-xl-test-840c52.log`); after comment removal: 238 passed, 1 skipped (`20261114T024137Z-pf5-r2-test-22d52e.log`)
- Boat clippy: exit 0 (`20261114T010322Z-pf5-xl-clippy-6ca495.log`, `20261114T024035Z-pf5-r2-clippy-df4762.log`)
- actionlint, shellcheck, release-workflows.test.js, check-bundled-plugin-claims.py: exit 0
- sh scripts/preflight.sh --check: preflight: OK
- Independent pro review (54f9ea44-86d1-4e5e-bd48-eeb1278a94cd): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:45:36 -08:00
CodeWhale BotandClaude Sonnet 5.5 11ad35d41d feat(plugins): apply installs, enables and MCP adds made by other processes when idle (CW-8183)
The plugin catalog stamp now includes a SHA-256 of the plugin state file, so an
install, enable or disable from another terminal is a catalog change by content
rather than by mtime. The registry refreshes its own stamp after its own state
writes, so a session's /plugin enable never looks like an outside change.

The TUI idle poll also digests the configured MCP config file. When a turn is
not running, an outside plugin or MCP change is applied through the existing
PluginRegistryChanged handler (transcript kept via SyncSession) and the
existing MCP reload; the transcript records the applied change and any added or
removed skills, and the engine's existing new-turn context_update appends the
skill delta and declares the tool-surface change, so the pinned prefix is not
spliced. While a turn runs, the toast stays and the change applies at the next
idle poll. A change to an active provider-declaring plugin says provider
changes apply in a new session.

The MCP digest is tracked in the TUI (not inside PluginCatalogStamp) because
the MCP config path is a per-session setting the plugins crate does not know.

Checks:
- Boat nextest: 60 passed, 5945 skipped (20261114T014906Z-pf3-xl-test-369f11.log); 2 passed, 235 skipped (20261114T015857Z-pf3-xl-tests-stamp-afa8c7.log)
- Boat clippy: exit 0 across -p codewhale-plugins --lib --tests (20261114T022305Z-pf3-xl-clippy-plugins-f426e8.log), -p codewhale-plugins -p codewhale-tui --lib (20261114T021037Z-pf3-xl-clippy-lib-81355c.log), and --lib --tests -A clippy::await_holding_lock (20261114T022307Z-pf3-xl-clippy-nolock-2d50ac.log); pre-existing 4 await_holding_lock tests fixed in cae4b853d8
- sh scripts/preflight.sh --check: preflight: OK
- Independent pro review (b77a1c47-62d3-4e2b-a9c6-36b819f4f499): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:41:11 -08:00
CodeWhale BotandClaude Sonnet 5.5 bd3bbe0451 feat(plugins): one-line install summary builder for plugins and DSH packages (CW-8182)
A pure function, install::build_summary, turns what an install already knows
into the single line a person reads afterward: name, version and source; on or
off; who approved; what it adds; which converted rows will not run and why
(manual ports and rows refused by the host version gate); that native rows need
the extension host while it is off; host services granted, shared, withheld or
not offered, with the pause-until-idle note; and that provider declarations
apply in a new session. An update names only what changed, plus a source
change and an allowed downgrade. Each clause carries a stable message id for
localization; the English text is the fallback. No caller yet: the shared
one-step function and the clients adopt it next.

Checks:
- Boat nextest -p codewhale-plugins: 10 passed, 236 skipped (20261114T010050Z-pf2-xl-test-c65100.log)
- Boat clippy -p codewhale-plugins: exit 0 (20261114T005929Z-pf2-xl-clippy-9d6161.log)
- sh scripts/preflight.sh --check: preflight: OK
- Independent pro review (0db7ce36-80c1-44e6-906b-0445fa9a151a): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:39:33 -08:00
CodeWhale Bot cae4b853d8 test(tui): allow clippy::await_holding_lock on gate lock tests
Allow clippy::await_holding_lock on the four #[tokio::test] functions in
crates/tui/src/tui/ui/tests.rs that serialize against
CHILD_TERMINAL_GATE_TEST_LOCK alongside synchronous #[test] callers,
matching existing suppressions in the same module.

Checks:
- Boat clippy (20261114T015733Z-tuiclip-clippy-d22a5e): cargo clippy --locked -p codewhale-tui --all-targets -- -D warnings -A clippy::uninlined_format_args -A clippy::too_many_arguments -A clippy::unnecessary_map_or passed (exit 0)
- sh scripts/preflight.sh --check (rebased): preflight: OK
- Independent pro review (620a76b8): LAND
2026-11-13 18:23:04 -08:00
CodeWhale Bot d7e88289cc engine(runtime_threads): add coalesced host-managed async completion wakes (CW-8215)
Add RuntimeTurnInputSource::AsyncCompletionWake and coalesced (~75ms) per-(manager, thread) host wake scheduling in RuntimeThreadManager for idle non-ACP threads when background shell jobs or direct child sub-agents finish after turn completion. Track sub-agent completion consumption across active turns via Resuming turn status markers so already-drained children do not re-wake, while keeping late or busy-turn completions buffered until terminal settlement and monitoring live background shell work until finished, cancelled, archived, or closed. Uses weak manager references in event pumps and idle monitors so dropping RuntimeThreadManager never leaks process-owner or event-pump drop guards.

Checks:
- Boat nextest (20261114T013254Z-h2wake-xl-005c7e): Summary [  10.824s] 327 tests run: 327 passed, 6938 skipped (-p codewhale-engine --lib -E 'test(runtime_threads)')
- Boat clippy (20261114T013306Z-h2wake-xl-clippy-c7154c): cargo clippy --locked -p codewhale-engine --lib --tests -- -D warnings passed (exit 0)
- Local hygiene: cargo fmt --all -- --check, sh scripts/preflight.sh --check, git diff --check passed; zero new // or /* comments
- Independent pro review (63ad0492): LAND
2026-11-13 18:21:37 -08:00
CodeWhale BotandClaude Sonnet 5.5 cdf8b544e3 feat(tui): the Extensions screen is called Plugins, its tabs filter by kind, and /plugin is a primary command (PL-01a)
The one terminal screen that lists hooks, plugins, the marketplace, skills
and MCP servers is titled Plugins. Its tabs read Hooks, Installed,
Marketplace, Skills, Connectors. ExtensionsTabPlugins becomes
ExtensionsTabInstalled and ExtensionsTabConnectors is new (ConfigSectionMcp
and HelpSkills are untouched, so the configuration panel and /help keep
their words); the pinned Extensions key count moves from 98 to 99 for
that. ExtensionsTitle, ExtensionsActionTabs, ExtensionsNoItems and
CmdHooksDescription change value. All 15 packs carry the changed strings,
machine-drafted, with no native review yet; ca drops its wrong
"Connectors" for the plugin tab.

/plugin leaves ADVANCED_DISCOVERY_COMMANDS and so moves to the Primary tier.
Bare /skills, /mcp, /hooks and /plugin still open the screen; no subcommand
changed. Identifiers keep their names.

The launch-card PTY test waited for the old title; it now waits for
"Plugins" together with "Connectors" (the final filter tab, kept visible
across all supported widths), and a new PTY test prints the screen at
80x24 and 120x32.

The tab row keeps every painted filter whole at any width: Marketplace
takes its compact label only when the row does not fit, and when it still
does not fit the leading filters scroll away so the active one stays in view
(at 40 columns /mcp opens on Connectors, which was clipped). The component
kind in recommended Marketplace rows keeps its MCP label. The screen-name
sentences in the MCP, skills, plugins and hooks guides and the zh_hans copies
say Plugins. The README front-page heading stays, because a heading change
makes all 18 README translations stale.

Checks (rebased on origin/main 812a29cc06, tree 8441b5ba80b):
- python3 scripts/check-tui-locale-parity.py: PASS (2669/2669 keys across all 15 packs)
- bash scripts/check-tui-product-vocabulary.sh: exit 0
- cargo fmt --all -- --check: exit 0
- cd web && node scripts/check-facts.mjs: OK (committed facts.generated.ts matches workspace)
- node web/scripts/check-runtime-routes.mjs: PASS (244 literal HTTP paths match source)
- sh scripts/preflight.sh --check: preflight: OK
- ./scripts/release/check-versions.sh: Feature release-note receipts OK; Version state OK
- Boat xlarge 20261114T010507Z-pl01a-loc-8639a8 (cargo nextest run --locked -p codewhale-localization): 54 passed, 0 skipped
- Boat xlarge 20261114T010514Z-pl01a-lib-7ca421 (cargo nextest run --locked -p codewhale-tui --lib -E 'test(/extensions|bare_.*opens|discovery|advanced|skills|mcp|hooks|plugin|palette_root|tier_lists|extension_host_tests|every_pack/)'): 344 passed, 5426 skipped
- Boat xlarge 20261114T010520Z-pl01a-freg-1ecb06 (cargo test --locked -p codewhale-tui --test feature_registry): 2 passed, 0 failed, 0 ignored, 0 measured, 0 filtered out
- Boat xlarge 20261114T010526Z-pl01a-pty-1173f3 (sh scripts/dev-test.sh tui-cucumber --features long-running-tests launch_card_pty -- --nocapture --test-threads=1): 11 passed, 0 failed, 6 ignored, 0 measured, 48 filtered out
- Boat xlarge 20261114T015130Z-pl01a-clippy-nolock-c58030 (cargo clippy --locked -p codewhale-tui -p codewhale-localization --all-targets --all-features -- -D warnings -A clippy::uninlined_format_args -A clippy::too_many_arguments -A clippy::unnecessary_map_or -A clippy::await_holding_lock): exit 0
  (strict run 20261114T010533Z-pl01a-clippy-a3d640 without -A clippy::await_holding_lock exited 101 solely on 4 pre-existing await_holding_lock errors in untouched crates/tui/src/tui/ui/tests.rs lines 31976, 32033, 36178, 36222 from origin/main 43c87b4599)

- Rebased on 01259e18ca: check-facts, check-runtime-routes, check-tui-locale-parity (2669/2669), preflight: OK, check-versions: OK
- Independent pro review (643ede0d): LAND

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 18:20:36 -08:00
CodeWhale Bot 01259e18ca fix(extension-host): gate dsh_settings through BrokerContext::act and add changelog (CW-7983)
Route dsh/settings reads and writes through BrokerContext::act so every broker call checks invocation budget and emits sealed broker_acts receipts on the tool result:

- Describe records a BrokerEffect::Read act against the plugin's authority name.
- update/replace/mutate validate caller posture (write_refusal) and reviewed roots/keys/values (validate_change) BEFORE calling BrokerContext::act(..., BrokerEffect::Write, ...) so invalid or out-of-scope edits fail cleanly without opening a permission gate prompt.
- Re-check write_refusal(call.context.tool()) after act() returns to close any live-posture TOCTOU window (switching to ApprovalMode::Never or Auto while the gate card is pending refuses the disk write).
- Wire BrokerActTool::all() into Fixture::new and add settings_acts_pass_the_gate_and_record_receipts covering read recording, pre-gate root rejection, allowed write receipts, gate denial ((error_code::DENIED, "denied")), and post-act live-posture revocation.
- Add an honest 0.10.3 note in CHANGELOG.md and sync crates/tui/CHANGELOG.md.

Checks (rebased on b7249e5e58):
- Boat xlarge nextest (20261114T010800Z-dsh09-test-78979b.log): 234 passed, 0 failed, 7046 skipped
- Boat xlarge clippy (20261114T010815Z-dsh09-clippy-c570dc.log): exit 0
- npm --prefix crates/engine/extension-host run typecheck: exit 0
- node --test crates/engine/extension-host/test/settings.test.mjs: 3 passed, 0 failed
- sh scripts/preflight.sh --check: preflight: OK
- Dual pro reviews: Correctness (edf327db) LAND, Bypass (7475b8a1) LAND
2026-11-13 17:55:14 -08:00
CodeWhale BotandClaude Fable 5.2 1dc5bb71cb feat(extension-host): fill the dsh/broker settings arm over the user's plugin config (CW-7983)
The settings arm decodes four typed methods (describe, update, replace,
mutate). Any other method is a typed unknown_method refusal and malformed
input is invalid_input. No method takes a namespace, a file or a tool name:
the plugin's name, its reviewed setting roots and the config file come from a
SettingsAuthority value, never from the wire.

BrokerCall does not carry that authority yet (the plugin's reviewed record
does not name setting roots, and the grant slice has not landed), so serve()
runs with none and refuses every call with not_available before it looks at
the method. serve_with() holds the whole behaviour and is what the grant
slice plugs into. Authority without a config file, a plugin name or at least
one reviewed root is also not_available: a missing policy is a refusal.

Reads and writes reuse the existing owners. describe reads through
codewhale_runtime::config::read_plugin_settings and the plugin_config check
that activation uses (convert, now pub(super)), and shows only the reviewed
roots, the saved revision (the same digest reconcile compares) and whether the
running plugin already has it. Writes go through
config_persistence::mutate_config_document: under the shared config lock the
arm re-reads [plugins."<name>".config], compares expected_revision, applies
the change, sets or unsets only the leaves that differ under the reviewed
roots, re-reads the edited document through the same check (16 KiB, depth 16,
no date-time) and requires it to equal what was asked. Anything else aborts
the closure, so nothing is written. Comments, unreviewed keys in the plugin's
table, other plugins and every other table survive. A change applies at the
next reload, and every answer says so ("applies": "reload"); nothing is
shown as live.

A write is refused when the caller is a child agent or a worker under an
authority envelope, when the live posture (re-read at each call, not copied
from the start of the tool call) is Never or Auto-Review, when the session is
read-only, and when the session has no posture at all. The plugin is told
only that settings cannot be changed in this session; the reason goes to the
log. A stale expected_revision is an answer (conflict: true), not a refusal.
Writer failures are a typed refusal with no path in it.

Bounds: 64 KiB of input (counted with an early exit before decoding), 64
edits and 16 path keys per call, 128-character keys with no control
characters, no null and no integer outside i64. File work runs under
spawn_blocking, one settings task at a time for the whole process, and a call
whose cancellation has fired does nothing.

Known limits: mutate paths pass through tables only (an array is replaced
whole); the arm attaches no record to the tool result (BrokerContext has no
way to yet) and writes one log line naming plugin and roots, never values; a
write that has started is not interrupted by a later cancel.

TypeScript: src/dsh/settings.ts adds settingsAdapter (describe, update,
replace, mutate) over AdapterHost.broker. It refuses another namespace
locally and turns a conflict answer into SettingsConflictError. It is not
registered and settings is not a provided service until the Rust authority is
wired.

Checks: farm nextest -p codewhale-engine --lib extension_host: 191 run, 191
passed, 0 failed (4 new in dsh_settings). extension-host: typecheck clean,
build leaves the tree clean, node tests 452 (445 pass, 7 skipped, 0 fail,
3 new), bun 446 pass, 6 skipped, 0 fail.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
2026-11-13 17:44:27 -08:00
CodeWhale Bot b7249e5e58 feat(extension-host): fill the dsh/broker catalog arm: install plans, approved routes, read-only balance (CW-7988)
The catalog arm decodes four typed methods with `deny_unknown_fields`.
Any other method is a typed `unknown_method` refusal, malformed or
over-specified input is `invalid_input`, and nothing is passed through.

- `install.plan` `{package, version, registry?}` never installs. It checks
  that the invocation carries a `PluginRegistry` snapshot (`not_available`
  otherwise), passes the request to `plugins::install::dsh::plan_npm_install`,
  and returns `review_required` with the reviewed installer's spec and
  command or `already_present` with the next review step from
  `lookup_reviewable_plugin`. Ranges, tags, non-https registries, or
  invalid npm names are refused. No method installs, removes, or trusts.
- `routes.list` and `routes.resolve` `{provider, model?}` answer only with
  validated provider declarations from the calling plugin's active manifest
  (`call.owner.plugin_id`, never an input field). Before every answer,
  `verify_plugin_component_authority` rechecks `PluginActivationCapability::Providers`
  inside `spawn_blocking` with propagated policy and test env scope, so
  revoking review refuses routes immediately even while the snapshot is cached.
- `balance` `{}` takes no input and returns `known` (`BalanceInfo` fields),
  `unknown`, or `absent` without leaking credentials, tokens, or account IDs.
- All four catalog methods check `call.context.act(..., BrokerEffect::Read, ...)`
  so every call participates in the shared 50-call invocation budget, honors
  mid-call cancellation/withdrawal, and leaves a sealed `broker_acts` receipt
  without opening an interactive approval card.

TypeScript: `crates/engine/extension-host/src/dsh/catalog.ts` adds typed
helpers (`approvedRoutes`, `resolveRoute`, `accountBalance`, `planInstall`)
and unregistered adapters (`pluginManagerAdapter`, `deepseekAccountAdapter`),
rejecting credential or state-mutating account methods with `NotAvailable`.
`compatibility.json` and `DSH_ADAPTERS` remain closed until host services
are offered.

Also adds the `CW-7988` release note under `[Unreleased]` in `CHANGELOG.md`
and syncs `crates/tui/CHANGELOG.md`.

Checks (rebased on origin/main efab90bc71 / pre-rebase e6d832ab5a5 with identical code tree):
- Boat xlarge nextest (`20261114T010152Z-dsh11-test-9dadab`): 345 passed, 0 failed, 7165 skipped (`codewhale-plugins`, `codewhale-engine`)
- Boat xlarge clippy (`20261114T010158Z-dsh11-clippy-2beb96`): exit 0, 0 warnings (`-p codewhale-plugins -p codewhale-engine --lib --tests -- -D warnings`)
- JS extension-host (`node --test crates/engine/extension-host/test/catalog.test.mjs`): 3 passed, 0 failed (`471 passed, 7 skipped` across `test/*.test.mjs` at `e6d832ab5a5`)
- `sh scripts/preflight.sh --check` on rebased tip: preflight: OK
- Dual independent `pro` reviews (`92f64cd3`, `dbfa2202`): Verdict LAND
2026-11-13 17:39:27 -08:00
CodeWhale BotandClaude Fable 5.2 adddc5cc85 feat(plugins): DSH market npm sources plan a reviewed registry-tarball install (CW-7988)
A DSH market index entry published on npm was listed as not installable
("Codewhale does not execute npm"), although the reviewed installer
already fetches one exact version's registry tarball for
`/plugin install npm:<name>@<version>` and converts a DSH bundle it finds
there. The market parser now plans that same fetch, so
`/plugin marketplace install` and the Runtime API install route take an
npm-sourced DSH entry through the one reviewed installer: the network
policy gate, staging, DSH conversion, the name-conflict check, a bundle
that lands disabled and untrusted, and the hash-bound trust review.
Nothing runs npm and no dependency is installed.

One planner decides, `install::dsh::plan_npm_install`, so the catalog and
any later caller cannot disagree:
- the name and version must be what the installer's own `npm:` parser
  accepts (lowercase name, optional scope, exact canonical SemVer). The
  planner round-trips through that parser instead of keeping a second
  grammar, and refuses when the parsed address differs from its own.
- no declared registry, or the public npm registry, gives the installer's
  `npm:` spec.
- another declared registry is fetched as that registry's own tarball
  address and is never replaced by the default one. It must be an https
  URL without credentials, query or fragment.
- a package whose name cannot form a native plugin name is refused before
  any download, as an incompatible `engines.dsh` range already is.

A refusal names a registry by host only, so a credential written into an
index never reaches the reason text. A version whose `registry` is not a
string is ignored with an `INVALID_REGISTRY` warning; it does not fall
back to the default registry.

What this does not do: the index format carries no digest for an npm
source, so integrity here is the exact version over https, the existing
content-hash review and the tarball checksum in the install marker. A
registry that serves tarballs anywhere other than
`<registry>/<name>/-/<file>-<version>.tgz` fails at fetch with nothing
installed. Plain-http registries are refused.

The marketplace test that pinned "an npm source must stay unsupported"
now pins the plan, and one new test covers the planner's refusals and the
shared catalog resolver.

Checks (Linux x86_64 build farm, hermetic test home):
- `sh scripts/with-hermetic-test-home.sh cargo test --locked
  -p codewhale-plugins -p codewhale-engine --lib -- dsh marketplace
  extension_host`: codewhale-plugins 45 passed, 0 failed, 0 ignored,
  167 filtered out. Both DSH npm tests ran by name.
- `cargo clippy --locked -p codewhale-plugins -p codewhale-engine
  --all-targets -- -D warnings`: exit 0, 0 warnings.
- `cargo fmt --all -- --check` (local): exit 0.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
2026-11-13 17:37:54 -08:00
CodeWhale Bot efab90bc71 fix(engine): admit linked sibling worktree cwd under managed root (CW-8215)
When spawning a sub-agent with an existing cwd outside the parent
workspace directory, validate_existing_child_cwd previously refused every
path outside parent_workspace, preventing follow-up or read/verify agents
from targeting a sibling linked worktree created by worktree=true.

Admit canonical paths outside parent_workspace only when:
1. try_git_toplevel(parent_workspace) resolves the repository root;
2. the canonical directory sits strictly beneath default_worktree_root(&repo_root)
   (.codewhale-worktrees/<repo>/) and passes worktree_path_within_root
   via relative_to_root rebasing (preserving Windows verbatim prefix
   compatibility and rejecting symlink escapes and the container root);
3. git rev-parse --path-format=absolute --git-common-dir canonicalizes to
   the exact same common git directory for both parent_workspace and the
   target directory, rejecting foreign or spoofed checkouts and plain
   directories under .codewhale-worktrees/.

Also extract canonicalize_or_self and git_rev_parse_path helpers in
worktree.rs to avoid duplicating canonicalize fallback and git rev-parse
output parsing, keeping blocking-call sites below the file's budget.

Checks:
- Boat nextest (20261114T003449Z-h1cwd-xl-49f4fd, farm-b6a592 xlarge):
  cargo nextest run --locked -p codewhale-engine --lib -E 'test(worktree)'
  Summary [0.382s] 42 tests run: 42 passed, 7221 skipped
- Boat clippy (20261113T183820Z-h1cwd-10ab50):
  cargo clippy --locked -p codewhale-engine --lib --tests -- -D warnings (exit 0)
- Local: cargo fmt --all -- --check, sh scripts/preflight.sh --check (preflight: OK)
2026-11-13 17:04:04 -08:00
CodeWhale Bot 812a29cc06 docs(changelog): note execution rules for built-in tests, workflow verify and send_later
Checks: nextest -p codewhale-engine -E 'test(shell_denial) | test(workflow_verify) | test(completion_gate)' 17 passed (7260 skipped); clippy --locked -p codewhale-engine --lib --tests -- -D warnings clean; preflight: OK; cargo fmt --all -- --check and git diff --check clean.
2026-11-13 16:32:16 -08:00
CodeWhale BotandClaude Sonnet 5.5 c741890f52 fix(tools): the workflow verify option and its gate runner follow the Bash deny rule
A workflow started with verify=true runs the workspace's default verifier
gates when it completes. Under a Bash deny rule that call is now refused
before the run starts, and the gate runner itself refuses too, so a resumed
run or any other caller cannot reach it. A context that holds bounded
verification (a verifier child grant or a bounded tool authority) keeps it,
as it does for the Run tool.

The structural test that checks every code-executing action is denied or
named as an exemption now walks the full agent surface, including workflow
and agent, and checks named input flags that start execution. Tests also
cover the in-process verifier child and a verifier spawned under a Bash-denied
parent.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:56:50 -08:00
CodeWhale BotandClaude Sonnet 5.5 38f544ac98 fix(tools): a Bash deny rule also refuses the built-in test and verifier runners
The default test and verifier actions run the workspace's own scripts and
build code, so a Bash deny rule now refuses every verification call and
send_later schedule, not only the unbounded spellings. Contexts that hold
bounded verification by design (a verifier child grant or a bounded tool
authority) keep the default and filtered runs, so the verifier role still
works under its raw-shell denylist.

The pinned test now asserts the denial, and a sweep test checks that every
registered action declared as executing code is either denied by a Bash rule
or on a named, reasoned exemption list.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:56:49 -08:00
CodeWhale BotandClaude Sonnet 5.5 43c87b4599 fix(tui): keep the event a retired terminal reader consumes when it wakes (CW-7997)
A restart detaches a reader wedged inside event::read and dropped its
channel, so the key that reader returned once the tty woke (it holds the
crossterm reader lock, so it is the user's next keystroke) was lost. The
replaced receivers are now retained, at most four, and their Events are
delivered ahead of the fresh reader's while heartbeats and errors from them
are ignored; receivers whose thread exited are pruned. Dropping the pump
still detaches without joining. A second restart no longer discards the first
retired reader.

While retired receivers exist, recv_timeout waits in 25 ms slices and checks
them before every wait and before returning a fresh event, so a retired key is
normally delivered ahead of the fresh reader's and does not wait out the caller's
deadline. A fresh event read before a retired one is seen is held in a
one-slot stash and delivered next.

A retired reader that is still wedged can take a key meant for a later child
(editor or shell) session. Child handoffs now record their window (pause
acknowledged until resume or guard drop) with unique token ownership, bound
completed windows to eight after close without evicting open windows, and
discard retired events observed inside an active child window.

Checks:
- Boat farm 20261113T081432Z-leg03-repaired-664299: nextest -p codewhale-tui --lib (14 explicit LEG03 tests): 14 passed, 5748 skipped, exit 0
- Boat farm 20261113T182736Z-leg03post-729b5a: cargo test --locked -p codewhale-tui --lib -- input_pump: 1 passed, 0 failed, 5766 filtered out, exit 0
- Independent review: artifacts/codex-coordinator-20261113/go-leg03-repair/NATIVE-REVIEW.md approve final repaired source
- preflight.sh --check: OK

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:55:01 -08:00
CodeWhale Bot b1ae673717 docs(changelog): note delayed-message workspaces and shell stdin approvals
Checks (macOS arm64, Rust 1.100.0): nextest -p codewhale-engine --lib (tools::send_later, tools::approval_cache, tools::approval_summary): 56 passed, 7211 skipped; clippy --locked -p codewhale-engine --lib --tests -- -D warnings: clean; preflight: OK.
2026-11-13 10:53:16 -08:00
CodeWhale BotandClaude Sonnet 5.5 12bd04f4f9 docs(runtime-api): a shell run given stdin is granted as the exact call
The remember grant-scope paragraph now says that a shell run carrying stdin,
input or data is granted as the exact call, alongside shell interact and wait
calls. The generated agent docs are regenerated from it.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:35:15 -08:00
CodeWhale BotandClaude Sonnet 5.5 722ea6d262 fix(engine): send_later keeps its workspace inside the session; a shell grant covers stdin
send_later schedule now resolves an explicit workspace through the session's
path guard, as tasks and automations already do: a directory the session
cannot reach is refused and a relative one is stored resolved. The approval
summary for send_later lists the workspace as a labeled line.

A session grant for a shell run is keyed on the exact call when the call
carries stdin (stdin, input or data), so a grant given for a command does not
cover the same command with different input.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:35:13 -08:00
CodeWhale BotandClaude Sonnet 5.5 a0c4f69f5e test(tui): real-terminal thinking stream, stop, restore and sign-in notice at four widths (CW-7876)
Adds a PTY scenario to the existing cucumber harness, compiled only on unix
with long-running-tests: stream thinking, stop, and restore at four terminal
widths, the one-time sign-in notice with Esc dismissal, and a saved choice to
hide thinking surviving the new default.

Checks: farm run 20261112T073916Z-ENG-TUI-rebased-pty4-b3fc94, 3 tests run,
3 passed (1 slow), 60 skipped, exit 0.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-13 10:11:14 -08:00
CodeWhale BotandClaude Sonnet 5.5 77934f7a2a fix(engine): a build with test-support loads the account only from a loopback account service (CW-7876)
CI builds the acceptance binary with `cargo build -p codewhale-cli --bin
codewhale --all-features`, but the workspace `cargo nextest run --workspace
--all-features` step that follows rebuilds target/debug/codewhale, and that
build unifies the engine test-support feature in (app-server's test-support
enables it, and the cli integration tests make cargo relink the binary). The
binary the real-terminal tests launch therefore has test-support on. To
reproduce: build that way, then run the cucumber test
account_preferences_notice_is_shown_once_per_sign_in_and_esc_dismisses_it.

Under test-support profile_constitution::capture never read the account, so the
loopback account service in the test was never asked and the one-time sign-in
notice never appeared; the test waited its full 80 seconds on Linux and macOS
and failed.

Under test-support the account is now loaded only when
CODEWHALE_CLOUD_API_BASE normalizes to a loopback host. An unset variable or a
non-loopback base (including the production base some secrets tests export for
the whole process) still skips the account, so a test-support build never sends
a stored bearer to a remote account service and a developer shell that exports
the variable cannot leak a real session into an engine turn. Builds without
test-support are unchanged. The comment that claimed unit tests never read
account credentials is removed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:11:13 -08:00
CodeWhale BotandClaude Sonnet 5.5 586ebff854 fix(tui): a paused stream shows the thinking it already received (CW-7876)
When the provider paused mid-stream, reasoning already received stayed
unrendered until the next token arrived. The owed revision now renders on a
later display beat (render_owed_revision), so the transcript shows what has
arrived. Covered by a_paused_stream_renders_the_reasoning_it_already_received
and the PTY scenario.

Checks: farm tests run 20261112T061929Z-ENG-TUI-rebased-tests-d67557, 7557
passed / 10 skipped, exit 0; clippy run 20261112T061950Z-ENG-TUI-rebased-clippy-50d460
clean, exit 0; PTY run 20261112T073916Z-ENG-TUI-rebased-pty4-b3fc94, 3 of 3
passed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:11:12 -08:00
CodeWhale BotandClaude Sonnet 5.5 b42a651c78 fix(tui): the footer clock says thinking, like the transcript (CW-7876)
The footer phase word and clock said reasoning while the transcript header
said thinking. Both now say thinking, in the 12 locale packs that carried the
old word and the localization library; uk and ru keep their own wording. The
ui tests assert the new clock text.

Checks: farm tests run 20261112T061929Z-ENG-TUI-rebased-tests-d67557, 7557
passed / 10 skipped, exit 0; locale parity PASS; vocabulary check exit 0.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:11:10 -08:00
CodeWhale BotandClaude Sonnet 5.5 542e57e6d7 feat(settings): model thinking shows by default; a saved choice is kept (CW-7876)
Thinking traces are first-class: show_thinking now defaults to true in the
runtime Settings and the config schema, while thinking_default_expanded stays
false (live while streaming, collapsed once the answer starts). Docs, the
features.toml row, the zh-Hans docs, the config panel goldens and the derived
web json follow. A show_thinking value already present in settings.toml is
read as an explicit choice and kept. Note that earlier saves wrote the old
default into that file, so only installs without the key get the new default;
whether to migrate those is a founder decision recorded in the review packet.

Checks: farm tests run 20261112T061929Z-ENG-TUI-rebased-tests-d67557, 7557
passed / 10 skipped, exit 0, including feature_registry (2 passed); clippy run
20261112T061950Z-ENG-TUI-rebased-clippy-50d460 clean, exit 0; check-facts OK.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:11:08 -08:00
CodeWhale BotandClaude Sonnet 5.5 b7e47c7e07 fix(tui): the sign-in notice is a one-time footer notice that Esc dismisses (CW-7876)
The "account preferences not loaded" notice was pushed into the transcript
on every refresh, so it repeated and scrolled away with the conversation.
It is now a standing footer notice shown once per sign-in
(account_preferences_notice_shown), and Esc dismisses it while idle. The
message names the remedy within 40 columns and carries meaning in words, not
colour. New string added to all 15 locale packs and the settings schema.

Checks: farm tests run 20261112T061929Z-ENG-TUI-rebased-tests-d67557, 7557
passed / 10 skipped, exit 0; clippy run 20261112T061950Z-ENG-TUI-rebased-clippy-50d460
clean, exit 0; PTY run 20261112T073916Z-ENG-TUI-rebased-pty4-b3fc94, 3 of 3
passed; locale parity PASS; vocabulary check exit 0.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 10:10:30 -08:00
CodeWhale Bot 212b5c3ef0 fix(tui-kit): adopt missing upstream widget and ocean improvements
Selectively port nine runtime files from codewhale-ratatui 1d061703:
narrow action wrapping, scrollbar ranges, fractional progress, selected
controls, panel edges and measured ocean/caller-clock water APIs. Preserve
Engine's Terminal panel, Update metric, u64 total, public unbounded composer
and local protocol dependency. Record current upstream hashes and retained
adaptations without claiming a whole-repository mirror.

Keep the host adapter's known-dark pane boundary: broader kit Light support
must not paint Engine's authored dark field on an explicit light pane.
Update the frozen transcript test renderer for the intentional scrollbar
correction. The mounted TUI still controls theme and motion adoption.

Validation: 137 kit library tests and 6 consumer regressions passed,
including the near-fit scrollbar off-by-one case. Preflight, vendor format
and diff checks passed. Separate source review verified 14 upstream hashes
and all 5 retained adaptations. Focused mounted transcript/ocean tests are
still running; hosted combined CI and coordinator-ordered landing pending.
No tool-catalog or prompt change; no provider/deploy/release claim.

Refs CW-8211
2026-11-13 10:07:54 -08:00
CodeWhale Bot 767e3aa373 fix(show-data): record measured catalogs and finish feature validation
Record the intentional full-catalog addition from provider-free measurements:
Plan 33 to 34 tools, Act/Operate 54 to 55, each +1259 serialized bytes.
Active catalogs and system/context/discovery budgets are unchanged. The
conformance prompt family confirms the initial eager KV-cache prefix is
unchanged; loading show_data follows the existing deferred-tool lifecycle.

Correct the feature summary to fit the 120-character registry contract and
clarify that chart output includes a text summary, not every trend sample.
The direct kit dependency shares validation with the renderer; Engine already
has a transitive ratatui dependency through localization.

Validation: Engine 8 passed; TUI chart/restore 2 passed; conformance update
36 passed and compare-only 36 passed; feature-registry 2 passed after fixing
its initial 138-character summary failure; runtime budget 55 metrics pass;
preflight, format, version checks and locale parity (15 packs) pass.
Independent source, golden and measured-budget reviews completed.
Hosted CI and coordinator-ordered main landing remain pending.

Known limits: settled Unicode charts with the dark truecolor kit theme;
no draw-in motion, desktop rendering, installed/provider qualification,
release or deployment claimed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Refs CW-8206
2026-11-13 10:07:52 -08:00
CodeWhale Bot e3b6bd8c33 test(conformance): record the discoverable show_data registry entry
Update six event recordings and 15 tool snapshots for the added registry-only tool. A semantic comparison confirmed no other changes. The prompt-family eager catalog, system text and initial prefix hashes remain unchanged.

Validation: focused TUI tests 2 passed; conformance update 36 passed; compare-only conformance 36 passed. Engine tests and measured contract budgets are still pending.

Refs CW-8206
2026-11-13 10:07:51 -08:00
CodeWhale Bot bcb9d30cd3 feat(tui): render show_data charts in live and restored transcripts
Add a read-only show_data tool using the kit schema and validator. Render
sparkline, bar and trend charts, preserve recorded failure/interruption on
restore, and handle wide labels and translated captions.

The engine already depends transitively on ratatui through localization;
the direct kit dependency keeps tool and renderer validation identical.
The full tool registry gains one static tool; the eager tool list is unchanged.
Conformance and runtime-contract measurements remain pending for this CI
candidate. No claim of unchanged prompt bytes is made before those checks.

Validation so far: preflight passes; locale parity passes for all 15 packs;
format and version checks pass; independent source review found no remaining
blocking issues. Native tests, generated conformance receipts, runtime budget
measurement and hosted CI are pending. This candidate is not ready for main.

Known limits: settled charts only, dark truecolor theme adapted by the
terminal backend, Unicode glyphs, and no desktop rendering.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Refs CW-8206
2026-11-13 10:07:48 -08:00
CodeWhale Bot 47ff2e57f2 docs(web): refresh /app desktop showcase screenshots and copy for 0.10.3
Regenerated approval.webp, conversation.webp, and model-picker.webp via codewhale-app's reproducible record:desktop pipeline and updated APP_COPY.shots alt and caption copy in web/lib/content/app-page.ts to match the 0.10.3 desktop captures.

Checks: npm test (7/7 node tests, 93 files / 1321 vitest tests passed); check:facts, check:docs, check:tokens, tsc --noEmit, preflight: OK.
2026-11-13 09:48:52 -08:00
CodeWhale Bot 287bf7d18d runtime: pass only listed DOTNET_ runtime and SDK switches to child processes
Child environments now allow LC_* only for the standard POSIX/glibc locale categories (LC_ALL, LC_CTYPE, LC_COLLATE, LC_MESSAGES, LC_MONETARY, LC_NUMERIC, LC_TIME, LC_PAPER, LC_NAME, LC_ADDRESS, LC_TELEPHONE, LC_MEASUREMENT, LC_IDENTIFICATION) plus LC_TERMINAL and LC_TERMINAL_VERSION.

DOTNET_* variables now pass only when the suffix after DOTNET_ matches the listed host, SDK and runtime switches or an SDK-owned family (ROOT_, CLI_, SYSTEM_, ROLL_FORWARD, WATCH_, MSBUILD_SDK_RESOLVER_), and still refuse nested-configuration separators ('__' or ':'), compact secret-shaped words (KEY, CONNSTR, PWD, PASS, DSN, AUTH, BEARER, COOKIE) and the shared secret-name check (with CONNECTIONSTRING added).

Checks (macOS arm64, Rust 1.100.0):
- codewhale-runtime --lib: 922 passed, 2 skipped
- codewhale-engine / codewhale-dependencies (-E 'test(parent_secret_env) | test(child_env) | test(sandboxed_runner)'): 11 passed, 7306 skipped
- clippy --locked -p codewhale-runtime --tests -- -D warnings: clean
- cargo fmt --all -- --check: clean
2026-11-13 09:38:13 -08:00
CodeWhale Bot 214a6f7705 ci(credit): skip Gemini co-author trailers in release credit check 2026-11-13 09:36:57 -08:00
CodeWhale Bot 39d4de9324 docs(installers): add neutral CHANGELOG note and cap note on download_with_cap 2026-11-13 09:16:02 -08:00
CodeWhale BotandClaude Sonnet 5.5 e452e07e66 fix(installers): bound archive entry names and keep the stream reader failed after it trips
Skill and plugin archive scans and extracts now reject any entry whose
path or link target is longer than 4096 bytes before the name is copied
or split into components, returning a dedicated entry-name error. The
bounded stream reader stays in its failed state once the limit is
reached, so a caller that ignores one read error cannot read on. The
decoded stream limit is clamped to 4 GiB however large the configured
size cap is. The plugin scan's package.json read now uses the bounded
body reader and propagates read errors instead of discarding them.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 09:16:00 -08:00
CodeWhale BotandClaude Sonnet 5.5 6b731a8ab9 fix(installers): enforce the size cap on the decoded archive stream and on every body read
The skill and plugin tarball installers now read archives through one shared
reader that errors once the decompressed stream passes four times the size
cap, instead of relying only on sizes declared in entry headers.

- The scan passes sum entry.size(), which includes extended size records,
  rather than the plain header field.
- Entry bodies are read through a bounded helper that stops one byte past the
  remaining budget and reports the typed oversize error.
- A stream-limit error from the reader is mapped to the installers' existing
  oversize errors, so users see the size message.
- An archive whose headers and padding alone exceed four times the cap is now
  rejected; with the default cap that takes roughly 19,000 entries.
- codewhale-plugins no longer lists flate2 and tar as runtime dependencies;
  only its tests use them now.

Tests: 287 passed, 1 skipped across codewhale-skill-install and
codewhale-plugins (6 new regression tests; they fail on the previous code).
clippy --all-targets --all-features -D warnings clean for both crates.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 09:15:59 -08:00
CodeWhale BotandGemini 4 Hydrogen 022ff5475e fix(install): stop saying Computer Use is included and needs Node.js 20 (CW-8194)
Removed the macOS Node.js 20 / Computer Use notice from scripts/release/install.sh and web/public/install.sh, replaced the corresponding assertion in scripts/release/install-safety.test.js with a negative check, updated docs/INSTALL.md, docs/zh_hans/INSTALL.md, README.md, the 18 docs/i18n/README.*.md translations, CHANGELOG.md, and synced crates/tui/CHANGELOG.md and web/data/agent-docs.generated.json.

Checks:
- node --test --test-concurrency=1 scripts/release/install-safety.test.js: 74 tests, 74 passed, 0 failed, 0 skipped
- bash -n scripts/release/install.sh && sh -n web/public/install.sh: exited 0
- ./scripts/preflight.sh --check: preflight: OK
- rg -l "Computer Use is included|Node\.js 20 or newer on PATH" . --glob '!node_modules' --glob '!target' --glob '!.git': only ./web/data/agent-docs.release.json

Worker: agy 1.5.7 with gemini-4-hydrogen-medium; reviewed and landed by the Codewhale coordinator.

Co-Authored-By: Gemini 4 Hydrogen <[email protected]>
2026-11-13 09:02:57 -08:00
CodeWhale Bot 7cceaae0be docs: add CHANGELOG entry for chat bridge thread ownership 2026-11-13 08:46:06 -08:00
CodeWhale Bot 1b5e4a708c fix(bridges): preserve pending-ownership recovery until Runtime confirmation settles
Add ThreadStore.ownershipPending(chatId, threadId, scope) so recovery paths
distinguish a migrated grant still awaiting Runtime confirmation from a
disowned thread while keeping ownsThread fail-closed.

When Runtime thread reads fail temporarily at startup, Telegram keeps
retained admissions and active turns and waits in its recovery loop until
confirmation completes; Feishu skips pending chats without clearing
activeTurnId and reattaches settled threads from settleLegacyOwners'
onSettled retry hook; Weixin skips dropping unsent admissions on pending
threads.

Tests: bridge-core 77 (76 pass, 1 Windows-only skipped), wecom 22, feishu 19,
telegram 39, weixin 59, all passing; node --check clean in all six bridges.
2026-11-13 08:46:04 -08:00
CodeWhale BotandClaude Sonnet 5.5 c309b3e263 fix(bridges): a chat keeps a migrated thread only when the bridge started it
When an existing thread map is migrated, each chat's current thread is now
a pending grant. At start the bridge reads that thread once from the Runtime
and confirms the grant only if the thread carries this bridge's system prompt
and the configured workspace. A thread started from another client, a missing
thread or a mismatch drops the grant; an unreachable Runtime leaves it
pending, answered with the fixed message, and is retried every 30 seconds.
Per-account Weixin bindings were recorded by the bridge itself and are kept.

Approval decisions on a thread the chat does not own now get the same fixed
message in the WeCom, Feishu, Telegram and Weixin bridges. The shared
decideApproval takes an opt-in ownsThread check, so the DingTalk bridge is
unchanged.

The bridge system prompts become named constants used both when creating a
thread and when confirming one. READMEs describe the one-time confirmation.

Tests: bridge-core 76 (75 pass, 1 Windows-only skipped), wecom 22, feishu 19,
telegram 39, weixin 58, all passing; node --check clean in all six bridges.
The new cases fail against the previous commit's sources.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:46:03 -08:00
CodeWhale BotandClaude Sonnet 5.5 f363dfa346 fix(bridges): a chat that lost its thread at upgrade can start over, and recovery stops at unowned threads
WeCom /new now always creates a thread, as it already did for Feishu,
Telegram and Weixin, so a chat whose current thread is not its own can
recover with the command the fixed reply names.

A chat whose current thread it does not own no longer carries recovery
state onto it. Feishu clears the stored turn id instead of reattaching at
start. Telegram clears a stored turn, question or pending admission before
handling a command, before each recovery pass and before a prompt, so the
stale turn neither streams nor blocks /new. Weixin drops an unsent
admission aimed at an unowned thread and refuses /compact there with the
same fixed reply.

The READMEs say that an existing thread map keeps only the thread each chat
is bound to. The test timeout is raised and the explanatory comment on the
migration is removed.

Tests: bridge-core 60 run, 59 pass, 1 skipped (Windows-only); wecom 22,
feishu 19, telegram 39, weixin 56 pass. Six new tests fail on the previous
commit and pass here: WeCom /new from a shared thread and from an owned
thread, Feishu and Telegram restart with a stored turn id, Weixin
/compact and a queued admission on an unowned thread.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:46:02 -08:00
CodeWhale BotandClaude Sonnet 5.5 4697320348 fix(bridges): chats list and resume only the threads they created
The shared bridge thread store now records which chat created each Runtime
thread, outside the per-chat state so replacing a chat's state keeps it.
The WeCom, Feishu, Telegram and Weixin bridges record the owner when they
create a thread, list only owned threads (asking the Runtime for those ids
and filtering the answer again), and answer /resume for any other id with
one fixed message before any Runtime call. A chat whose current thread is
not owned gets the same message for prompts, interrupt and compact.

An existing thread map is migrated once on load: a thread only one chat
points at stays with that chat, and a thread several chats point at is
owned by none of them. Weixin's local binding list moves onto the shared
store and is imported by the same migration, keeping its account scope.

Tests: bridge-core 57 pass, 1 skipped (Windows-only); wecom 22, feishu 19,
telegram 39, weixin 54 pass. New: a runtime-level two-chat suite for the
WeCom, Feishu and Telegram bridges plus store unit tests.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:46:00 -08:00
CodeWhale BotandClaude Opus 5.5 8c659c36f6 chore(web): regenerate agent docs for the integrated 0.10.3 batch
Ten independently reviewed slices (SEAM-2, REF-04, REF-08, first-run,
quota error, RU-15, release checklist, telemetry collector, Pet lockfile,
Rust 1.100.0 pin) merge onto main without conflict. The combined crate
surface changes the generated agent-docs index.

Validation: web check-facts and check-release-docs OK;
scripts/preflight.sh --check OK. Exact-SHA CI and RC dispatched.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-13 08:38:30 -08:00
CodeWhale Bot 78d538b920 Merge commit 'e94fbf6e11' into coord-integrate-3 2026-11-13 08:38:28 -08:00
CodeWhale Bot 2649dc36ca Merge commit '00d3962085' into coord-integrate-3 2026-11-13 08:38:26 -08:00
CodeWhale Bot 41ae12e8cb Merge commit '705417fccd' into coord-integrate-3 2026-11-13 08:38:25 -08:00
CodeWhale Bot 05dc93df4e Merge commit '224764dc08' into coord-integrate-3 2026-11-13 08:38:23 -08:00
CodeWhale Bot 29943ab189 Merge commit '64b7bfa4af' into coord-integrate-3 2026-11-13 08:38:22 -08:00
CodeWhale Bot 7a7da5dceb Merge commit '4b1e691f3b' into coord-integrate-3 2026-11-13 08:38:20 -08:00
CodeWhale Bot 6136914486 Merge commit '346cde38bd' into coord-integrate-3 2026-11-13 08:38:18 -08:00
CodeWhale Bot 82ec8d9a7e Merge commit '61d068842b' into coord-integrate-3 2026-11-13 08:38:16 -08:00
CodeWhale Bot 51e2dce014 Merge commit 'db73296af6' into coord-integrate-3 2026-11-13 08:38:14 -08:00
CodeWhale Bot 611e6c09d1 Merge commit '0d8db3eb6b' into coord-integrate-3 2026-11-13 08:38:13 -08:00
CodeWhale BotandClaude Opus 5.5 4af8faa747 fix(toolchain): install rustfmt and clippy with the pinned 1.100.0 toolchain
Pinning channel = "1.100.0" made rustup install the pinned release with the
default component set on hosted runners, so rustfmt was absent in the Test jobs.
Five engine tests that format Rust through rustfmt then failed on all three OSes
with "rustfmt must be on PATH for the formatting tests" (CI 43387758446:
19419/19424 Linux and macOS, 18627/18632 Windows). The workflow contract forbids
per-workflow component adds, so the toolchain file owns the components.

Validation: .github/scripts/release-workflows.test.js OK; rustfmt resolves
under the pinned toolchain locally; scripts/preflight.sh --check OK. Exact-SHA
CI and RC dispatched. Refs CW-8205.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-13 08:38:11 -08:00
CodeWhale BotandClaude Sonnet 5.5 f05b2677ab build: pin source toolchains and container recipes to Rust1.100.0
Carry the saved RU01 and RU08 work together so local builds, workflows,
Docker, Dev Containers and CNB agree. Keep the minimum supported Rust1.89
job isolated for its entire execution. Retain existing release source
pinning, macOS timeout and optional compiler-cache guards.

Harden the existing contract test against misplaced MSRV selectors,
unknown or expression-valued toolchains, and quoted stale image tags.
The old Tencent example now follows the same version.

Checks: full Node workflow contract passed (211 included Rust inputs,
11 wrapper event cases across12 steps,13 hermetic invocation checks);
22/22 dev-cache checks;13/13 independent negative probes rejected;
prebuild, generated-data freshness, formatting and preflight passed.
No native/image/hosted qualification is claimed by these source checks.

Original saved changes:09431b60c593 and2c12736202b2.
Worker: Codewhale/OpenCode Go deepseek-v4.1-flash, medium effort.
Independent Codex review approved the final25-path source.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:38:10 -08:00
CodeWhale BotandClaude Sonnet 5.5 68864e2da4 Bring pet/tui/Cargo.lock in line with the workspace protocol crate versions
The lock still named codewhale-app-server-protocol 0.10.2 without its chrono
dependency, so cargo fetch --locked in the Pet conformance workflow refused.
Regenerated with cargo update --workspace; no other version changed.

Checks: current integrated cargo metadata --locked --offline passed;
201 packages resolved. Historical dedicated Pet workflow43216899528
passed4/4 jobs on the unchanged lockfile; current-parent Pet workflow
will be dispatched before landing. No native rebuild claimed here.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:38:09 -08:00
CodeWhale BotandClaude Sonnet 5.5 bc3a01af8d chore(telemetry): qualify schema 6 collector compatibility
Preserve schema1 through5 validation and append bounded schema6 fields without changing the existing model-use columns. Schema6 remains first-party only. This prepares collector source; client emission stays schema4 until founder deployment and live v4/v6 endpoint acceptance.

Checks:225/225 tests in8files, TypeScript typecheck, localWrangler bundle dry-run passed. Independent correctness and privacy reviews approved. Actual Codewhale/OpenCodeGo deepseek-v4.1-flash prepared read-only source audit; coordinator rederived source and ordering. No deployment or live endpoint proof.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:38:07 -08:00
CodeWhale Bot b880409ca1 docs: align release checklist with exact-source CI
Remove stale Windows compiler-cache and main-push coverage claims. Keep release-candidate acceptance separate from normal CI and record confidential counts only in the private release record. Existing gate checkboxes remain unchecked.

Prepared by agy Gemini Hydrogen, then corrected and independently reviewed by Codex. Node release-doc check and diff check passed; this documentation change does not claim any release gate or publication.
2026-11-13 08:38:06 -08:00
CodeWhale BotandClaude Sonnet 5.5 8733df6e2d docs: reword Windows smoke sccache checklist item as checkable (CW-7953)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:38:04 -08:00
CodeWhale BotandClaude Sonnet 5.5 3a5c1a5012 docs: release checklist gains RC SHA, windows sccache, telemetry and security gates
Docs only; no CHANGELOG line.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:38:03 -08:00
CodeWhale Bot 3c3389431f test(web): keep installer fixture directories private across umasks
Create the happy-path install and home directories with mode0700 so a
host umask0002 does not turn them into destinations that the installer
correctly refuses. Production ownership and permission checks are unchanged.

Checks on Linux with the exact two-line fixture patch:33/33 installer
contracts at umask0002 and33/33 at0022;7/7 Node tests;1306/1306 Vitest
tests across92files. OpenNext/Cloudflare packaging completed and Wrangler
local dry-run exited0. TypeScript passed after the normal prebuild step.
These are source/local packaging receipts, not a deployment.

Independent Codex source review approved. Earlier Gemini Hydrogen
website dependency and adapter repairs are retained unchanged.
2026-11-13 08:38:02 -08:00
CodeWhale Bot acb7d98847 fix(web): scope legacy lint allowances and retire the upstreamed adapter patch
Keep React lint errors enabled globally. Expose nine existing effect diagnostics as warnings on six exact component files and limit the server-clock purity exception to the revalidated home page. Preserve hydration, privacy and resource lifecycle behavior.

OpenNext 1.20.17 already includes server/preview-props.json, so remove the obsolete exact-byte monkey patch rather than rewriting the upstream loader. The old Linux packaging attempt failed its hash gate; a final clean packaging run is still pending.

Worker: agy Gemini Hydrogen, medium effort; independently reviewed by Codex. Independent ESLint checked 591 files with zero errors and nine warnings, and calculated configs prove exact exception scope. Pinned installed loader source/hash and its Next consumer match. Worker-reported test counts are not promoted to verified evidence without raw logs. No deploy or publication.
2026-11-13 08:38:00 -08:00
CodeWhale Bot fe9f51bdf4 chore(web): next 16.4.6, wrangler 4.161.0 and the OpenNext adapter update (CW-8205 RU-15)
Checks in web/:
- npm ci: exited 0 (880 packages)
- npm run build: exited 0 (Next.js 16.4.6 Turbopack, 979/979 static pages)
- npm run lint: exited 0 (0 errors, 0 warnings)
- npm test: 7/7 node --test (dependency-security.test.mjs), 92/92 vitest files (1306/1306 tests)
- npx tsc --noEmit: exited 0
- npm audit: 10 moderate before, 10 moderate after; npm run audit:dependencies exited 0 (10/10 mitigated, 0 blocked)

Worker: agy 1.5.7 with gemini-4-hydrogen-medium; reviewed and landed by the Codewhale coordinator.
2026-11-13 08:37:59 -08:00
CodeWhale Bot c772f689f4 fix(llm): reuse duration formatting and avoid repeated Go hints
Recover the saved quota-window fix using the existing command-contract duration formatter, so 70168 seconds reads as 19h29m and unknown resets omit the clause. Keep the OpenCode Go key guidance from repeating when an already-hinted error is displayed again. Extend the existing display regression and add mirrored release notes.

Worker: Codewhale on OpenCode Go deepseek-v4.1-flash, medium effort. Original saved Kimi-worker commit and author credit remain intact. Independent source review, rustfmt, changelog sync and diff checks passed. Focused exact-source Linux test is running; native and hosted gates remain required before landing.
2026-11-13 08:37:58 -08:00
CodeWhale BotandClaude Sonnet 5.5 88707e7ee7 fix(llm): print quota-window reset as plain text (CW-8209)
LlmError::QuotaWindow Display formatted retry_after with {:?}, leaking
Some(70168s)/None into user-visible text. Print the reset as plain text
("19 h 29 min", "H h", "N min", "N s") and omit the reset clause
entirely when no reset time is known. QuotaWindow and RateLimited gain an
OpenCode Go hint when the message names that provider.

Worker: Codewhale (kimi-k3 on OpenCode Go)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:37:56 -08:00
CodeWhale BotandClaude Sonnet 5.5 17b3dc558d fix: make first-run download and trust choices clearer
Retry installer downloads up to three times, remove failed partials,
and use explicit digit choices for directory trust. Keep configured
provider errors while clarifying the fresh custom-route placeholder.
Align the Chinese trust keybindings and mirrored release notes.

Original author: CodeWhale Bot <[email protected]>
Carried only these three original commits:
  cb1505a37773270451cacd66de438221d7a08e1e
  20bfae466dbd131373f0d0e2a08f9e090779c654
  2c1e49c99901ff7a9a2f791e8e3f1ecd3a371c3e

The old gen-strings --check current claim is withdrawn: prior reviews
found the app check red on both base and candidate. No app-string
qualification is claimed here. Hard-link delivery and the original
saved-working-endpoint report remain unresolved separate acceptance.

Integrated-source checks: installer safety 76/76; website contract 33/33;
prebuild, fmt and diff checks pass. preflight passes. Rust gates pending.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:37:55 -08:00
CodeWhale BotandClaude Sonnet 5.5 c90a25dd12 refactor: retire unused APIs and the duplicate agent catalog
Preserve the reviewed REF-08 deletions and retained locale consumers.
Move the remaining static model admission into route_preferences while
retaining push18 live-catalog ownership and custom-route enumeration.
Regenerate web facts/docs and document removed crate APIs.

Original author: CodeWhale Bot <[email protected]>
Squashed from:
  2268c0d8a749c8794acc71af92c58e227186734e
  051cdc69126d8bbf8ed89ba371d3e605beda1820
  b4d6e16f196f1e72955140ff7d33960ee76a5863
  2a4c733b387aa1e8b2da10e4f6224f2fa99ad1d6
  c13c580ab7dfe5ed10b452fda2987eda960abc62
  3e6bb3616f377b9614fe17e7e4da7c0db5595ef4
  7620f050fbfd5cd47c48213fcfaa404e3d5f74d1
  459c0933eda34453ff739d292e1b1b42f049c308
  863fb512eb4ba6f49fe987287b2db36589711f91

Checks on this integrated tree: fmt, preflight, provider-registry and
diff checks passed; native workspace/all-target/all-feature Clippy is running; tests pending. Preflight: 109 dead-code
suppressions, 666 blocking-call sites within budget, 244 runtime routes.
Historical exact-tip evidence at 863fb512eb: 147/147 small-crate tests
and engine/core/hooks/localization check passed. Earlier full-suite
counts belong to prior trees and are not claimed for this integration.
Independent integrated-source review approved; exact-candidate native and
hosted CI gates remain required before landing. Corrected retained schema
documentation and the mirrored API-removal count after review.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:37:53 -08:00
CodeWhale BotandClaude Opus 5.5 e10606c64b fix(subagent): remove the workspace-wide coordination lock
The v3 transaction lock and per-writer boot leases already own coordination.
Remove the lifetime process lock and its read-only fallback so one stale
session cannot keep later sessions read-only. Retry failed store admission
on later delegated writes, preserving image validation and lease ownership.

Pin an admitted Windows workspace directory without delete sharing. Keep
ordinary directory handles unchanged for plugin staging. The pin protects
the workspace and its ancestors, not children under .codewhale; that residual
boundary remains explicit in the review and release acceptance.

Integrated from 9a1e8e3b9f993a6adad30ad26d17a6d16e90737c; preserve the
original candidate and historical evidence. Updated lifecycle/architecture
docs, conformance fixtures, call sites and mirrored release notes together.

Validation on the integrated source: formatting, preflight and diff checks
passed (121 dead-code suppressions, 665 blocking sites, 244 runtime routes).
Independent source review approved. Focused Linux tests are running; exact
Windows convergence PASS and app-server compilation on all three platforms
remain required before landing. Earlier test counts are historical and are
not claimed for this integrated candidate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:37:52 -08:00
CodeWhale BotandClaude Sonnet 5.5 6feca24b5f feat(extension-host): gate service acts and retain sealed act receipts
Service acts use the session permission gate with child write/network
requirements. Reserve plugin management for the built-in creator module.
Record allowed and refused broker acts, and return those receipts when the
host throws, cancels or unregisters the tool. Seal the broker-act record
before returning so a late gate answer cannot authorize an unrecorded act.
Ordinary core-call receipts may remain in flight on cancellation; this does
not claim synchronous completion of all effects.

Preserves the reviewed SEAM-2 candidate and its real-host regression,
with complementary held-gate and successful-answer-after-seal tests.
Original candidate commits: 1d43f49bd41b55c287e2ced0702558cf2c10f3f9,
852332ad26 and 05c9de15c23394135129eedce9cc0f6f311e6fa6.

Checks on the frozen 22-file source over 622a493078:
- macOS mandatory Node selection: 277/277 passed; compiled Native: 2/2.
- Engine all-target Clippy with -D warnings: exit 0; fmt clean.
- Linux XL run 20261113T060722Z-seam2-xl-915835: 277/277 passed,
  6981 skipped, exit 0. The separate Large run is still pending.
The initial two missing-native-artifact failures were resolved by building
and explicitly supplying the supported fixture; original logs retained.
Integration adds mirrored changelog and regenerated docs; final candidate
full hosted CI and Windows qualification remain pending.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 08:37:48 -08:00
CodeWhale Bot 6ec8176f97 chore(changelog): document maintainer and cron secret rules 2026-11-13 05:34:06 -08:00
CodeWhale Bot d544046fb3 web: refuse periodic repeated patterns in maintainer and cron secrets
Update secretStrongEnough to reject values that repeat a pattern with a period up to half the secret length, closing repeated short strings that pass the 8-distinct-character floor while leaving random hex, UUID and base64 secrets accepted. Update the 503 unavailability messages, operator docs and weak-secret test cases accordingly.

Checks: web npm test (7 node + 1316 vitest in 92 files), npx tsc --noEmit, npm run lint, npm run check:docs and npm run check pass.
2026-11-13 05:34:05 -08:00
CodeWhale BotandClaude Sonnet 5.5 06a5f9bbe1 web: accept hex and UUID secrets, and refuse secrets that contain lone surrogates
The distinct-character floor for the maintainer token and the cron secret is now 8
instead of 16, so a random hex secret (16 possible symbols) or a UUID is accepted.
A secret containing an unpaired UTF-16 surrogate is refused, since such a value is
encoded as the replacement character when it is compared. The documentation states
the new floor and gives a command for generating a secret.

Checks: web npm test 7 node + 1316 vitest in 92 files, tsc, lint and check:docs pass.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 05:34:03 -08:00
CodeWhale BotandClaude Sonnet 5.5 bed977ba04 web: bind admin sessions to the maintainer token and apply one strength rule to both secrets
A session now records a SHA-256 fingerprint of the MAINTAINER_TOKEN it was
issued under, and validation requires that fingerprint to match the token
currently configured. Changing the token therefore ends every open admin
session, sessions stored without a fingerprint are no longer accepted, and
a session is only issued when the configured token meets the minimum. The
draft post route, merch operator route, merch interest list and admin page
pass the configured token to validation; the extra length checks added at
those gates are dropped because validation now covers them.

The minimum is one rule shared by MAINTAINER_TOKEN and CRON_SECRET: 32 to
512 characters, counted by code point, with at least 16 distinct
characters. 512 matches the sign-in form's submit limit. A CRON_SECRET that
does not meet it leaves the manual cron trigger answering 503 with the
requirement, before the limiter or the comparison. The admin page logs the
requirement when the configured token is too weak, so operators can find it
in the Worker logs.

Tests: session binding and legacy records (community-agent), rotation at
login and at the merch interest list, weak token and weak cron secret
patterns, and the exact minimum. Web suite 1314 passed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 05:34:02 -08:00
CodeWhale BotandClaude Sonnet 5.5 b6b3013e7e web: rate limit the manual cron trigger and require a 32-character maintainer token
The manual /api/cron trigger now consults the shared rate limiter under its
own key before comparing the submitted secret, answers 429 with Retry-After
when the budget is spent, and answers 503 when the limiter is missing or
fails, so the secret is never compared without a bound on attempts.

Maintainer sign-in treats a MAINTAINER_TOKEN shorter than 32 characters as
unavailable: the login route answers 503 with a message naming the minimum,
and the admin page, the draft post route, the merch operator route and the
merch interest list apply the same check so an existing session stops being
honoured while the configured token is too short.

Tests: cron route (5), admin login (6, two new), merch interest (one new);
existing fixtures that used short tokens now use 32+ characters.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-13 05:34:00 -08:00
376 changed files with 23028 additions and 8837 deletions

No files matched your search

+3 -3
View File
@@ -48,7 +48,7 @@
tags: cnb:arch:amd64
cpus: 16
docker:
image: rust:1.88-bookworm
image: rust:1.100.0-bookworm
# codewhale-cnb-bridge GitHub App credentials from the CNB KeyStore
# (codewhale.net/codewhale-ci-secrets, github-bridge.yml), injected as
# environment variables. Values are never printed.
@@ -112,7 +112,7 @@
tags: cnb:arch:amd64
cpus: 16
docker:
image: rust:1.88-bookworm
image: rust:1.100.0-bookworm
stages:
- name: install release dependencies
script: |
@@ -163,7 +163,7 @@ main:
$:
tag_push:
- docker:
image: rust:1.88-bookworm
image: rust:1.100.0-bookworm
stages:
- name: build linux x64 release assets (static)
# The static full-LTO link can also outlive CNB's default no-output
+1 -1
View File
@@ -1,4 +1,4 @@
ARG RUST_VERSION=1.88
ARG RUST_VERSION=1.100.0
FROM rust:${RUST_VERSION}-slim-bookworm
# Native DBus headers and pkg-config are required by libdbus-sys during
+1 -1
View File
@@ -4,7 +4,7 @@
"context": "..",
"dockerfile": "Dockerfile",
"args": {
"RUST_VERSION": "1.88"
"RUST_VERSION": "1.100.0"
}
},
"customizations": {
+1
View File
@@ -75,6 +75,7 @@ security boundaries and adds bridge-specific controls.
- **Token gate**: All runtime API calls carry `CODEWHALE_RUNTIME_TOKEN`
- **Chat allowlist**: Only chats/users listed in `WECOM_CHAT_ALLOWLIST` can interact. First-pairing mode (`WECOM_ALLOW_UNLISTED=true`) is meant for onboarding only
- **Approval required**: Tool calls from WeCom sessions must be approved — either via explicit `/allow <id>` commands or natural-language keywords (`允许`, `yes`, `ok`, etc.)
- **Thread ownership**: `/threads` and `/resume` cover only the threads the chat itself created; any other thread ID is refused with one fixed reply
- **No workspace exposure**: Only prompts, status summaries, and approval requests are sent to WeCom. Workspace contents, shell output, and runtime internals stay on the local machine
### Reporting WeCom Bridge vulnerabilities
+72 -2
View File
@@ -248,7 +248,7 @@ assert.match(nightly, /startsWith\(matrix\.target, 'aarch64-'\).*runner\.arch ==
const nightlyArmMuslSetup = namedStep(nightly, "Install Linux ARM64 musl toolchain");
assert.match(nightlyArmMuslSetup, /matrix\.target == 'aarch64-unknown-linux-musl'/);
assert.match(nightlyArmMuslSetup, /apt-get install -y binutils musl-tools/);
assert.match(nightlyArmMuslSetup, /rustup target add --toolchain stable aarch64-unknown-linux-musl/);
assert.match(nightlyArmMuslSetup, /rustup target add aarch64-unknown-linux-musl/);
const nightlyArmStaticSmoke = namedStep(
nightly,
"Verify static Linux ARM64 binary and launch",
@@ -373,7 +373,7 @@ assert.deepEqual([...new Set(valuesForKey(artifacts, "target"))].sort(), expecte
const releaseMuslBuild = namedStep(artifacts, "Build static Linux binaries (musl)");
assert.match(releaseMuslBuild, /endsWith\(matrix\.target, '-unknown-linux-musl'\)/);
assert.match(releaseMuslBuild, /apt-get install -y binutils musl-tools/);
assert.match(releaseMuslBuild, /rustup target add --toolchain stable \$\{\{ matrix\.target \}\}/);
assert.match(releaseMuslBuild, /rustup target add \$\{\{ matrix\.target \}\}/);
assert.match(
releaseMuslBuild,
/cargo build --profile dist --locked --target \$\{\{ matrix\.target \}\} -p codewhale-cli/,
@@ -934,6 +934,76 @@ assert.ok(
"release parity must allow the full CI suite's cold-build budget",
);
const pinnedToolchain = read("rust-toolchain.toml").match(/^channel\s*=\s*"([^"]+)"/m)[1];
assert.match(pinnedToolchain, /^\d+\.\d+\.\d+$/, "rust-toolchain.toml must pin an exact Rust release");
const msrv = read("Cargo.toml").match(/^rust-version\s*=\s*"([^"]+)"/m)[1];
assert.match(msrv, /^\d+\.\d+$/, "Cargo.toml must declare a minor MSRV");
const escapeRegExp = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const stripWorkflowComments = (source) =>
source
.split("\n")
.map((line) => line.replace(/(^|\s)#.*$/, ""))
.join("\n");
const workflowDir = path.join(repoRoot, ".github", "workflows");
const workflowSources = new Map();
for (const file of fs.readdirSync(workflowDir).filter((name) => /\.ya?ml$/.test(name)).sort()) {
workflowSources.set(file, stripWorkflowComments(read(`.github/workflows/${file}`)));
}
const ciSource = workflowSources.get("ci.yml");
const msrvHeader = /^ msrv:[ \t]*$/m.exec(ciSource);
assert.ok(msrvHeader, "ci.yml must keep an MSRV job");
const msrvStart = msrvHeader.index;
const afterMsrvHeader = msrvStart + msrvHeader[0].length;
const nextJob = /^ \S/m.exec(ciSource.slice(afterMsrvHeader));
const msrvEnd = nextJob ? afterMsrvHeader + nextJob.index : ciSource.length;
const selectorValue = (value) => value.trim().replace(/^(["'])(.*)\1$/, "$2");
const inMsrvJob = (index) => index >= msrvStart && index < msrvEnd;
for (const [file, source] of workflowSources) {
const selectors = [
...[...source.matchAll(/dtolnay\/rust-toolchain@([^\s]+)/g)].map((m) => ["action ref", m[1], m.index]),
...[...source.matchAll(/^[ \t]+toolchain:[ \t]*([^\r\n]*)$/gm)].map((m) => ["toolchain input", selectorValue(m[1]), m.index]),
...[...source.matchAll(/^[ \t]+rust-version:[ \t]*([^\r\n]*)$/gm)].map((m) => ["rust-version input", selectorValue(m[1]), m.index]),
...[...source.matchAll(/^[ \t]+RUSTUP_TOOLCHAIN:[ \t]*([^\r\n]*)$/gm)].map((m) => ["RUSTUP_TOOLCHAIN", selectorValue(m[1]), m.index]),
...[...source.matchAll(/\bcargo\s+\+\s*["']?([^\s"']+)/g)].map((m) => ["cargo + selector", m[1], m.index]),
];
for (const [kind, value, index] of selectors) {
if (kind === "action ref" && (/^[0-9a-f]{40}$/.test(value) || value === "master")) continue;
const expected = file === "ci.yml" && inMsrvJob(index) ? msrv : pinnedToolchain;
assert.equal(value, expected, `${file}: ${kind} ${value} must equal ${expected}`);
}
assert.doesNotMatch(source, /rustup (?:target|component) add --toolchain/, `${file}: rustup adds must follow rust-toolchain.toml`);
}
const msrvJob = ciSource.slice(msrvStart, msrvEnd);
assert.match(msrvJob, new RegExp(`RUSTUP_TOOLCHAIN: "${escapeRegExp(msrv)}"`), "the MSRV job must override rust-toolchain.toml for every step");
assert.match(msrvJob, new RegExp(`toolchain: "?${escapeRegExp(msrv)}"?`));
assert.match(msrvJob, new RegExp(`cargo \\+${escapeRegExp(msrv)} check --workspace --locked`));
assert.doesNotMatch(msrvJob, new RegExp(`\\b${escapeRegExp(pinnedToolchain)}\\b`), "the MSRV job must not select the pinned release toolchain");
const rootDockerfile = read("Dockerfile");
const devcontainerDockerfile = read(".devcontainer/Dockerfile");
const devcontainerConfig = read(".devcontainer/devcontainer.json");
const tencentCnb = read("deploy/tencent-lighthouse/cnb/cnb.yml.example");
const pinnedTag = `${pinnedToolchain}-bookworm`;
for (const [label, source] of [
["Dockerfile", rootDockerfile],
[".devcontainer/Dockerfile", devcontainerDockerfile],
]) {
assert.match(source, new RegExp(`^ARG RUST_VERSION=${escapeRegExp(pinnedToolchain)}$`, "m"), `${label} must default RUST_VERSION to the pinned toolchain`);
assert.match(source, /^FROM .*\brust:\$\{RUST_VERSION\}/m, `${label} must build from the RUST_VERSION image`);
}
assert.match(devcontainerConfig, new RegExp(`"RUST_VERSION":\\s*"${escapeRegExp(pinnedToolchain)}"`), ".devcontainer/devcontainer.json must pass the pinned RUST_VERSION build arg");
for (const [label, source] of [
[".cnb.yml", cnb],
["deploy/tencent-lighthouse/cnb/cnb.yml.example", tencentCnb],
]) {
const tags = [...stripWorkflowComments(source).matchAll(/^[ \t]*image:[ \t]*([^\r\n]*)$/gm)]
.map((m) => selectorValue(m[1]))
.filter((value) => value.startsWith("rust:"))
.map((value) => value.slice("rust:".length));
assert.ok(tags.length > 0, `${label} must declare a Rust build image`);
assert.deepEqual([...new Set(tags)], [pinnedTag], `${label}: Rust build images must match rust-toolchain.toml`);
}
console.log(
"Workflow contracts OK: 6-target/12-asset single-runtime nightly and exact-head 8-target/38-asset release candidate.",
);
+5 -5
View File
@@ -23,17 +23,17 @@ jobs:
continue-on-error: ${{ matrix.checks == 'advisories' }}
steps:
- uses: actions/checkout@v7
# rust-toolchain.toml pins `channel = "stable"` with no target, so
# rust-toolchain.toml pins `channel = "1.100.0"` with no target, so
# inside the action's alpine (musl) container rustup resolves it to
# `stable-x86_64-unknown-linux-musl` — which the container does not
# `1.100.0-x86_64-unknown-linux-musl`, which the container does not
# have. Without provisioning, the action's `rustup show` step prints
# "error: override toolchain 'stable-x86_64-unknown-linux-musl' is not
# "error: override toolchain '1.100.0-x86_64-unknown-linux-musl' is not
# installed" and then auto-installs mid-run (network-dependent ~12s
# detour, hard failure if the download stalls). `rust-version` makes the
# entrypoint run `rustup default stable` (same musl host triple) before
# entrypoint run `rustup default 1.100.0` (same musl host triple) before
# cargo-deny touches the workspace, so the toolchain file resolves to an
# already-installed toolchain on every run.
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check ${{ matrix.checks }}
rust-version: stable
rust-version: 1.100.0
+13 -11
View File
@@ -203,7 +203,7 @@ jobs:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- uses: actions/[email protected]
with:
node-version: 22
@@ -428,7 +428,7 @@ jobs:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
toolchain: 1.100.0
- uses: mozilla-actions/[email protected]
id: sccache
# The GitHub Actions cache backend is main-only, mirroring
@@ -487,7 +487,7 @@ jobs:
- uses: dtolnay/rust-toolchain@master
if: needs.changes.outputs.heavy == 'true'
with:
toolchain: stable
toolchain: 1.100.0
components: rustfmt, clippy
- uses: mozilla-actions/[email protected]
id: sccache
@@ -728,6 +728,8 @@ jobs:
if: github.event_name == 'workflow_dispatch' && needs.changes.outputs.heavy == 'true'
timeout-minutes: 45
runs-on: ubuntu-latest
env:
RUSTUP_TOOLCHAIN: "1.89"
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@master
@@ -751,7 +753,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- uses: mozilla-actions/[email protected]
id: sccache
# Main-only GitHub Actions cache backend; see the Safety gate job.
@@ -825,11 +827,11 @@ jobs:
if: needs.changes.outputs.heavy == 'true' && matrix.os == 'windows-latest'
shell: pwsh
run: ./scripts/installer/installer-path-regression.tests.ps1 -AllowUserPathMutation
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
if: needs.changes.outputs.heavy == 'true' && (matrix.os != 'ubuntu-latest' || github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request')
- name: Install macOS LLVM tools
if: needs.changes.outputs.heavy == 'true' && matrix.os == 'macos-latest'
run: rustup component add llvm-tools --toolchain stable
run: rustup component add llvm-tools
- name: Omit macOS build debug metadata
if: needs.changes.outputs.heavy == 'true' && matrix.os == 'macos-latest'
shell: bash
@@ -1222,7 +1224,7 @@ jobs:
if: needs.changes.outputs.heavy == 'true'
shell: pwsh
run: ./scripts/installer/update-user-path.tests.ps1
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
if: needs.changes.outputs.heavy == 'true'
- name: Check every workspace target compiles on Windows
if: needs.changes.outputs.heavy == 'true'
@@ -1267,7 +1269,7 @@ jobs:
runs-on: macos-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- uses: Swatinem/rust-cache@v2
with:
cache-bin: false
@@ -1305,7 +1307,7 @@ jobs:
run: echo "No executable Rust changes detected; preserving required npm wrapper smoke context."
- uses: actions/checkout@v7
if: needs.changes.outputs.heavy == 'true' && (matrix.os != 'ubuntu-latest' || github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request')
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
if: needs.changes.outputs.heavy == 'true' && (matrix.os != 'ubuntu-latest' || github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request')
- uses: mozilla-actions/[email protected]
id: sccache
@@ -1364,7 +1366,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- uses: mozilla-actions/[email protected]
id: sccache
# Main-only GitHub Actions cache backend; see the Safety gate job.
@@ -1423,7 +1425,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- name: Install Linux system dependencies
if: runner.os == 'Linux'
run: |
+1 -1
View File
@@ -48,7 +48,7 @@ jobs:
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
toolchain: 1.100.0
components: rustfmt
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
+2 -2
View File
@@ -67,7 +67,7 @@ jobs:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
toolchain: 1.100.0
targets: ${{ matrix.target }}
- uses: mozilla-actions/[email protected]
id: sccache
@@ -99,7 +99,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y binutils musl-tools
rustup target add --toolchain stable aarch64-unknown-linux-musl
rustup target add aarch64-unknown-linux-musl
- name: Build
shell: bash
# Nightly artifacts are disposable smoke binaries (14-day retention),
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
steps:
- uses: actions/checkout@v7
- name: Install Rust target
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@1.100.0
with:
targets: aarch64-unknown-linux-ohos
- name: Install OpenHarmony native SDK
+2 -2
View File
@@ -21,7 +21,7 @@ jobs:
node-version: '22'
cache: npm
cache-dependency-path: pet/package-lock.json
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- run: npm --prefix pet ci --ignore-scripts
- run: npm --prefix pet run check
- run: npm --prefix pet run sync
@@ -60,7 +60,7 @@ jobs:
cache-dependency-path: pet/package-lock.json
- run: npm --prefix pet ci --ignore-scripts
- run: npm --prefix pet run check
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
if: runner.os == 'macOS'
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
if: runner.os == 'macOS'
+2 -2
View File
@@ -170,7 +170,7 @@ jobs:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # master 2026-07-18
with:
toolchain: stable
toolchain: 1.100.0
targets: ${{ matrix.target }}
- uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11
if: runner.os == 'Linux'
@@ -215,7 +215,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y binutils musl-tools
rustup target add --toolchain stable ${{ matrix.target }}
rustup target add ${{ matrix.target }}
cargo build --profile dist --locked --target ${{ matrix.target }} -p codewhale-cli
- name: Configure Android NDK linker
if: matrix.target == 'aarch64-linux-android' && runner.os == 'Linux'
+3 -1
View File
@@ -35,7 +35,7 @@ jobs:
package-manager-cache: false
- uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # stable 2026-07-18
with:
toolchain: stable
toolchain: 1.100.0
- name: Match dispatch to the requested commit
id: source
shell: bash
@@ -68,6 +68,8 @@ jobs:
echo "sha=${actual}" >> "${GITHUB_OUTPUT}"
echo "version=${workspace_version}" >> "${GITHUB_OUTPUT}"
- name: Check the first-party catalog snapshot is current
run: ./scripts/refresh-first-party-marketplace.sh --check
- name: Check version and OHOS release contracts
# --require-dated-release, as auto-tag.yml and release.yml run it:
# release.yml only accepts an RC receipt for the exact tag SHA, so an
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # master 2026-07-18
with:
toolchain: stable
toolchain: 1.100.0
components: clippy, rustfmt
- uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11
id: sccache
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
contents: read
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- name: Install cargo-audit
run: cargo install cargo-audit
- name: Run cargo audit
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
with:
ref: ${{ inputs.expected_sha }}
fetch-depth: 1
- uses: dtolnay/rust-toolchain@stable
- uses: dtolnay/rust-toolchain@1.100.0
- uses: taiki-e/install-action@nextest
- name: Run focused tests
shell: bash
+35 -1
View File
@@ -7,7 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
- **Agent `cwd` in sibling worktrees:** spawning an agent with `cwd` pointing to an existing linked worktree (or a subdirectory inside it) under the repository's `.codewhale-worktrees/<repo>/` root is now accepted when both share the same repository common git directory; outside directories and foreign or plain directories under `.codewhale-worktrees/` stay refused. (CW-8215)
- **DSH market npm installs and extension catalog service:** DSH market `npm` entries with an exact semantic version (and optional `https` registry) now resolve to reviewed registry-tarball install plans (`npm:<pkg>@<ver>` or exact `.tgz` URL) without executing `npm` or bypassing plugin trust review. Inside the extension host, the `dsh/catalog` broker arm now plans reviewed installs without writing to disk, returns only verified approved provider routes scoped to the calling plugin, and serves account balance read-only while withholding credentials and account mutations (CW-7988).
- **Plugin settings service backend (0.10.3):** adds the internal `settings` host-service handler (`describe`, `update`, `replace`, `mutate`) and extension-host adapter helper so plugins can inspect and atomically edit only their own reviewed `[plugins."<name>".config]` keys in `config.toml` through the session permission gate and live write posture, leaving comments, unrelated tables and other plugins' configuration untouched (writes return `applies: "reload"` and take effect at the next reload). This backend is not yet wired into live host-service dispatch or plugin grants, so no active plugin exposes it yet. (CW-7983)
- **Runtime thread completion wakes:** idle non-ACP runtime threads automatically coalesce finished background shell jobs and completed child sub-agents into a bounded runtime continuation turn without dropping completions during busy turns or triggering wakes on cancelled, archived, or already-drained work. (CW-8215)
- **Plugin install summary builder (crate API):** `codewhale-plugins` adds `install::build_summary`, a pure one-line install and update summary builder with stable message IDs for plugins and DSH packages; no CLI, TUI or desktop caller uses it yet. (CW-8182)
- **Live reload for external plugin and MCP changes:** when another process installs, enables or disables a plugin or updates the configured MCP file, an idle TUI session now applies the change automatically and records any added or removed skills in the transcript (provider plugin changes still note they apply in a new session); while a turn is running, a toast notifies you and applies the change once idle. (CW-8183)
- **First-party plugin catalog updates:** saved `codewhale` marketplace catalogs that came from an earlier embedded snapshot revision now update to the shipped snapshot on load while leaving custom, moving-branch, desktop-bundled, and removed catalogs untouched; release-candidate checks also verify that the embedded snapshot matches the marketplace files it installs from. (CW-8185)
- **Network guard and portable config bundles:** `/network allow` now admits a local or private address only for `fetch_url`, `web.run`, and `config import` when the host is a `localhost` name (resolving to loopback only) or a local/private IP literal you typed and explicitly allowed (with IPv6 `deny` rules matching across bracketed and uncompressed spellings and wildcard rules never matching IP literals); domain names resolving to private addresses, skill and plugin installs, extension-host requests, and all cloud metadata forms (including `64:ff9b:1::/48` and `::ffff:0:0:0/96` embedded addresses) plus `0.0.0.0/8` stay refused on every hop. `codewhale config import <url>` now validates every redirect hop and refuses cross-host redirects to visibly local hosts, and portable config bundles reject on import and omit on export visibly local `network.allow` entries (including `localhost` names, local-domain wildcards such as `.localdomain` and `*.localhost`, and local IP literals).
- **Skill install and update controller:** installing an already installed skill now updates it in place (reporting any replaced source and preserving `skills_state.toml` enabled/disabled state), local folders and `.tar.gz`/`.tgz` archives install and update as managed skills with path-safety and archive-bound checks, person-started global installs are trusted automatically without marking project skills, and mutations emit a one-line outcome receipt. (CW-8184)
- **Removed (crate API):** the unwired `codewhale-hooks` dispatcher and its stdout, JSONL and Unix-socket sinks; `codewhale-core`'s `session`, `journal` and `ids` modules and the `hooks` argument of `Runtime::new`; `codewhale-runtime::resource_telemetry`; ten unreferenced public functions; and the duplicate `codewhale-agent` crate. `hook_sinks.unix_socket_path` still parses and is ignored. CLI, TUI and app-server behavior is unchanged.
- **First-run setup:** installer downloads retry up to three times and remove partial files after a failed attempt. Folder trust uses 1 to trust, 2 to continue untrusted, and 3 or Esc to quit; letters do nothing. Provider pickers consistently say "needs key", and the unconfigured custom placeholder says "needs endpoint" while configured-route errors remain visible.
- **Quota-window reset text:** a rate-limited turn now gives the reset time in plain words such as "19h29m" instead of Rust debug output like `Some(70168s)`, and leaves the reset clause out when the provider gives no reset time. An OpenCode Go message points at that provider's own key with `codewhale auth set --provider opencode-go`, without repeating a hint the message already carries. (CW-8209)
- **Usage-reporting collector:** accepts runtime schema 6 alongside existing versions and keeps schema 6 reports first-party only.
- **Source-build toolchain:** Rust 1.100.0 is pinned consistently across local source builds, CI, Docker, Dev Containers and CNB recipes. The minimum supported Rust version remains 1.89.
- **Terminal input recovery:** retains a wake event from a replaced reader, suppresses retired events observed during a child-terminal handoff, and closes only the handoff window owned by that scope.
- **Plugin index hooks:** hooks with the same name and event now have distinct row IDs, so clients can keep every configured hook visible.
- **Extension service approvals:** plugin service acts use the session permission gate and retain their approved or refused act records when a tool call fails. Closing a call seals its broker-act record so a late approval cannot authorize an unrecorded act. Ordinary nested-call receipts may still show an in-flight outcome when cancellation ends the call early.
- **Delayed-message workspaces and shell stdin approvals:** `send_later` `schedule` now keeps its workspace inside the session's reachable paths instead of accepting an arbitrary directory, and `send_later` approval prompts show the requested workspace; session shell grants now match exact inputs when `stdin` (`stdin`, `input`, `data`) is supplied instead of reusing a command-only grant across different `stdin`.
- **Installers no longer say Computer Use comes bundled:** the macOS install script and the install guide no longer print or state that Computer Use ships with the binary or needs Node.js 20. Computer Use is released separately and stays off until you turn it on.
- **Charts in the transcript:** the model can now show numbers as a small chart with the new read-only `show_data` tool: a one-row sparkline, up to 24 labelled bars, or a multi-row trend. The TUI paints it in the transcript with the terminal kit, including after a session is restored, and every value is also in words, so the model and other clients get the same facts as text. Input the chart cannot show honestly (a non-finite number, more than 1024 points or 24 bars, a label over 120 characters) is refused with the limit it broke.
- **Charts in the transcript:** the model can now show numbers as a small chart with the new read-only `show_data` tool: a one-row sparkline, up to 24 labelled bars, or a multi-row trend. The TUI paints it in the transcript with the terminal kit, including after a session is restored, and a plain-text summary accompanies each chart for the model and other clients. Input the chart cannot show honestly (a non-finite number, more than 1024 points or 24 bars, a label over 120 characters) is refused with the limit it broke.
- **Terminal widgets:** narrow action hints wrap, scrollbars reach the final row, progress bars show fractional cells, focused tabs and panels are clearer, and the shared terminal kit supports measured ocean themes and reduced-motion-aware water drift.
- **Renaming sessions with Chinese titles:** the session picker no longer rejects a Chinese, Japanese, Korean or emoji title of more than about 33 characters as "Title must be 1–100 characters". It now counts characters, as the message says and as `/rename` and the Runtime API already did, so any title up to 100 characters is accepted. Thanks [@luochen211](https://github.com/luochen211) (PR 7322).
- **Non-ASCII file names:** `/diff`, the `@git` and `@diff` mentions and the Runtime API workspace diff now show paths with Chinese, emoji and other non-ASCII characters as written instead of as quoted octal escapes, and the turn context no longer lists the workspace root as an empty "Active paths" entry. Thanks [@luochen211](https://github.com/luochen211) (PR 7317).
@@ -21,8 +45,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Plugin index (Runtime API):** a new `GET /v1/apps/plugin-index` route lists plugins, skills, connectors, hooks and hand-written commands with kind, source and state, and `/v1/commands` rows say which plugin (`plugin_id`) added a command and whether a command came from the project (`workspace_sourced`) (CW-8092).
- **Model commands and routing:** `codewhale model list` now lists every custom endpoint you have configured, under its own name. `codewhale model resolve` no longer refuses a model on a custom endpoint, router or local server whose model list has not been fetched yet; once a list is fetched, a model missing from it is still refused. Automatic routing marks a context window as verified only when the route itself supplies it, and no longer reports a guessed output limit.
- **ACP:** when a plugin tool asks the core to run a tool that needs approval, the editor now gets a `session/request_permission` on the plugin tool's call that names the nested tool and its input; before, the prompt ended with an internal error (CW-8050).
- **Website maintainer and cron secrets:** the manual cron trigger is rate limited, admin sign-in and the cron secret require a 32 to 512 character secret with at least 8 distinct characters that is not a repeated pattern, and changing the maintainer token signs out existing admin sessions.
- **Chat bridges thread ownership:** Chat bridges now list and resume only the threads the chat itself created: `/threads` shows owned threads, `/resume` refuses any other thread id with one fixed message, and `/new` always starts a new thread. In an existing thread map each chat keeps only the thread it is bound to, so threads it created earlier and left are no longer listed or resumable; a thread that several chats already pointed at is usable from none of them (send `/new`). Review `thread-map.json` after upgrading.
- **Skill and plugin installs:** skill and plugin installs now enforce the uncompressed size cap on the whole decompressed archive, and reject archive entries with names longer than 4096 bytes.
- **Built-in execution rules:** a `Bash` deny rule now also blocks built-in test and verifier runs (`Run` / `run_tests` / `run_verifiers`), `workflow` runs with `verify: true`, and delayed-message scheduling (`send_later` `schedule`) while keeping task reads, status checks, management actions, and the bounded verifier's own default verification runs working.
- **README and contributor credit:** the repository front page now opens the way the website does: the line "The model can be anyone's. The agent should be yours.", one sentence on what Codewhale is, the install command and one picture, with reference detail moved into the linked guides; the 18 translated READMEs follow the same structure and say they are machine-drafted. The contributor wall and the website now count the same 433 people (407 GitHub accounts and the commit authors whose email is not linked to an account), where the website had counted accounts only. `DEPENDENCY_SECURITY.md` moved into `docs/dependency-maintenance.md`, and `.github/SUPPORT.md` says where to ask for help.
- **Website Plugins page:** The website's Plugins page now describes what the released version does, and a labelled section says what 0.10.3 adds: one Plugins place, Standard and Creator mode, and the Permissions control.
- **Website `/app` captures:** the `/app` page's desktop build screenshots (`conversation.webp`, `approval.webp`, `model-picker.webp`) and their alt and caption text have been refreshed from the current 0.10.3 desktop UI.
- **First-time guide on the website:** a new docs page, "Use Codewhale for the first time" (`/docs/first-time`, English and Simplified Chinese), walks someone who has never used a terminal through opening one, installing, choosing the folder Codewhale works in and when that choice matters, opening a terminal in a folder from Finder or File Explorer, connecting a model, a first task, and stopping, undoing and quitting. The install page gains a short "New to the terminal?" block for macOS and Windows that links to it, and `llms.txt` carries the same steps.
- **Plugin docs and plugin-creator:** Docs now say everything you add is a plugin. The bundled plugin-creator skill no longer tells an agent never to trust or enable a plugin (in Full Access it may, for one it wrote and installed from its project), points code and screen plugins at cordis-plugin-development, and an untouched copy from 0.10.2 now upgrades. The Chinese extension and Runtime API docs now say editor ACP sessions get plugin tools and configured MCP servers.
- **Model thinking:** New installs show model thinking by default: it streams live in the transcript and collapses under a "Thought for …" header when the answer starts. Existing installs keep their saved setting; turn it on under Model reasoning in chat in /config.
@@ -103,6 +132,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- **Credential redaction:** text, JSON, and workflow-disclosure redaction now masks multi-line private-key blocks across PGP armor headers (including `Comment:` URLs), longer preambles, and 16-character wrapped lines; masks credential values inside nested or escaped JSON assignments, escaped quotes, `\u003c`/`\u003e` delimiters, and spaced or nested JSON objects and arrays; and treats plural token keys (`API_TOKENS`, `ACCESS_TOKENS`) and credential-shaped `tokens` values as credentials while keeping model and tokenizer metric names (`max_tokens`, `tokenizer`, `token_limit`, `tokens=5`) visible.
- Child processes now inherit only the standard locale variables (`LC_ALL`, `LC_CTYPE` and the other locale categories, plus `LC_TERMINAL` and `LC_TERMINAL_VERSION`) and a listed set of `DOTNET_` host, SDK and runtime settings. Other `DOTNET_` variables, including application configuration keys, are no longer passed to child processes or MCP servers.
- **Deny rules and command safety:** a command whose command word is not a plain literal, or that redefines a command name earlier in the same call, is now refused while a deny rule is configured; the background destroyer hold reads commands the shell tokenizer rejects and decodes ANSI-C escapes inside shell payloads; and `sed`, `find`, `sort`, `rg`, `fd`, `uniq`, `env`, `less`, `more`, `ag`, `info` and `git` count as read-only only when their words are literal and use read-only flags, with `codewhale --version` and Windows trailing-backslash paths normalized before checking.
- **Starting after a hard kill:** the Engine now starts when its process was killed and its data directory was then deleted. A socket owner record whose process is dead and whose data directory, the one this start selected, no longer exists is discarded instead of failing every start with "Failed to resolve ... tasks/runtime". A home reached through a symbolic link, such as one under macOS /tmp, is not covered yet.
- The install guide's note for a first message that gets no answer now describes v0.10.2: connect a model with `/provider`, and restart if messages stay saved for later after you connect. It described v0.10.0's key screen. The Chinese guide carries the same correction.
- Project activity on the site now comes from a server-side snapshot with a freshness record, the browser no longer calls GitHub, and count sentences use the right plural form in every language.
@@ -454,6 +486,8 @@ work is declared complete here. Items moved to 0.10.4 by name are marked below.
### Changed
- **Terminal Plugins screen:** The terminal's Extensions screen is now called Plugins; its tabs filter by kind (Installed, Skills, Connectors, Hooks, Marketplace), and `/plugin` is listed with the main commands. `/skills`, `/mcp` and `/hooks` work as before.
- **`read` pages in lines only (breaking).** One `read` call returns at
most 2,000 lines — the default and the maximum `limit` — inside a 50 KiB
output cap, and a line longer than 2,000 chars comes back truncated with
@@ -689,7 +723,7 @@ work is declared complete here. Items moved to 0.10.4 by name are marked below.
- **`codewhale exec --add-dir <path>`** (repeatable) and `addDirs` on `POST /v1/threads` let the file tools read and write a sibling directory, a second checkout or a shared scratch root without turning on Full Access.
- **Unsettled usage is persisted in production, not only in tests.** The interactive sink now records a legacy usage receipt when a provider response has no settlement, so children and turns stop reporting settlement failure.
- **A full usage journal no longer fails a turn before dispatch.** The request runs with cost unknown instead of stopping on “native usage journal capacity reached”.
- **Reverting a file works in any posture.** Restoring a file the thread's own turns changed no longer requires `/trust` or Full Access; the exact snapshot and reviewed-hash checks still apply. **`fetch_url` can reach a local or private host you allowed** with `/network allow <host>`; cloud metadata stays blocked. Both refusals now name the setting.
- **Reverting a file works in any posture.** Restoring a file the thread's own turns changed no longer requires `/trust` or Full Access; the exact snapshot and reviewed-hash checks still apply. **`fetch_url` can reach a typed `localhost` name or local IP literal you allowed** with `/network allow <host>` (domain names resolving to private addresses and cloud metadata stay blocked). Both refusals now name the setting.
- **`codewhale exec` offers tools by default**, still under the configured sandbox and approval posture. `--no-tools` is the one-shot. A pipe on stdin is the prompt when no arguments are given, or is appended to them.
- **A child no longer dies when usage settlement is not acknowledged.** The turn continues and cost stays unknown, matching the parent path.
- **Workspace memory works without a git origin.** A plain folder or a repo with no `origin` remote gets a stable workspace id. `codewhale plugin list` is an alias of `plugins status`. A heavy command runs after 30 seconds instead of waiting forever for an admission slot. An MCP registration 4xx names the status and the manual client-id path.
Generated
+10 -26
View File
@@ -912,14 +912,6 @@ dependencies = [
"zeroize",
]
[[package]]
name = "codewhale-agent"
version = "0.10.3"
dependencies = [
"codewhale-config",
"serde",
]
[[package]]
name = "codewhale-agent-contract"
version = "0.10.3"
@@ -943,7 +935,6 @@ dependencies = [
"chrono",
"clap",
"codewhale-account-cli",
"codewhale-agent",
"codewhale-agent-contract",
"codewhale-app-server-protocol",
"codewhale-build-support",
@@ -1046,9 +1037,11 @@ dependencies = [
"codewhale-engine",
"codewhale-execpolicy",
"codewhale-lane",
"codewhale-net",
"codewhale-paths",
"codewhale-release",
"codewhale-runtime",
"codewhale-sanitize",
"codewhale-secrets",
"codewhale-state",
"codewhale-telemetry",
@@ -1118,6 +1111,7 @@ dependencies = [
"codewhale-execpolicy",
"codewhale-paths",
"codewhale-protocol",
"codewhale-sanitize",
"codewhale-secrets",
"fd-lock",
"libc",
@@ -1141,7 +1135,6 @@ name = "codewhale-core"
version = "0.10.3"
dependencies = [
"codewhale-config",
"codewhale-hooks",
"codewhale-protocol",
"codewhale-state",
"regex",
@@ -1150,7 +1143,6 @@ dependencies = [
"sha2 0.11.1",
"tempfile",
"thiserror 2.0.21",
"tokio",
"uuid",
]
@@ -1185,7 +1177,6 @@ dependencies = [
"base64 0.23.1",
"chrono",
"clap",
"codewhale-agent",
"codewhale-agent-contract",
"codewhale-app-server-protocol",
"codewhale-build-support",
@@ -1206,6 +1197,7 @@ dependencies = [
"codewhale-paths",
"codewhale-plugins",
"codewhale-protocol",
"codewhale-ratatui",
"codewhale-release",
"codewhale-repl",
"codewhale-runtime",
@@ -1220,7 +1212,6 @@ dependencies = [
"codewhale-util",
"codewhale-workflow",
"codewhale-workflow-js",
"cucumber",
"dirs 7.0.0",
"encoding_rs",
"fd-lock",
@@ -1239,12 +1230,10 @@ dependencies = [
"objc2-foundation",
"parking_lot",
"portable-pty",
"pretty_assertions",
"proc-macro2",
"regex",
"reqwest",
"ring",
"rio-vt",
"rmcp",
"rusqlite",
"rustls",
@@ -1260,9 +1249,7 @@ dependencies = [
"tar",
"tempfile",
"thiserror 2.0.21",
"tiny_http",
"tokio",
"tokio-tungstenite 0.30.0",
"tokio-util",
"toml 1.1.6+spec-1.1.0",
"toml_edit 0.25.15+spec-1.1.0",
@@ -1306,14 +1293,11 @@ name = "codewhale-hooks"
version = "0.10.3"
dependencies = [
"anyhow",
"async-trait",
"chrono",
"codewhale-app-server-protocol",
"codewhale-paths",
"codewhale-protocol",
"codewhale-release",
"reqwest",
"serde",
"serde_json",
"tempfile",
"tokio",
@@ -1433,7 +1417,7 @@ version = "0.10.3"
dependencies = [
"codewhale-config",
"codewhale-runtime",
"codewhale-secrets",
"codewhale-sanitize",
"dirs 7.0.0",
"regex",
"tempfile",
@@ -1610,12 +1594,12 @@ dependencies = [
"codewhale-paths",
"codewhale-protocol",
"codewhale-release",
"codewhale-sanitize",
"codewhale-secrets",
"codewhale-util",
"colored",
"dirs 7.0.0",
"fd-lock",
"futures-util",
"ignore",
"jsonwebtoken",
"libc",
@@ -1733,9 +1717,11 @@ dependencies = [
"codewhale-runtime",
"codewhale-skill-install",
"codewhale-util",
"flate2",
"serde",
"serde_json",
"sha2 0.11.1",
"tar",
"tempfile",
"tokio",
"toml 1.1.6+spec-1.1.0",
@@ -1883,9 +1869,7 @@ dependencies = [
"reqwest",
"rio-vt",
"rquickjs",
"rusqlite",
"rustls",
"schemars 1.2.2",
"semver",
"serde",
"serde_json",
@@ -1895,7 +1879,6 @@ dependencies = [
"tempfile",
"tiny_http",
"tokio",
"tokio-tungstenite 0.30.0",
"tokio-util",
"toml 1.1.6+spec-1.1.0",
"toml_edit 0.25.15+spec-1.1.0",
@@ -1919,7 +1902,7 @@ version = "0.10.3"
dependencies = [
"codewhale-palette",
"codewhale-ratatui",
"codewhale-secrets",
"codewhale-sanitize",
"crossterm",
"ratatui",
"similar",
@@ -1960,6 +1943,7 @@ name = "codewhale-workflow"
version = "0.10.3"
dependencies = [
"codewhale-protocol",
"codewhale-sanitize",
"serde",
"serde_json",
"sha2 0.11.1",
-1
View File
@@ -1,7 +1,6 @@
[workspace]
members = [
"crates/account-cli",
"crates/agent",
"crates/agent-contract",
"crates/app-server",
"crates/app-server-protocol",
+1 -1
View File
@@ -12,7 +12,7 @@
# Or mount an env file:
# docker run --rm -it --env-file .env codewhale
ARG RUST_VERSION=1.88
ARG RUST_VERSION=1.100.0
# ── Stage 1: Build ────────────────────────────────────────────────────
FROM --platform=$BUILDPLATFORM rust:${RUST_VERSION}-slim-bookworm AS builder
+3 -6
View File
@@ -119,12 +119,9 @@ Connect [MCP servers](docs/MCP.md), add [skills](docs/SKILLS.md) and
[plugins](docs/PLUGINS.md), and run [hooks](docs/HOOKS.md) on session and tool
events. Code mode (experimental, on by default) lets the model write one
`execute_tools` program that calls several MCP and plugin tools; each call
still meets your approval rules and keeps its receipt. The Computer Use plugin
bundled with v0.10.2 (experimental, macOS only) can observe and operate other
apps, and it stays off until you review its access and enable it
([guide](https://github.com/codewhale-hq/Codewhale/blob/v0.10.2/crates/tui/plugins/computer-use/README.md)).
Coming in 0.10.3: the plugin is released
[separately](https://github.com/codewhale-hq/codewhale-cu-plugin).
still meets your approval rules and keeps its receipt. [Computer Use](https://github.com/codewhale-hq/codewhale-cu-plugin)
is a separate plugin released on its own (experimental, macOS only), and stays
off until you review its access and turn it on.
### Ways to run it
-15
View File
@@ -1,15 +0,0 @@
[package]
name = "codewhale-agent"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true
description = "Model/provider registry and fallback strategy for Codewhale"
[lints]
workspace = true
[dependencies]
codewhale-config = { path = "../config", version = "0.10.3" }
serde.workspace = true
File diff suppressed because it is too large. Load diff
-1
View File
@@ -32,7 +32,6 @@ codewhale-app-server-protocol = { path = "../app-server-protocol", version = "0.
anyhow.workspace = true
axum.workspace = true
chrono.workspace = true
codewhale-agent = { path = "../agent", version = "0.10.3" }
codewhale-config = { path = "../config", version = "0.10.3" }
codewhale-core = { path = "../core", version = "0.10.3" }
codewhale-hooks = { path = "../hooks", version = "0.10.3" }
+2 -33
View File
@@ -14,7 +14,6 @@ use crate::stdio::{BoundedLines, run_stdio_loop};
use anyhow::Context as _;
use anyhow::{Result, anyhow};
use axum::http::{Method, StatusCode};
use codewhale_agent::ModelRegistry;
use codewhale_app_server_protocol::rpc::{
RequestId, RpcCall, RpcErrorDetails, RpcErrorResponse, RpcResponse,
};
@@ -23,7 +22,6 @@ use codewhale_app_server_protocol::runtime::MAX_RUNTIME_IMAGE_BODY_BYTES;
use codewhale_app_server_protocol::runtime::RuntimeImageInput;
use codewhale_config::ConfigStore;
use codewhale_core::Runtime;
use codewhale_hooks::{HookDispatcher, JsonlHookSink, StdoutHookSink, UnixSocketHookSink};
use codewhale_protocol::EventFrame;
use codewhale_state::StateStore;
use serde::Deserialize;
@@ -364,7 +362,6 @@ pub struct AppState {
/// Bookkeeping config and read-only historical archive access.
/// Actual turns and transcript writes belong to the captured owner.
runtime: Arc<RwLock<Runtime>>,
registry: ModelRegistry,
auth_token: Option<String>,
#[cfg(test)]
runtime_bridge: Arc<Mutex<Option<SharedRuntimeBridge>>>,
@@ -797,46 +794,19 @@ async fn build_state_off_runtime(
fn build_state_with_transport(
config_path: Option<PathBuf>,
auth_token: Option<String>,
transport: AppTransport,
_transport: AppTransport,
) -> Result<AppState> {
let has_explicit_config_path = config_path.is_some();
let store = ConfigStore::load(config_path)?;
let config_path = has_explicit_config_path.then(|| store.path().to_path_buf());
let config = store.config.clone();
let registry = ModelRegistry::default();
let state_db_path = config_path
.as_ref()
.and_then(|p| p.parent().map(|parent| parent.join("state.db")));
let state_store = StateStore::open(state_db_path)?;
let mut hooks = HookDispatcher::default();
// Stdio carries JSON-RPC on stdout: printing raw hook events there
// corrupts the protocol stream (#5165). HTTP mode keeps the stdout
// sink for local development visibility.
if transport == AppTransport::Http {
hooks.add_sink(Arc::new(StdoutHookSink));
}
// The hook log sits beside the state database: next to an explicit
// config, otherwise in the directory the state store resolved under the
// Codewhale home. Never relative to the process cwd (#6513).
let hook_log_path = state_store
.db_path()
.parent()
.unwrap_or_else(|| Path::new("."))
.join("events.jsonl");
hooks.add_sink(Arc::new(JsonlHookSink::new(hook_log_path)));
if let Some(socket_path) = config
.hook_sinks
.as_ref()
.and_then(|sinks| sinks.unix_socket_path.as_ref())
.filter(|path| !path.as_os_str().is_empty())
{
hooks.add_sink(Arc::new(UnixSocketHookSink::new(socket_path.clone())));
}
let runtime = Runtime::new(config.clone(), state_store, hooks);
let runtime = Runtime::new(config.clone(), state_store);
Ok(AppState {
captured_owner: None,
@@ -846,7 +816,6 @@ fn build_state_with_transport(
config_path,
config: Arc::new(RwLock::new(config)),
runtime: Arc::new(RwLock::new(runtime)),
registry,
auth_token,
#[cfg(test)]
runtime_bridge: Arc::new(Mutex::new(None)),
+12 -5
View File
@@ -111,7 +111,14 @@ pub(crate) async fn runtime_app_models(state: &AppState) -> Result<Vec<Value>> {
providers.len() <= 256,
"Runtime provider catalog exceeds its limit"
);
let aliases = state.registry.list();
let aliases = codewhale_config::catalog::reviewed::bundled_reviewed()
.selections
.iter()
.filter_map(|row| {
codewhale_config::ProviderKind::parse_config_identity(&row.provider)
.map(|provider| (provider, row))
})
.collect::<Vec<_>>();
let mut models = Vec::new();
let mut routes = std::collections::HashSet::new();
for provider in providers {
@@ -149,13 +156,13 @@ pub(crate) async fn runtime_app_models(state: &AppState) -> Result<Vec<Value>> {
.context("Runtime model identity is invalid")?;
let compatibility_aliases = aliases
.iter()
.find(|entry| {
.find(|(provider, row)| {
parsed != codewhale_config::ProviderKind::Custom
&& provider_id.is_none_or(|identity| identity == kind)
&& entry.provider == parsed
&& entry.id == id
&& *provider == parsed
&& row.id == id
})
.map(|entry| entry.aliases.clone())
.map(|(_, row)| row.aliases.clone())
.unwrap_or_default();
let capabilities: codewhale_config::route::RouteCapabilities =
match model.pointer("/facts/capabilities") {
-75
View File
@@ -367,81 +367,6 @@ fn build_state_keeps_resolved_explicit_config_path() {
);
}
#[tokio::test]
async fn stdio_transport_never_registers_the_stdout_hook_sink() {
let tmp = tempfile::tempdir().expect("tempdir");
let config_path = tmp.path().join("config.toml");
fs::write(&config_path, "api_key = \"sk-deepseek-secret\"\n").expect("write config");
let http_state =
build_state_with_transport(Some(config_path.clone()), None, AppTransport::Http)
.expect("http state");
let stdio_state = build_state_with_transport(Some(config_path), None, AppTransport::Stdio)
.expect("stdio state");
let http_sinks = http_state.runtime.read().await.hooks.sink_count();
let stdio_sinks = stdio_state.runtime.read().await.hooks.sink_count();
assert_eq!(
http_sinks,
stdio_sinks + 1,
"HTTP mode keeps StdoutHookSink + JsonlHookSink; stdio must drop the stdout sink (#5165)"
);
}
// Points CODEWHALE_HOME at a temp dir for one test and restores it after.
// The variable is process-global, so these tests hold a lock.
static CODEWHALE_HOME_TEST_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
struct CodewhaleHomeGuard {
prior: Option<std::ffi::OsString>,
_lock: std::sync::MutexGuard<'static, ()>,
}
impl CodewhaleHomeGuard {
fn set(value: &Path) -> Self {
let lock = CODEWHALE_HOME_TEST_LOCK
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let prior = std::env::var_os("CODEWHALE_HOME");
// SAFETY: serialised by CODEWHALE_HOME_TEST_LOCK.
unsafe { std::env::set_var("CODEWHALE_HOME", value) };
Self { prior, _lock: lock }
}
}
impl Drop for CodewhaleHomeGuard {
fn drop(&mut self) {
// SAFETY: serialised by CODEWHALE_HOME_TEST_LOCK.
unsafe {
match &self.prior {
Some(value) => std::env::set_var("CODEWHALE_HOME", value),
None => std::env::remove_var("CODEWHALE_HOME"),
}
}
}
}
#[tokio::test]
async fn hook_log_without_config_path_goes_beside_state_db_not_cwd() {
let home = tempfile::tempdir().expect("tempdir");
let _home = CodewhaleHomeGuard::set(home.path());
let state = build_state_with_transport(None, None, AppTransport::Stdio).expect("state");
state
.runtime
.read()
.await
.hooks
.emit(codewhale_hooks::HookEvent::ResponseStart {
response_id: "resp-6513".to_string(),
})
.await;
let log = fs::read_to_string(home.path().join("events.jsonl"))
.expect("hook log beside state.db in CODEWHALE_HOME");
assert!(log.contains("resp-6513"), "{log}");
}
async fn response_body_json(response: Response) -> Value {
let bytes = to_bytes(response.into_body(), usize::MAX)
.await
+2
View File
@@ -34,10 +34,12 @@ codewhale-engine = { path = "../engine", version = "0.10.3" }
codewhale-app-server = { path = "../app-server", version = "0.10.3" }
codewhale-config = { path = "../config", version = "0.10.3" }
codewhale-lane = { path = "../lane", version = "0.10.3" }
codewhale-net = { path = "../net", version = "0.10.3" }
codewhale-workflow = { path = "../workflow", version = "0.10.3" }
codewhale-execpolicy = { path = "../execpolicy", version = "0.10.3" }
codewhale-paths = { path = "../paths", version = "0.10.3" }
codewhale-release = { path = "../release", version = "0.10.3" }
codewhale-sanitize = { path = "../sanitize", version = "0.10.3" }
codewhale-secrets = { path = "../secrets", version = "0.10.3" }
codewhale-state = { path = "../state", version = "0.10.3" }
codewhale-telemetry = { path = "../telemetry", version = "0.10.3" }
+334 -73
View File
@@ -15,6 +15,9 @@
//! - Input size is capped (5 MiB, matching the skill installer's cap).
//! - HTTPS only for remote fetch, except plain `http` on loopback; redirects
//! are followed at most a bounded number of times within the same scheme.
//! The first request and every redirect honor the `[network]` policy and
//! refuse loopback, private, link-local and cloud-metadata destinations,
//! except a visibly local host the person listed or typed as the URL.
//! - Bundle-declared file paths must resolve inside the target config
//! directory; traversal and symlink escapes are refused.
//! - Project scope never mutates the user-global document and vice versa.
@@ -379,7 +382,7 @@ fn url_carries_credential(text: &str) -> bool {
// `***`; count masks rather than compare bytes, because it re-encodes the
// query and would otherwise flag benign URLs.
let masks = |value: &str| value.matches("***").count();
if masks(&codewhale_secrets::sanitize::redact_url_for_display(text)) > masks(text) {
if masks(&codewhale_sanitize::sanitize::redact_url_for_display(text)) > masks(text) {
return true;
}
let host = url.host_str().unwrap_or_default().to_ascii_lowercase();
@@ -573,13 +576,29 @@ fn nonportable_path_reason(path: &str) -> Option<&'static str> {
None
}
fn is_network_allow_path(path: &str) -> bool {
let mut segments = path.split('.').map(normalize_bundle_key).rev();
matches!(
(segments.next().as_deref(), segments.next().as_deref()),
(Some("allow"), Some("network"))
)
}
/// Telemetry opt-out is safe to move between machines, but opt-in is durable
/// user consent coupled to SetupState. A portable bundle may tighten that
/// consent (`false`); it must never manufacture or transfer `true`.
fn nonportable_value_reason(path: &str, value: &toml::Value) -> Option<&'static str> {
let top_level_telemetry = !path.contains('.') && normalize_bundle_key(path) == "telemetry";
(top_level_telemetry && matches!(value, toml::Value::Boolean(true)))
.then_some("would port telemetry opt-in consent between machines")
if top_level_telemetry && matches!(value, toml::Value::Boolean(true)) {
return Some("would port telemetry opt-in consent between machines");
}
if is_network_allow_path(path)
&& let toml::Value::String(host) = value
&& codewhale_net::guard::is_visibly_local_host(host)
{
return Some("network.allow entry would grant local-network authority");
}
None
}
// ---------------------------------------------------------------------------
@@ -798,67 +817,85 @@ pub fn resolve_bounded_path(base_dir: &Path, candidate: &str) -> Result<PathBuf>
// ---------------------------------------------------------------------------
/// Fetch a bundle over HTTPS (or plain http on loopback only) with a hard
/// size cap, a timeout, and bounded redirects. Mirrors the skill installer's
/// fetch bounds.
pub fn fetch_bundle(url: &str) -> Result<Vec<u8>> {
let mut current_url = reqwest::Url::parse(url).map_err(|_| anyhow!("invalid bundle URL"))?;
validate_bundle_url(&current_url)?;
let initial_scheme = current_url.scheme().to_string();
/// size cap, a timeout, and bounded redirects. The first request and every
/// redirect go through the shared guarded download: the network policy
/// applies, the host is resolved and checked, and loopback, private,
/// link-local and cloud-metadata destinations are refused unless the person
/// listed a visibly local host in `network.allow`. A loopback host typed as
/// the URL itself is admitted without a listing.
pub fn fetch_bundle(
url: &str,
network: &codewhale_net::network_policy::NetworkPolicy,
) -> Result<Vec<u8>> {
let initial_url = reqwest::Url::parse(url).map_err(|_| anyhow!("invalid bundle URL"))?;
validate_bundle_url(&initial_url)?;
let initial_scheme = initial_url.scheme().to_string();
let initial_host = initial_url
.host_str()
.context("bundle URL must include a host")?
.to_string();
let client = codewhale_release::platform_blocking_http_client_builder()
.timeout(std::time::Duration::from_secs(FETCH_TIMEOUT_SECS))
// Redirect targets must pass the same scheme/host policy as the
// initial request, so redirects are followed explicitly below.
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|_| anyhow!("building bundle fetch client failed"))?;
let mut redirects = 0usize;
let response = loop {
let response = client
.get(current_url.clone())
.send()
// reqwest errors can include the full URL (including its query or
// userinfo), so keep transport failures deliberately URL-free.
.map_err(|_| anyhow!("bundle fetch request failed"))?;
if !response.status().is_redirection() {
break response;
}
if redirects >= MAX_REDIRECTS {
bail!("bundle fetch exceeded the five-redirect limit");
}
let location = response
.headers()
.get(reqwest::header::LOCATION)
.ok_or_else(|| anyhow!("bundle redirect is missing a valid Location header"))?
.to_str()
.map_err(|_| anyhow!("bundle redirect is missing a valid Location header"))?;
let next_url = current_url
.join(location)
.map_err(|_| anyhow!("bundle redirect Location is invalid"))?;
validate_bundle_redirect(&initial_scheme, &next_url)?;
current_url = next_url;
redirects += 1;
};
if !response.status().is_success() {
bail!(
"bundle fetch failed with HTTP status {}",
response.status().as_u16()
);
let mut policy = network.clone();
if is_loopback_bundle_host(&initial_host) {
policy.add_allow(&initial_host);
}
let check =
move |next: &reqwest::Url| validate_bundle_redirect(&initial_scheme, &initial_host, next);
let request = codewhale_net::guard::GuardedGet::new(
url,
&policy,
"config_import",
codewhale_net::guard::PrivateTargets::TypedLocalHosts,
)
.timeouts(
std::time::Duration::from_secs(FETCH_TIMEOUT_SECS),
std::time::Duration::from_secs(FETCH_TIMEOUT_SECS),
)
.max_redirects(MAX_REDIRECTS)
.redirect_check(&check);
// Read at most MAX_BUNDLE_BYTES + 1 so an oversize body is detected
// rather than silently truncated.
let mut buffer = Vec::new();
let body = response;
body.take(MAX_BUNDLE_BYTES + 1)
.read_to_end(&mut buffer)
.map_err(|_| anyhow!("reading remote bundle failed"))?;
if buffer.len() as u64 > MAX_BUNDLE_BYTES {
bail!("remote bundle exceeds the {MAX_BUNDLE_BYTES} byte limit; refused");
}
Ok(buffer)
run_blocking(async move {
let mut response = request.send().await?;
if !response.status().is_success() {
bail!(
"bundle fetch failed with HTTP status {}",
response.status().as_u16()
);
}
if response
.content_length()
.is_some_and(|length| length > MAX_BUNDLE_BYTES)
{
bail!("remote bundle exceeds the {MAX_BUNDLE_BYTES} byte limit; refused");
}
let mut buffer = Vec::new();
while let Some(chunk) = response
.chunk()
.await
.map_err(|_| anyhow!("reading remote bundle failed"))?
{
if (buffer.len() + chunk.len()) as u64 > MAX_BUNDLE_BYTES {
bail!("remote bundle exceeds the {MAX_BUNDLE_BYTES} byte limit; refused");
}
buffer.extend_from_slice(&chunk);
}
Ok(buffer)
})
}
fn run_blocking<T: Send>(future: impl std::future::Future<Output = Result<T>> + Send) -> Result<T> {
std::thread::scope(|scope| {
scope
.spawn(|| {
tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.map_err(|_| anyhow!("building bundle fetch runtime failed"))?
.block_on(future)
})
.join()
.map_err(|_| anyhow!("bundle fetch thread failed"))?
})
}
fn validate_bundle_url(url: &reqwest::Url) -> Result<()> {
@@ -877,19 +914,37 @@ fn validate_bundle_url(url: &reqwest::Url) -> Result<()> {
}
}
fn validate_bundle_redirect(initial_scheme: &str, next_url: &reqwest::Url) -> Result<()> {
fn validate_bundle_redirect(
initial_scheme: &str,
initial_host: &str,
next_url: &reqwest::Url,
) -> Result<()> {
validate_bundle_url(next_url)?;
if next_url.scheme() != initial_scheme {
bail!("bundle redirects may not change URL scheme");
}
let next_host = next_url
.host_str()
.context("bundle URL must include a host")?;
if codewhale_net::guard::is_visibly_local_host(next_host)
&& !same_bundle_host(initial_host, next_host)
{
bail!("bundle redirects may not target a different local host");
}
Ok(())
}
fn same_bundle_host(left: &str, right: &str) -> bool {
let normalize = |host: &str| host.trim().trim_end_matches('.').to_ascii_lowercase();
normalize(left) == normalize(right)
}
fn is_loopback_bundle_host(host: &str) -> bool {
let normalized = host
.strip_prefix('[')
.and_then(|value| value.strip_suffix(']'))
.unwrap_or(host);
.unwrap_or(host)
.trim_end_matches('.');
normalized.eq_ignore_ascii_case("localhost")
|| normalized.to_ascii_lowercase().ends_with(".localhost")
|| normalized
@@ -1571,7 +1626,13 @@ pub fn run_import(
}
buffer
} else if remote_source {
fetch_bundle(&args.source)?
let network = store
.config
.network
.clone()
.map(codewhale_net::network_policy::NetworkPolicy::from)
.unwrap_or_default();
fetch_bundle(&args.source, &network)?
} else {
let path = PathBuf::from(&args.source);
let metadata = std::fs::metadata(&path)
@@ -1672,6 +1733,7 @@ pub fn run_export(args: &ExportArgs, store: &codewhale_config::ConfigStore) -> R
mod tests {
use super::*;
use codewhale_config::ConfigStore;
use codewhale_net::network_policy::NetworkPolicy;
use std::io::Write;
use std::net::{Ipv4Addr, TcpListener};
@@ -4183,7 +4245,7 @@ label = "safe-nested-workspace-label"
#[test]
fn http_non_loopback_fetch_is_refused_without_network_access() {
let err = fetch_bundle("http://example.com/bundle.toml")
let err = fetch_bundle("http://example.com/bundle.toml", &NetworkPolicy::default())
.expect_err("plain http to a public host must be refused");
assert!(err.to_string().contains("loopback"), "{err:#}");
assert!(!err.to_string().contains("example.com"), "{err:#}");
@@ -4196,7 +4258,8 @@ label = "safe-nested-workspace-label"
let response = http_response("302 Found", &[("Location", location.as_str())], b"");
let (url, server) = spawn_bundle_http_server(vec![response]);
let error = fetch_bundle(&url).expect_err("redirect target must be revalidated");
let error = fetch_bundle(&url, &NetworkPolicy::default())
.expect_err("redirect target must be revalidated");
let rendered = format!("{error:#}");
assert!(rendered.contains("loopback"), "{rendered}");
assert!(!rendered.contains("example.com"), "{rendered}");
@@ -4204,13 +4267,149 @@ label = "safe-nested-workspace-label"
assert_eq!(server.join().expect("server joins"), 1);
}
#[test]
fn visibly_local_network_allow_entries_are_rejected_while_public_hosts_import() {
let local_bundle = parse_bundle_str(
r#"
schema_version = 1
kind = "codewhale.portable-config"
[global.network]
default = "prompt"
allow = ["docs.example.com", "localhost", "10.0.0.5"]
"#,
"local-allow.toml",
)
.expect("local allow bundle parses");
let rejected = find_rejected_entries(&local_bundle);
assert_eq!(rejected.len(), 1, "{rejected:?}");
assert_eq!(rejected[0].key, "global.network");
assert!(rejected[0].reason.contains("network.allow"), "{rejected:?}");
assert!(!rejected[0].reason.contains("localhost"), "{rejected:?}");
assert!(!rejected[0].reason.contains("10.0.0.5"), "{rejected:?}");
let mut store = isolated_store();
let before = std::fs::read(store.path()).expect("config before import");
let dir = store.path().parent().expect("config dir").to_path_buf();
apply_bundle(&local_bundle, &mut store, BundleScope::Global, &dir)
.expect_err("bundle adding visibly local network.allow entries must fail");
assert_eq!(
std::fs::read(store.path()).expect("config after rejected import"),
before
);
let public_bundle = parse_bundle_str(
r#"
schema_version = 1
kind = "codewhale.portable-config"
[global.network]
default = "prompt"
allow = ["docs.example.com"]
"#,
"public-allow.toml",
)
.expect("public allow bundle parses");
assert!(find_rejected_entries(&public_bundle).is_empty());
let receipt = apply_bundle(&public_bundle, &mut store, BundleScope::Global, &dir)
.expect("public network.allow entry must import");
assert!(receipt.plan.rejected.is_empty());
assert_eq!(
store
.config
.network
.as_ref()
.map(|network| network.allow.as_slice()),
Some(&["docs.example.com".to_string()][..])
);
}
#[test]
fn localdomain_and_localhost_wildcard_network_allow_entries_are_rejected_and_scrubbed() {
for (section, entry) in [
("global", ".localdomain"),
("global", "*.localdomain"),
("global", ".localhost"),
("global", "*.localhost"),
("global", "*.dev.localhost"),
("project", "*.localdomain"),
] {
let bundle_text = format!(
r#"
schema_version = 1
kind = "codewhale.portable-config"
[{section}.network]
default = "prompt"
allow = ["docs.example.com", "{entry}"]
"#
);
let bundle = parse_bundle_str(&bundle_text, "wildcard-local-allow.toml")
.expect("wildcard allow bundle parses");
let rejected = find_rejected_entries(&bundle);
assert_eq!(rejected.len(), 1, "{section} {entry}: {rejected:?}");
assert_eq!(rejected[0].key, format!("{section}.network"));
assert!(rejected[0].reason.contains("network.allow"), "{rejected:?}");
assert!(!rejected[0].reason.contains(entry), "{rejected:?}");
}
let combined = parse_bundle_str(
r#"
schema_version = 1
kind = "codewhale.portable-config"
[global.network]
default = "prompt"
allow = [".localdomain", "*.localdomain", "*.localhost"]
"#,
"combined-wildcard-local-allow.toml",
)
.expect("combined wildcard bundle parses");
let mut store = isolated_store();
let before = std::fs::read(store.path()).expect("config before import");
let dir = store.path().parent().expect("config dir").to_path_buf();
apply_bundle(&combined, &mut store, BundleScope::Global, &dir)
.expect_err("bundle adding local-domain wildcard network.allow entries must fail");
assert_eq!(
std::fs::read(store.path()).expect("config after rejected import"),
before
);
let config: ConfigToml = toml::from_str(
r#"
[network]
default = "prompt"
allow = ["docs.example.com", ".localdomain", "*.localdomain", ".localhost", "*.localhost", "localhost", "10.0.0.5"]
"#,
)
.expect("config with local network.allow entries parses");
let exported = export_bundle(&config, BundleScope::Global, BundleMetadata::default())
.expect("local network.allow entries are scrubbed on export");
let body = serialize_bundle(&exported).expect("serialize scrubbed export");
for forbidden in [
".localdomain",
"*.localdomain",
".localhost",
"*.localhost",
"\"localhost\"",
"10.0.0.5",
] {
assert!(
!body.contains(forbidden),
"export retained {forbidden}: {body}"
);
}
assert!(body.contains("docs.example.com"), "{body}");
assert!(find_rejected_entries(&exported).is_empty(), "{exported:?}");
}
#[test]
fn https_redirect_cannot_downgrade_to_loopback_http() {
let secret = "downgrade-secret-must-not-leak";
let target = reqwest::Url::parse(&format!("http://127.0.0.1/internal?token={secret}"))
.expect("test target URL");
let error = validate_bundle_redirect("https", &target)
let error = validate_bundle_redirect("https", "example.com", &target)
.expect_err("HTTPS redirect must not downgrade to loopback HTTP");
let rendered = format!("{error:#}");
assert!(rendered.contains("scheme"), "{rendered}");
@@ -4218,6 +4417,59 @@ label = "safe-nested-workspace-label"
assert!(!rendered.contains("127.0.0.1"), "{rendered}");
}
fn spawn_counting_server(
response: Vec<u8>,
) -> (u16, std::sync::Arc<std::sync::atomic::AtomicUsize>) {
use std::sync::atomic::{AtomicUsize, Ordering};
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).expect("bind counting fixture");
let port = listener.local_addr().expect("fixture address").port();
let hits = std::sync::Arc::new(AtomicUsize::new(0));
let counter = std::sync::Arc::clone(&hits);
std::thread::spawn(move || {
for stream in listener.incoming() {
let Ok(mut stream) = stream else { return };
counter.fetch_add(1, Ordering::SeqCst);
let mut chunk = [0_u8; 1024];
let _ = stream.read(&mut chunk);
let _ = stream.write_all(&response);
}
});
(port, hits)
}
#[test]
fn redirect_to_another_local_host_is_refused() {
use std::sync::atomic::Ordering;
let (port, hits) = spawn_counting_server(http_response(
"200 OK",
&[("Content-Type", "text/plain")],
VALID_TOML.as_bytes(),
));
let location = format!("http://localhost:{port}/bundle.toml");
let redirect = http_response("302 Found", &[("Location", location.as_str())], b"");
let (url, entry_server) = spawn_bundle_http_server(vec![redirect.clone()]);
let error = fetch_bundle(&url, &NetworkPolicy::default())
.expect_err("a hop to another local host must be refused");
assert!(!format!("{error:#}").contains("bundle.toml"), "{error:#}");
assert_eq!(entry_server.join().expect("entry joins"), 1);
assert_eq!(hits.load(Ordering::SeqCst), 0);
let (listed_url, listed_entry_server) = spawn_bundle_http_server(vec![redirect]);
let listed_policy = NetworkPolicy {
allow: vec!["localhost".to_string()],
..NetworkPolicy::default()
};
let listed_error = fetch_bundle(&listed_url, &listed_policy)
.expect_err("even a listed local host must be refused on redirect from another host");
assert!(
!format!("{listed_error:#}").contains("bundle.toml"),
"{listed_error:#}"
);
assert_eq!(listed_entry_server.join().expect("listed entry joins"), 1);
assert_eq!(hits.load(Ordering::SeqCst), 0);
}
#[test]
fn relative_loopback_redirect_fetches_bundle() {
let redirect = http_response("302 Found", &[("Location", "/bundle.toml")], b"");
@@ -4225,7 +4477,8 @@ label = "safe-nested-workspace-label"
let success = http_response("200 OK", &[("Content-Type", "text/plain")], body);
let (url, server) = spawn_bundle_http_server(vec![redirect, success]);
let fetched = fetch_bundle(&url).expect("relative redirect remains allowed");
let fetched = fetch_bundle(&url, &NetworkPolicy::default())
.expect("relative redirect remains allowed");
assert_eq!(fetched, body);
assert_eq!(server.join().expect("server joins"), 2);
}
@@ -4240,16 +4493,23 @@ label = "safe-nested-workspace-label"
.collect();
let (url, server) = spawn_bundle_http_server(responses);
let error = fetch_bundle(&url).expect_err("sixth redirect must be refused");
assert!(error.to_string().contains("five-redirect"), "{error:#}");
let error = fetch_bundle(&url, &NetworkPolicy::default())
.expect_err("sixth redirect must be refused");
assert!(
error.to_string().contains("too many redirects"),
"{error:#}"
);
assert_eq!(server.join().expect("server joins"), MAX_REDIRECTS + 1);
}
#[test]
fn bundle_url_credentials_are_rejected_without_echoing_them() {
let secret = "credential-secret-must-not-leak";
let error = fetch_bundle(&format!("https://user:{secret}@example.com/bundle.toml"))
.expect_err("URL userinfo must be refused");
let error = fetch_bundle(
&format!("https://user:{secret}@example.com/bundle.toml"),
&NetworkPolicy::default(),
)
.expect_err("URL userinfo must be refused");
let rendered = format!("{error:#}");
assert!(rendered.contains("credentials"), "{rendered}");
assert!(!rendered.contains(secret), "{rendered}");
@@ -4258,7 +4518,8 @@ label = "safe-nested-workspace-label"
#[test]
fn unsupported_schemes_are_refused() {
let err = fetch_bundle("file:///etc/passwd").expect_err("file scheme refused");
let err = fetch_bundle("file:///etc/passwd", &NetworkPolicy::default())
.expect_err("file scheme refused");
assert!(err.to_string().contains("scheme"), "{err:#}");
}
+1
View File
@@ -25,6 +25,7 @@ semver.workspace = true
schemars = { version = "1.2.1", optional = true, features = ["derive", "preserve_order"] }
codewhale-execpolicy = { path = "../execpolicy", version = "0.10.3" }
codewhale-paths = { path = "../paths", version = "0.10.3" }
codewhale-sanitize = { path = "../sanitize", version = "0.10.3" }
codewhale-secrets = { path = "../secrets", version = "0.10.3" }
fd-lock = "4.0.4"
libc = "0.2"
+2 -2
View File
@@ -300,7 +300,7 @@ pub(crate) fn remove_plaintext_api_keys_in_array(array: &mut toml_edit::Array) {
pub(crate) fn backup_value_carries_secret(value: &toml_edit::Value) -> bool {
value
.as_str()
.is_some_and(codewhale_secrets::sanitize::contains_secret)
.is_some_and(codewhale_sanitize::sanitize::contains_secret)
}
pub(crate) fn scrub_backup_decor(decor: &mut toml_edit::Decor) {
@@ -331,7 +331,7 @@ pub(crate) fn scrub_backup_comments(text: &str) -> String {
comment
.split_once('=')
.is_some_and(|(key, _)| is_sensitive_config_key(key))
|| codewhale_secrets::sanitize::contains_secret(comment)
|| codewhale_sanitize::sanitize::contains_secret(comment)
})
})
.collect()
+1 -1
View File
@@ -99,7 +99,7 @@ pub fn toml_error_location(source: Option<&str>, err: &toml::de::Error) -> Strin
.and_then(|rest| rest.strip_prefix("\nin `"))
.and_then(|rest| rest.strip_suffix("`\n"))
.filter(|keys| !keys.is_empty())
.map(|keys| format!("in `{}`", codewhale_secrets::redact::redact_secrets(keys)));
.map(|keys| format!("in `{}`", codewhale_sanitize::redact::redact_secrets(keys)));
match (position, key_path) {
(Some(position), Some(keys)) => format!("{position}, {keys}"),
(Some(position), None) => position,
+1 -1
View File
@@ -128,7 +128,7 @@ pub(crate) fn redact_secret(secret: &str) -> String {
pub fn is_sensitive_config_key(key: &str) -> bool {
key.rsplit('.')
.next()
.is_some_and(codewhale_secrets::sanitize::is_sensitive_key_name)
.is_some_and(codewhale_sanitize::sanitize::is_sensitive_key_name)
}
/// Resolve dotted paths without treating a dotted key as a top-level literal.
-7
View File
@@ -5,13 +5,6 @@ use std::path::PathBuf;
/// On-disk schema for the `[hook_sinks]` table.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct HookSinksToml {
/// Unix domain socket path used by the app-server event sink.
///
/// Known limitation: only `codewhale app-server` reads this table; the
/// TUI and the Runtime API do not dispatch hook events to it.
///
/// When unset, no Unix socket sink is registered. There is deliberately no
/// shared `/tmp` default because socket ownership should be explicit.
#[serde(default)]
pub unix_socket_path: Option<PathBuf>,
}
+2 -2
View File
@@ -314,10 +314,10 @@ fn rollback(snapshots: &[Snapshot]) {
}
// FEAT-025 D4: the pure secret-redaction primitives moved to
// `codewhale-secrets::redact` so portable command helpers can share one
// `codewhale-sanitize::redact` so portable command helpers can share one
// implementation without depending on this crate. Re-exported here to keep
// the existing `codewhale_config::persistence::*` public API stable.
pub use codewhale_secrets::redact::{
pub use codewhale_sanitize::redact::{
REDACTED, RedactionPolicy, redact_json_model_bound_secrets, redact_json_secrets,
redact_model_bound_secrets, redact_secrets, redact_secrets_with,
};
+2 -2
View File
@@ -59,7 +59,7 @@ pub fn check_config_toml_choice(key: &str, value: &str) -> Result<()> {
} else {
""
};
let value = codewhale_secrets::redact::redact_secrets(value);
let value = codewhale_sanitize::redact::redact_secrets(value);
bail!(
"invalid value '{value}' for '{key}': config.toml accepts {}.{settings_note} \
No value was changed.\nfix: codewhale config set {key} {}",
@@ -205,7 +205,7 @@ pub fn load_project_config_outcome(workspace: &Path) -> ProjectConfigOutcome {
// echoed; an ordinary typo stays readable.
Some(name) => format!(
"unknown provider '{}'",
codewhale_secrets::redact::redact_secrets(&name)
codewhale_sanitize::redact::redact_secrets(&name)
),
None => "unknown provider".to_string(),
},
+7 -1
View File
@@ -605,7 +605,7 @@ pub const SETTINGS_SCHEMA: &[SettingDef] = &[
def(
"show_thinking",
SettingKind::Bool(SHOW_THINKING),
"false",
"true",
ui(
TAB_APPEARANCE,
"display",
@@ -1546,6 +1546,12 @@ pub const SETTINGS_SCHEMA: &[SettingDef] = &[
"false",
None,
),
def(
"account_preferences_notice_shown",
SettingKind::String,
"",
None,
),
// Round 3 work-bar placement migration ran once (`top` → `bottom`).
def(
"work_surface_bottom_migrated",
-2
View File
@@ -14,7 +14,6 @@ workspace = true
serde.workspace = true
thiserror.workspace = true
codewhale-config = { path = "../config", version = "0.10.3" }
codewhale-hooks = { path = "../hooks", version = "0.10.3" }
codewhale-protocol = { path = "../protocol", version = "0.10.3" }
codewhale-state = { path = "../state", version = "0.10.3" }
regex = "1.11"
@@ -24,4 +23,3 @@ serde_json = { workspace = true, features = ["preserve_order"] }
[dev-dependencies]
tempfile.workspace = true
uuid.workspace = true
tokio = { workspace = true, features = ["macros", "rt", "time"] }
-9
View File
@@ -1,9 +0,0 @@
//! `ThreadId` / `SessionId` for the `crates/core` boundary (issue #5261).
//!
//! Re-exports the protocol ids so every crate that depends on `core` (the
//! TUI, CLI, app-server) speaks the same typed ids without depending on
//! `protocol` directly. The persisted JSON shape stays a plain string
//! (`"thread-…"` / `"session-…"`) so existing `state.json` / `threads/`
//! files need no migration.
pub use codewhale_protocol::ids::{SessionId, ThreadId};
-78
View File
@@ -1,78 +0,0 @@
//! Session tree journal placeholder (issue #5262).
//!
//! The journal is append-only with an in-memory tree projection:
//! every non-header entry carries `id` + `parentId`, the active position is
//! a `leafId`, appending creates a child of the leaf, and branching only
//! moves the leaf — it never rewrites history. This file lands the entry
//! shape that #5262's tree operations hang off of; compaction and
//! branch-summary entry kinds are included as first-class kinds but their
//! *strategies* are deferred.
//!
//! Re-exports the protocol journal as the canonical shape so `protocol` and
//! `core` agree on the wire. `core` adds the `SessionJournal` wrapper that
//! owns the `current_leaf_id` column in `state.threads`.
pub use codewhale_protocol::journal::{Journal, JournalEntry};
use serde::{Deserialize, Serialize};
/// Persisted thread metadata extension for the tree. This is the
/// `current_leaf_id` column added to `state.threads`; `None` before the
/// first turn, `Some(id)` after. The existing `threads` JSON shape is
/// otherwise unchanged (back-compat: old rows read as `None` and the next
/// append mints the header leaf).
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct ThreadLeafState {
pub thread_id: String,
pub leaf_id: Option<String>,
}
/// First-class journal entry kinds (data shape lands now; strategies later).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum JournalKind {
Header,
User,
Assistant,
ToolResult,
Compaction,
BranchSummary,
}
impl JournalKind {
#[must_use]
pub fn as_str(self) -> &'static str {
match self {
Self::Header => "header",
Self::User => "user",
Self::Assistant => "assistant",
Self::ToolResult => "tool_result",
Self::Compaction => "compaction",
Self::BranchSummary => "branch_summary",
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn leaf_state_roundtrip() {
let s = ThreadLeafState {
thread_id: "thread-1".into(),
leaf_id: Some("entry-abc".into()),
};
let j = serde_json::to_string(&s).unwrap();
let back: ThreadLeafState = serde_json::from_str(&j).unwrap();
assert_eq!(back, s);
}
#[test]
fn journal_append_is_child_of_leaf() {
let mut j = Journal::new();
let a = j.append("header", json!({}));
let b = j.append("user", json!("hi"));
assert_eq!(j.get(&b).unwrap().parent_id.as_deref(), Some(a.as_str()));
}
}
+5 -18
View File
@@ -1,12 +1,9 @@
pub mod context_reference;
pub mod fragments;
pub mod ids;
pub mod journal;
pub mod prefix_cache;
pub mod request;
pub mod role;
pub mod secret_eq;
pub mod session;
pub mod tool_parser;
pub use context_reference::{
@@ -15,10 +12,9 @@ pub use context_reference::{
};
use codewhale_config::{ConfigToml, ProviderKind};
use codewhale_hooks::HookDispatcher;
use codewhale_state::StateStore;
/// Compatibility configuration and hooks.
/// Compatibility configuration.
///
/// Conversation execution and durable history belong to the held canonical
/// RuntimeThreadManager/Engine. Legacy State data is retained for read-only
@@ -26,16 +22,11 @@ use codewhale_state::StateStore;
pub struct Runtime {
pub config: ConfigToml,
state: StateStore,
pub hooks: HookDispatcher,
}
impl Runtime {
pub fn new(config: ConfigToml, state: StateStore, hooks: HookDispatcher) -> Self {
Self {
config,
state,
hooks,
}
pub fn new(config: ConfigToml, state: StateStore) -> Self {
Self { config, state }
}
/// Existing compatibility data and receipts; no transcript controller.
@@ -103,7 +94,7 @@ mod tests {
fn runtime_bookkeeping_keeps_the_legacy_store_read_only() {
let store = temp_core_state("read-only-runtime");
let before = store.list_threads(Default::default()).unwrap();
let mut runtime = Runtime::new(ConfigToml::default(), store, HookDispatcher::default());
let mut runtime = Runtime::new(ConfigToml::default(), store);
runtime.update_config(ConfigToml::default());
assert_eq!(
runtime
@@ -113,11 +104,7 @@ mod tests {
.len(),
before.len()
);
let reopened = Runtime::new(
runtime.config.clone(),
runtime.state_store().clone(),
HookDispatcher::default(),
);
let reopened = Runtime::new(runtime.config.clone(), runtime.state_store().clone());
assert!(
reopened
.state_store()
-135
View File
@@ -1,135 +0,0 @@
//! `Thread` / `Session` split (issue #5261).
//!
//! `codewhale`'s `Session` was really a thread. The new split is:
//! - `Thread` — durable, persisted, owns the append-only `Journal` and the
//! `leafId` cursor. One row in `state.threads`, one directory on disk.
//! - `Session` — ephemeral, per-turn / per-engine-lifetime, owns the
//! in-memory `TurnContext` plus the live approval/sandbox posture for this
//! `SessionId`. Many sessions can attach to one thread over time, but only
//! one `Session` drives a turn for a given `ThreadId` at a time.
//!
//! The canonical RuntimeThreadManager creates sessions without a TUI and drives
//! the same Engine turn loop used by mounted clients. These types preserve the
//! shared model, append log, prefix stability and message revision state; the
//! compatibility Core Runtime owns only configuration, hooks and job bookkeeping.
use std::path::PathBuf;
use serde::{Deserialize, Serialize};
use crate::ids::{SessionId, ThreadId};
use crate::journal::Journal;
/// Durable thread (the former `Session`). One per conversation, persisted in
/// `state.threads`. The only new field vs the old `Session` is `leaf_id` — the
/// journal cursor — plus the typed `ThreadId`. All other fields keep their
/// persisted JSON shape unchanged.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Thread {
pub thread_id: ThreadId,
/// Active branch tip. `None` before the first journal header.
#[serde(skip_serializing_if = "Option::is_none")]
pub leaf_id: Option<String>,
/// Journal (append-only). In-memory projection of the persisted
/// `threads/turns/items/events` layout is derived root→leaf.
#[serde(default)]
pub journal: Journal,
pub model: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_effort: Option<String>,
pub workspace: PathBuf,
#[serde(default)]
pub ephemeral: bool,
}
impl Thread {
#[must_use]
pub fn new(thread_id: ThreadId, workspace: PathBuf, model: impl Into<String>) -> Self {
Self {
thread_id,
leaf_id: None,
journal: Journal::new(),
model: model.into(),
reasoning_effort: None,
workspace,
ephemeral: false,
}
}
#[must_use]
pub fn leaf_id(&self) -> Option<&str> {
self.leaf_id.as_deref()
}
pub fn set_leaf(&mut self, leaf: Option<String>) {
self.leaf_id = leaf;
}
}
/// Ephemeral session within a thread (one engine lifetime / one turn's
/// live posture). The TUI's `EngineHandle` and the headless `exec` both
/// hold a `Session` that points at the same `ThreadId` but with different
/// `SessionId`s.
#[derive(Debug, Clone)]
pub struct Session {
pub session_id: SessionId,
pub thread_id: ThreadId,
/// Model for this session's next turn (may differ from thread default).
pub model: String,
pub workspace: PathBuf,
/// Monotonic `messages_revision` for prefix-cache memoization (carried
/// from the former `Session::messages_revision`).
pub messages_revision: u64,
}
impl Session {
#[must_use]
pub fn new(thread_id: ThreadId, workspace: PathBuf, model: impl Into<String>) -> Self {
Self {
session_id: SessionId::new(),
thread_id,
model: model.into(),
workspace,
messages_revision: 0,
}
}
pub fn bump_revision(&mut self) {
self.messages_revision = self.messages_revision.wrapping_add(1);
}
}
/// Split helper: derive a `Session` from an existing `Thread` without
/// cloning the journal. Headless and TUI call the same constructor so
/// the request shape stays identical.
#[must_use]
pub fn session_for_thread(thread: &Thread, workspace: PathBuf) -> Session {
Session::new(thread.thread_id.clone(), workspace, thread.model.clone())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn thread_and_session_ids_are_distinct_scopes() {
let t = Thread::new(ThreadId::new(), PathBuf::from("/tmp"), "deepseek-v4-flash");
let s1 = Session::new(t.thread_id.clone(), PathBuf::from("/tmp"), &t.model);
let s2 = Session::new(t.thread_id.clone(), PathBuf::from("/tmp"), &t.model);
assert_eq!(s1.thread_id, s2.thread_id);
assert_ne!(s1.session_id, s2.session_id);
}
#[test]
fn leaf_is_moved_not_rewritten() {
let mut t = Thread::new(ThreadId::new(), PathBuf::from("/tmp"), "m");
let a = t.journal.append("header", serde_json::json!({}));
let b = t.journal.append("user", serde_json::json!("b"));
t.leaf_id = t.journal.leaf_id.clone();
assert_eq!(t.leaf_id.as_deref(), Some(b.as_str()));
assert!(t.journal.branch_to(&a));
t.leaf_id = t.journal.leaf_id.clone();
assert_eq!(t.leaf_id.as_deref(), Some(a.as_str()));
assert_eq!(t.journal.len(), 2); // history never rewritten; branching only moved the leaf
}
}
+1 -6
View File
@@ -79,7 +79,6 @@ codewhale-app-server-protocol = { path = "../app-server-protocol", version = "0.
codewhale-sanitize = { path = "../sanitize", version = "0.10.3" }
codewhale-memory = { path = "../memory", version = "0.10.3" }
anyhow.workspace = true
codewhale-agent = { path = "../agent", version = "0.10.3" }
codewhale-agent-contract = { path = "../agent-contract", version = "0.10.3" }
codewhale-config = { path = "../config", version = "0.10.3" }
codewhale-command-contract = { path = "../command-contract", version = "0.10.3" }
@@ -172,6 +171,7 @@ sha2.workspace = true
# Reuse the locked cryptographic implementation for exact-call user decisions.
ring = "0.17"
codewhale-localization = { path = "../localization", version = "0.10.3" }
codewhale-ratatui = { path = "../../vendor/codewhale-ratatui", version = "0.2.0" }
shell-words = "1.1.1"
wiremock = { version = "0.6", optional = true }
@@ -203,14 +203,9 @@ codewhale-util = { path = "../util", version = "0.10.3", features = ["test-suppo
codewhale-models = { path = "../models", version = "0.10.3", features = [
"test-support",
] }
cucumber = "0.23.0"
wiremock = "0.6"
tiny_http = "0.12"
pretty_assertions = "1.4"
rio-vt = "0.5.1"
rusqlite.workspace = true
schemars = { version = "1.2.1", features = ["derive", "preserve_order"] }
tokio-tungstenite = "0.30"
[target.'cfg(unix)'.dependencies]
libc = "0.2"
+1 -1
View File
@@ -18,7 +18,7 @@ var ErrorCode = {
Denied: -32003,
Cancelled: -32800
};
var DSH_SERVICES = ["workspace", "llm", "network", "settings", "catalog", "terminal", "remote"];
var DSH_SERVICES = ["workspace", "llm", "network", "settings", "catalog", "terminal", "remote", "plugins"];
var METHODS = [
{ name: "host/initialize", direction: "core_to_host", request: true, params: "InitializeParams", tiers: ["plugin", "builtin"] },
{ name: "host/ping", direction: "core_to_host", request: true, params: "EmptyParams", tiers: ["plugin", "builtin"] },
@@ -0,0 +1,193 @@
import { Service } from '@deepseek-ai/cordis'
import { ErrorCode } from '../protocol.ts'
import { RpcError } from '../rpc.ts'
import type { AdapterHost, DshAdapter } from './adapters.ts'
export interface ApprovedRoute {
provider: string
base_url: string
default_model: string | null
models: string[]
}
export interface ResolvedRoute extends ApprovedRoute {
model: string | null
}
export interface InstallPlan {
status: 'review_required' | 'already_present'
package: string
version: string
plugin: string
spec?: string
command: string
message: string
}
export type BalanceAnswer =
| { status: 'known', currency: string, total: string, topped_up: string, granted: string }
| { status: 'unknown' }
| { status: 'absent' }
interface AccountWallet {
currency: string
balance: string
}
type AccountBalance = { status: 'ready', value: AccountWallet[], bonusWallets: AccountWallet[] } | { status: 'failed' }
interface ChangeResult {
changed: false
application: 'failed'
stage: 'install' | 'remove'
target: string
error: { code: string, diagnostic: string, message: string }
}
const EXACT_VERSION_ONLY = 'Codewhale plans a plugin install only for one exact npm version, written [email protected]'
const REMOVE_IN_CODEWHALE = 'Codewhale removes a plugin only from its own plugin manager, after you disable it: /plugin uninstall'
const WITHHELD_ACCOUNT_METHODS = [
'getState', 'getProfile', 'getUnnotifiedBonuses', 'ackBonusNotified', 'startSignIn', 'cancelSignIn', 'signOut', 'watch',
'resolveToken', 'rejectToken', 'getPlatformSession', 'getDeviceIdentity',
] as const
function invalid(what: string): RpcError {
return new RpcError(ErrorCode.Internal, `invalid catalog broker ${what}`)
}
function record(value: unknown, what: string): Record<string, unknown> {
if (value === null || typeof value !== 'object' || Array.isArray(value)) throw invalid(what)
return value as Record<string, unknown>
}
function text(value: unknown, what: string): string {
if (typeof value !== 'string' || value.length === 0) throw invalid(what)
return value
}
function named(value: unknown, what: string): string {
if (typeof value !== 'string' || value.length === 0) throw new RpcError(ErrorCode.InvalidParams, `${what} is a non-empty string`)
return value
}
function approvedRoute(value: unknown): ApprovedRoute {
const row = record(value, 'route')
const models = row.models
const defaultModel = row.default_model
if (!Array.isArray(models) || models.some((model) => typeof model !== 'string') || (defaultModel !== null && typeof defaultModel !== 'string')) throw invalid('route')
return { provider: text(row.provider, 'route'), base_url: text(row.base_url, 'route'), default_model: defaultModel, models: [...models] }
}
export async function approvedRoutes(host: AdapterHost, ctx: unknown): Promise<ApprovedRoute[]> {
const routes = record(await host.broker(ctx, 'catalog', 'routes.list', {}), 'routes').routes
if (!Array.isArray(routes)) throw invalid('routes')
return routes.map(approvedRoute)
}
export async function resolveRoute(host: AdapterHost, ctx: unknown, provider: unknown, model?: unknown): Promise<ResolvedRoute> {
const input = model === undefined || model === null
? { provider: named(provider, 'a provider route') }
: { provider: named(provider, 'a provider route'), model: named(model, 'a model') }
const answer = record(await host.broker(ctx, 'catalog', 'routes.resolve', input), 'route')
const resolved = answer.model
if (resolved !== null && typeof resolved !== 'string') throw invalid('route')
return { ...approvedRoute(answer.route), model: resolved }
}
export async function accountBalance(host: AdapterHost, ctx: unknown): Promise<BalanceAnswer> {
const answer = record(await host.broker(ctx, 'catalog', 'balance', {}), 'balance')
const status = answer.status
if (status === 'unknown' || status === 'absent') return { status }
const toppedUp = answer.topped_up
const granted = answer.granted
if (status !== 'known' || typeof toppedUp !== 'string' || typeof granted !== 'string') throw invalid('balance')
return { status, currency: text(answer.currency, 'balance'), total: text(answer.total, 'balance'), topped_up: toppedUp, granted }
}
export async function planInstall(host: AdapterHost, ctx: unknown, spec: unknown, registry?: unknown): Promise<InstallPlan> {
const exact = typeof spec === 'string' ? spec.lastIndexOf('@') : -1
if (typeof spec !== 'string' || exact <= 0 || exact === spec.length - 1) throw new RpcError(ErrorCode.InvalidParams, EXACT_VERSION_ONLY)
const request = { package: spec.slice(0, exact), version: spec.slice(exact + 1) }
const input = registry === undefined || registry === null ? request : { ...request, registry: named(registry, 'a registry URL') }
const plan = record(await host.broker(ctx, 'catalog', 'install.plan', input), 'install plan')
const status = plan.status
const planned = plan.spec
if ((status !== 'review_required' && status !== 'already_present') || (planned !== undefined && typeof planned !== 'string')) throw invalid('install plan')
return {
status,
package: text(plan.package, 'install plan'),
version: text(plan.version, 'install plan'),
plugin: text(plan.plugin, 'install plan'),
...(planned === undefined ? {} : { spec: planned }),
command: text(plan.command, 'install plan'),
message: text(plan.message, 'install plan'),
}
}
function notChanged(stage: ChangeResult['stage'], target: unknown, code: string, diagnostic: string): ChangeResult {
return { changed: false, application: 'failed', stage, target: typeof target === 'string' ? target : '', error: { code, diagnostic, message: diagnostic } }
}
function declined(error: unknown): string | undefined {
if (!(error instanceof Error)) return undefined
if (error instanceof RpcError && error.code === ErrorCode.InvalidParams && error.data === undefined) return error.message
const refusal = (error as { data?: { dsh_refusal?: unknown } }).data?.dsh_refusal
return refusal === 'refused' || refusal === 'invalid_input' ? error.message : undefined
}
export const pluginManagerAdapter: DshAdapter = {
service: 'pluginManager',
define(host) {
class PluginManagerService extends Service {
constructor(ctx: any) { super(ctx, 'pluginManager') }
async installBundle(spec: unknown, options?: { registry?: unknown } | null): Promise<ChangeResult> {
try {
const plan = await planInstall(host, this.ctx, spec, options?.registry)
return notChanged('install', spec, plan.status === 'already_present' ? 'already-installed' : 'management-required', plan.message)
} catch (error) {
const diagnostic = declined(error)
if (diagnostic === undefined) throw error
return notChanged('install', spec, 'invalid-spec', diagnostic)
}
}
async removeBundle(target: unknown): Promise<ChangeResult> {
return notChanged('remove', target, 'management-required', REMOVE_IN_CODEWHALE)
}
async cancelInstall(): Promise<{ status: 'not-running' }> {
return { status: 'not-running' }
}
}
Object.freeze(PluginManagerService.prototype)
return PluginManagerService
},
}
export const deepseekAccountAdapter: DshAdapter = {
service: 'deepseekAccount',
define(host) {
class DeepSeekAccountService extends Service {
constructor(ctx: any) { super(ctx, 'deepseekAccount') }
async getBalance(): Promise<AccountBalance | null> {
const answer = await accountBalance(host, this.ctx)
if (answer.status === 'absent') return null
if (answer.status === 'unknown') return { status: 'failed' }
const { currency, total, topped_up: toppedUp, granted } = answer
const wallet = (balance: string): AccountWallet[] => balance.length > 0 ? [{ currency, balance }] : []
return { status: 'ready', value: wallet(toppedUp.length > 0 ? toppedUp : total), bonusWallets: wallet(granted) }
}
}
for (const method of WITHHELD_ACCOUNT_METHODS) {
Object.defineProperty(DeepSeekAccountService.prototype, method, {
value() {
throw new RpcError(ErrorCode.NotAvailable, `deepseekAccount.${method} is not provided: Codewhale answers a read-only balance and never gives a plugin an account credential or a sign-in`)
},
})
}
Object.freeze(DeepSeekAccountService.prototype)
return DeepSeekAccountService
},
}
@@ -0,0 +1,91 @@
import { Service } from '@deepseek-ai/cordis'
import { ErrorCode } from '../protocol.ts'
import { RpcError } from '../rpc.ts'
import type { AdapterHost, DshAdapter } from './adapters.ts'
export interface SettingsDescriptor {
ns: string
writable: boolean
roots: string[]
value: Record<string, unknown>
revision: string
applied: boolean | null
applies: 'reload'
}
export type SettingsPathOp =
| { op: 'set', path: readonly string[], value: unknown }
| { op: 'unset', path: readonly string[] }
export interface SettingsWrite {
written: boolean
revision: string
applies: 'reload'
}
export class SettingsConflictError extends Error {
readonly code = 'SETTINGS_CONFLICT'
constructor(readonly ns: string, readonly expected: string | undefined, readonly actual: string) {
super(`settings namespace "${ns}" changed since it was read`)
this.name = 'SettingsConflictError'
}
}
function plain(value: unknown, what: string): Record<string, unknown> {
if (typeof value !== 'object' || value === null || Array.isArray(value)) throw new RpcError(ErrorCode.InvalidParams, `${what} is a plain object`)
return value as Record<string, unknown>
}
function revision(value: unknown): string | undefined {
if (value === undefined) return undefined
if (typeof value !== 'string' || value.length === 0) throw new RpcError(ErrorCode.InvalidParams, 'an expected revision is the revision string describe returned')
return value
}
function descriptor(value: unknown): SettingsDescriptor {
const row = value as Partial<SettingsDescriptor> | null
if (row === null || typeof row !== 'object' || typeof row.ns !== 'string' || typeof row.revision !== 'string' || typeof row.writable !== 'boolean'
|| !Array.isArray(row.roots) || typeof row.value !== 'object' || row.value === null || row.applies !== 'reload') {
throw new RpcError(ErrorCode.Internal, 'invalid settings broker response')
}
return { ns: row.ns, writable: row.writable, roots: row.roots.map(String), value: row.value, revision: row.revision, applied: typeof row.applied === 'boolean' ? row.applied : null, applies: 'reload' }
}
export const settingsAdapter: DshAdapter = {
service: 'settings',
define(host: AdapterHost) {
class SettingsService extends Service {
constructor(ctx: any) { super(ctx, 'settings') }
async describe(): Promise<SettingsDescriptor[]> {
return [descriptor(await host.broker(this.ctx, 'settings', 'describe', {}))]
}
async update(ns: unknown, patch: unknown, expectedRevision?: unknown): Promise<SettingsWrite> {
return this.write(ns, 'update', { patch: plain(patch, 'a settings patch') }, expectedRevision)
}
async replace(ns: unknown, section: unknown, expectedRevision?: unknown): Promise<SettingsWrite> {
return this.write(ns, 'replace', { value: plain(section, 'a settings section') }, expectedRevision)
}
async mutate(ns: unknown, ops: unknown, expectedRevision?: unknown): Promise<SettingsWrite> {
if (!Array.isArray(ops)) throw new RpcError(ErrorCode.InvalidParams, 'settings edits are an array of path operations')
return this.write(ns, 'mutate', { ops }, expectedRevision)
}
private async write(ns: unknown, method: 'update' | 'replace' | 'mutate', input: Record<string, unknown>, expectedRevision: unknown): Promise<SettingsWrite> {
if (typeof ns !== 'string' || ns.length === 0) throw new RpcError(ErrorCode.InvalidParams, 'a settings namespace is a non-empty string')
const owner = host.ownerOf(this.ctx)
if (owner === undefined || ns !== owner.pluginName) throw new RpcError(ErrorCode.Refused, 'a plugin may edit only its own settings namespace')
const expected = revision(expectedRevision)
const result = await host.broker(this.ctx, 'settings', method, expected === undefined ? input : { ...input, expected_revision: expected }) as { written?: unknown, conflict?: unknown, revision?: unknown } | null
if (result === null || typeof result !== 'object' || typeof result.written !== 'boolean' || typeof result.revision !== 'string') throw new RpcError(ErrorCode.Internal, 'invalid settings broker response')
if (result.conflict === true) throw new SettingsConflictError(ns, expected, result.revision)
return { written: result.written, revision: result.revision, applies: 'reload' }
}
}
Object.freeze(SettingsService.prototype)
return SettingsService
},
}
@@ -25,7 +25,7 @@ export const ErrorCode = {
export type Direction = 'core_to_host' | 'host_to_core'
export const DSH_SERVICES = ['workspace', 'llm', 'network', 'settings', 'catalog', 'terminal', 'remote'] as const
export const DSH_SERVICES = ['workspace', 'llm', 'network', 'settings', 'catalog', 'terminal', 'remote', 'plugins'] as const
export type DshService = typeof DSH_SERVICES[number]
/** Every method either side may send, and the trust tiers it is allowed on; nothing else is admitted. */
@@ -0,0 +1,158 @@
import { test, before } from 'node:test'
import assert from 'node:assert/strict'
import { mkdtempSync, rmSync } from 'node:fs'
import { join, dirname } from 'node:path'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { build } from 'esbuild'
const here = dirname(fileURLToPath(import.meta.url))
let catalog
before(async () => {
const dir = mkdtempSync(join(here, '..', 'node_modules', '.cw-catalog-'))
const outfile = join(dir, 'catalog.mjs')
await build({ entryPoints: [join(here, '../src/dsh/catalog.ts')], outfile, bundle: true, platform: 'node', format: 'esm', packages: 'external', absWorkingDir: join(here, '..'), logLevel: 'silent' })
catalog = await import(pathToFileURL(outfile).href)
process.on('exit', () => rmSync(dir, { recursive: true, force: true }))
})
function brokerHost(answers) {
const calls = []
return {
calls,
host: {
tier: 'plugin',
broker: async (ctx, service, method, input) => {
calls.push({ service, method, input })
const answer = typeof answers[method] === 'function' ? answers[method](input) : answers[method]
if (answer instanceof Error) throw answer
return answer
},
},
}
}
function serviceOf(adapter, host) {
const service = Object.create(adapter.define(host).prototype)
service.ctx = {}
return service
}
const brokerRefusal = (code, refusal, message) => Object.assign(new Error(message), { code, data: { dsh_refusal: refusal, service: 'catalog' } })
test('a balance widget reads an absent account, an unknown balance and a known balance, and never a key', async () => {
const answers = { balance: { status: 'absent' } }
const { host, calls } = brokerHost(answers)
const account = serviceOf(catalog.deepseekAccountAdapter, host)
const client = { version: '0.4.2', locale: 'zh_CN', timezoneOffsetSeconds: 28800 }
assert.equal(await account.getBalance(client), null)
answers.balance = { status: 'unknown' }
assert.deepEqual(await account.getBalance(client), { status: 'failed' })
answers.balance = { status: 'known', currency: 'CNY', total: '123.45', topped_up: '100.00', granted: '23.45' }
const ready = await account.getBalance(client)
assert.deepEqual(ready, { status: 'ready', value: [{ currency: 'CNY', balance: '100.00' }], bonusWallets: [{ currency: 'CNY', balance: '23.45' }] })
const shown = [...ready.value, ...ready.bonusWallets].reduce((sum, wallet) => sum + Number(wallet.balance), 0)
assert.equal(shown.toFixed(2), '123.45')
answers.balance = { status: 'known', currency: 'USD', total: '12.50', topped_up: '', granted: '' }
assert.deepEqual(await account.getBalance(client), { status: 'ready', value: [{ currency: 'USD', balance: '12.50' }], bonusWallets: [] })
assert.deepEqual(calls.map((call) => [call.service, call.method, call.input]), Array(4).fill(['catalog', 'balance', {}]))
for (const method of ['resolveToken', 'getPlatformSession', 'getState', 'startSignIn', 'signOut', 'rejectToken']) {
assert.throws(() => account[method]('https://api.deepseek.com'), (error) => error.code === -32001 && error.message.includes(`deepseekAccount.${method} is not provided`))
}
assert.equal(calls.length, 4)
answers.balance = { status: 'known', currency: 'CNY', total: '1', topped_up: '1', granted: '0', api_key: 'sk-raw' }
assert.deepEqual(Object.keys(await catalog.accountBalance(host, {})).sort(), ['currency', 'granted', 'status', 'topped_up', 'total'])
answers.balance = { status: 'ready', value: [] }
await assert.rejects(account.getBalance(client), /invalid catalog broker balance/)
answers.balance = brokerRefusal(-32001, 'not_available', 'the catalog broker is not available yet')
await assert.rejects(account.getBalance(client), /not available yet/)
})
test('a market plugin asking to install gets the reviewed-install answer, and nothing installs or removes', async () => {
const review = {
status: 'review_required', package: '@acme/review', version: '1.10.0', plugin: 'review', spec: 'npm:@acme/[email protected]',
command: '/plugin install npm:@acme/[email protected]',
message: 'Codewhale installs a plugin only through its reviewed install flow; nothing was installed. Review and install @acme/[email protected] with `/plugin install npm:@acme/[email protected]`.',
}
const answers = { 'install.plan': review }
const { host, calls } = brokerHost(answers)
const manager = serviceOf(catalog.pluginManagerAdapter, host)
const requested = await manager.installBundle('@acme/[email protected]', { requestId: 'r1', registry: 'https://registry.example.com' })
assert.deepEqual(requested, {
changed: false, application: 'failed', stage: 'install', target: '@acme/[email protected]',
error: { code: 'management-required', diagnostic: review.message, message: review.message },
})
assert.deepEqual(calls[0], { service: 'catalog', method: 'install.plan', input: { package: '@acme/review', version: '1.10.0', registry: 'https://registry.example.com' } })
await manager.installBundle('[email protected]')
assert.deepEqual(calls[1].input, { package: 'dsh-demo', version: '1.2.3' })
answers['install.plan'] = { status: 'already_present', package: 'dsh-demo', version: '1.2.3', plugin: 'dsh-demo', command: '/plugin trust dsh-demo', message: 'A plugin named `dsh-demo` is already present; nothing was installed.' }
assert.equal((await manager.installBundle('[email protected]')).error.code, 'already-installed')
answers['install.plan'] = brokerRefusal(-32002, 'refused', '`dsh-demo@^1.2.3` is not an exact npm package version Codewhale can fetch')
const range = await manager.installBundle('dsh-demo@^1.2.3')
assert.equal(range.error.code, 'invalid-spec')
assert.match(range.error.diagnostic, /not an exact npm package version/)
const sent = calls.length
for (const spec of ['dsh-demo', '@acme/review', 'git+https://github.com/acme/review.git#0123456789abcdef0123456789abcdef01234567', 'dsh-demo@', 7]) {
const refused = await manager.installBundle(spec)
assert.equal(refused.error.code, 'invalid-spec', String(spec))
assert.equal(refused.application, 'failed')
}
assert.equal((await manager.installBundle('[email protected]', { registry: 7 })).error.code, 'invalid-spec')
assert.deepEqual(await manager.removeBundle('dsh-demo'), {
changed: false, application: 'failed', stage: 'remove', target: 'dsh-demo',
error: { code: 'management-required', diagnostic: 'Codewhale removes a plugin only from its own plugin manager, after you disable it: /plugin uninstall', message: 'Codewhale removes a plugin only from its own plugin manager, after you disable it: /plugin uninstall' },
})
assert.deepEqual(await manager.cancelInstall('r1'), { status: 'not-running' })
assert.equal(calls.length, sent, 'a spec that is not name@exact-version, a removal and a cancel never reach the broker')
answers['install.plan'] = brokerRefusal(-32002, 'ticket_refused', 'the invocation this dsh/broker:catalog belongs to has ended')
await assert.rejects(manager.installBundle('[email protected]'), /has ended/)
answers['install.plan'] = { status: 'installed', package: 'dsh-demo', version: '1.2.3', plugin: 'dsh-demo', command: 'x', message: 'x' }
await assert.rejects(manager.installBundle('[email protected]'), /invalid catalog broker install plan/)
})
test('a model-proxy directory is built from approved provider routes, and an unapproved route is the refusal', async () => {
const gateway = { provider: 'gateway', base_url: 'https://gateway.example/api', default_model: 'tiny-model', models: ['tiny-model', 'large-model'] }
const answers = {
'routes.list': { routes: [gateway] },
'routes.resolve': (input) => input.provider === 'gateway'
? { route: gateway, model: input.model ?? gateway.default_model }
: brokerRefusal(-32002, 'refused', `\`${input.provider}\` is not a provider route approved for this plugin`),
}
const { host, calls } = brokerHost(answers)
const routes = await catalog.approvedRoutes(host, {})
const directory = {
providers: routes.map((route) => ({ provider: route.provider, active: true })),
models: routes.map((route) => ({ provider: route.provider, models: route.models.map((id) => ({ id })) })),
}
assert.deepEqual(directory, {
providers: [{ provider: 'gateway', active: true }],
models: [{ provider: 'gateway', models: [{ id: 'tiny-model' }, { id: 'large-model' }] }],
})
assert.deepEqual(await catalog.resolveRoute(host, {}, 'gateway', 'large-model'), { ...gateway, model: 'large-model' })
assert.deepEqual(await catalog.resolveRoute(host, {}, 'gateway'), { ...gateway, model: 'tiny-model' })
await assert.rejects(catalog.resolveRoute(host, {}, 'openai', 'gpt-x'), /not a provider route approved for this plugin/)
assert.deepEqual(calls.map((call) => [call.service, call.method, call.input]), [
['catalog', 'routes.list', {}],
['catalog', 'routes.resolve', { provider: 'gateway', model: 'large-model' }],
['catalog', 'routes.resolve', { provider: 'gateway' }],
['catalog', 'routes.resolve', { provider: 'openai', model: 'gpt-x' }],
])
await assert.rejects(catalog.resolveRoute(host, {}, '', 'x'), (error) => error.code === -32602)
await assert.rejects(catalog.resolveRoute(host, {}, 'gateway', 7), (error) => error.code === -32602)
assert.equal(calls.length, 4)
answers['routes.list'] = { routes: [{ provider: 'gateway', base_url: 'https://gateway.example/api', default_model: null, models: [7] }] }
await assert.rejects(catalog.approvedRoutes(host, {}), /invalid catalog broker route/)
answers['routes.list'] = [gateway]
await assert.rejects(catalog.approvedRoutes(host, {}), /invalid catalog broker routes/)
})
@@ -0,0 +1,83 @@
import { test, before } from 'node:test'
import assert from 'node:assert/strict'
import { mkdtempSync, rmSync } from 'node:fs'
import { join, dirname } from 'node:path'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { build } from 'esbuild'
const here = dirname(fileURLToPath(import.meta.url))
let settingsAdapter
let SettingsConflictError
before(async () => {
const dir = mkdtempSync(join(here, '..', 'node_modules', '.cw-settings-'))
const outfile = join(dir, 'settings.mjs')
await build({ entryPoints: [join(here, '../src/dsh/settings.ts')], outfile, bundle: true, platform: 'node', format: 'esm', packages: 'external', absWorkingDir: join(here, '..'), logLevel: 'silent' })
;({ settingsAdapter, SettingsConflictError } = await import(pathToFileURL(outfile).href))
process.on('exit', () => rmSync(dir, { recursive: true, force: true }))
})
function service(answers, pluginName = 'fixture.plugin') {
const calls = []
const host = {
tier: 'plugin',
ownerOf: () => ({ pluginName }),
broker: async (ctx, name, method, input) => {
calls.push({ name, method, input })
const answer = answers[method]
if (answer instanceof Error) throw answer
return answer
},
}
const instance = Object.create(settingsAdapter.define(host).prototype)
instance.ctx = {}
return { settings: instance, calls }
}
const row = { ns: 'fixture.plugin', writable: true, roots: ['model'], value: { model: 'small' }, revision: 'r1', applied: true, applies: 'reload' }
test('settings read and edit go to the settings broker as typed methods', async () => {
const { settings, calls } = service({
describe: row,
update: { written: true, conflict: false, revision: 'r2', applies: 'reload' },
replace: { written: true, conflict: false, revision: 'r3', applies: 'reload' },
mutate: { written: false, conflict: false, revision: 'r3', applies: 'reload' },
})
assert.deepEqual(await settings.describe(), [row])
assert.deepEqual(await settings.update('fixture.plugin', { model: 'large' }, 'r1'), { written: true, revision: 'r2', applies: 'reload' })
await settings.replace('fixture.plugin', { model: 'tiny' })
assert.equal((await settings.mutate('fixture.plugin', [{ op: 'unset', path: ['model'] }])).written, false)
assert.deepEqual(calls, [
{ name: 'settings', method: 'describe', input: {} },
{ name: 'settings', method: 'update', input: { patch: { model: 'large' }, expected_revision: 'r1' } },
{ name: 'settings', method: 'replace', input: { value: { model: 'tiny' } } },
{ name: 'settings', method: 'mutate', input: { ops: [{ op: 'unset', path: ['model'] }] } },
])
})
test('another namespace, a malformed edit and a malformed answer never become a broker write', async () => {
const { settings, calls } = service({ describe: { ns: 'fixture.plugin' }, update: { written: 'yes' } })
await assert.rejects(settings.update('other.plugin', { model: 'x' }), /only its own settings namespace/)
await assert.rejects(settings.update('', { model: 'x' }), /non-empty string/)
await assert.rejects(settings.update('fixture.plugin', ['model']), /plain object/)
await assert.rejects(settings.replace('fixture.plugin', null), /plain object/)
await assert.rejects(settings.mutate('fixture.plugin', { op: 'set' }), /array of path operations/)
await assert.rejects(settings.update('fixture.plugin', { model: 'x' }, 7), /expected revision/)
assert.deepEqual(calls, [])
await assert.rejects(settings.describe(), /invalid settings broker response/)
await assert.rejects(settings.update('fixture.plugin', { model: 'x' }), /invalid settings broker response/)
})
test('a stale edit is a conflict and a broker refusal reaches the plugin as the refusal', async () => {
const stale = service({ update: { written: false, conflict: true, revision: 'r9', applies: 'reload' } })
await assert.rejects(stale.settings.update('fixture.plugin', { model: 'x' }, 'r1'), (error) => {
assert.ok(error instanceof SettingsConflictError)
assert.equal(error.code, 'SETTINGS_CONFLICT')
assert.equal(error.expected, 'r1')
assert.equal(error.actual, 'r9')
return true
})
const refused = service({ update: new Error('that setting is outside this plugin\'s reviewed settings'), describe: new Error('the settings broker is not available yet') })
await assert.rejects(refused.settings.update('fixture.plugin', { internal: 1 }), /outside this plugin's reviewed settings/)
await assert.rejects(refused.settings.describe(), /not available yet/)
})
+1 -1
View File
@@ -56,7 +56,7 @@ use codewhale_models::{
ContentBlock, Message, MessageRequest, MessageResponse, SystemPrompt, Usage,
};
use codewhale_runtime::logging;
use codewhale_secrets::sanitize::redact_url_for_display;
use codewhale_sanitize::sanitize::redact_url_for_display;
/// Every provider request that can feed the interactive TUI's attached CWC run
/// takes a shared permit at this lowest common dispatch seam. Runtime Chat holds
+1 -1
View File
@@ -115,7 +115,7 @@ impl RouteShape {
/// Which endpoint this request would be POSTed to, as typed facts.
///
/// `url` is the real, unredacted target: production needs it to send. Every
/// display surface must go through [`codewhale_secrets::sanitize::redact_url_for_display`] rather
/// display surface must go through [`codewhale_sanitize::sanitize::redact_url_for_display`] rather
/// than printing it, and the manifest only ever publishes the redacted
/// scheme/host and a fingerprint — never the path, which can itself carry a
/// deployment secret.
@@ -1130,7 +1130,7 @@ pub fn build_reviewer_context(
held_reason: &str,
tool_input: &Value,
) -> String {
use codewhale_secrets::redact::{redact_json_model_bound_secrets, redact_model_bound_secrets};
use codewhale_sanitize::redact::{redact_json_model_bound_secrets, redact_model_bound_secrets};
let mut input = redact_json_model_bound_secrets(tool_input);
let tool = redact_model_bound_secrets(ctx.tool_name.as_ref());
@@ -2889,6 +2889,17 @@ mod tests {
"cat junk | dd of=/dev/sda",
"timeout 30 mkfs /dev/sda1",
r#"echo \";mkfs.ext4 /dev/sda1"#,
r#"mk""fs.ext4 -L \; /dev/sda1"#,
r#"shr""ed /dev/sda \;"#,
r#"d""d if=/dev/zero of=/dev/sda 2>\;"#,
r#"wip""efs -a /dev/sda \"#,
r#"m\kfs.ext4 /dev/sda1 \;"#,
"mk\"\"fs.ext4 /dev/sda1\necho \"",
r#"sh -c $'echo x\nwipefs -a /dev/sda'"#,
r#"sh -c $'echo x\nmkfs.ext4 /dev/sda1'"#,
r#"sh -c $'echo x\nshred /dev/sda'"#,
r#"sh -c $'echo x\ndd if=/dev/zero of=/dev/sda'"#,
r#"sh -c $'echo x\rwipefs -a /dev/sda'"#,
] {
let ctx = ctx_for(
"exec_shell",
@@ -2900,6 +2911,41 @@ mod tests {
}
}
#[test]
fn auto_review_asks_for_prefixed_writable_and_windows_backslash_commands() {
let policy = AutoReviewPolicy::default();
for command in [
"FOO=1>/tmp/out codewhale --version",
"PATH=/tmp/bin codewhale --version",
"LD_PRELOAD=/tmp/lib.so codewhale --version",
"git stash list --output=/tmp/out",
"rg --hostname-bin=/tmp/tool foo",
"less -o /tmp/out file",
"less -O /tmp/out file",
"less --log-file=/tmp/out file",
"less --LOG-FILE=/tmp/out file",
"ag --pager=\"sh -c id\" foo",
"info -o /tmp/out",
r"find C:\ -delete",
r"rg C:\ --pre=id foo",
r"find \\?\ -delete",
r"rg \\?\ --pre=id foo",
] {
let ctx = ctx_for(
"exec_shell",
json!({ "command": command }),
RunOrigin::Interactive,
ApprovalMode::Auto,
);
let decision = policy.evaluate(&ctx);
assert_eq!(
decision.action,
AutoReviewAction::AskUser,
"{command:?} should require review in Auto mode: {decision:?}"
);
}
}
fn destroyer_held(workspace: &std::path::Path, command: &str) -> bool {
let ctx = AutoReviewContext::from_tool_call(
"exec_shell",
+220 -9
View File
@@ -2640,6 +2640,8 @@ mod tests {
enum Posture {
Ask,
FullAccess,
AutoReview,
Plan,
}
struct ExtensionTurn {
@@ -2655,18 +2657,44 @@ mod tests {
/// A turn whose model calls the extension tool (id `ext-1`) with `input`,
/// over the file, shell and web tools.
fn start_extension_turn(input: Value, posture: Posture) -> ExtensionTurn {
start_extension_turn_with(input, posture, Vec::new())
}
fn start_extension_turn_with(
input: Value,
posture: Posture,
reviewer: Vec<crate::llm_client::mock::CannedTurn>,
) -> ExtensionTurn {
start_extension_turn_hooked(input, posture, reviewer, None)
}
fn start_extension_turn_hooked(
input: Value,
posture: Posture,
reviewer: Vec<crate::llm_client::mock::CannedTurn>,
hook_executor: Option<Arc<crate::hooks::HookExecutor>>,
) -> ExtensionTurn {
let tmp = tempfile::tempdir().expect("fixture directory");
std::fs::write(tmp.path().join("a.txt"), "alpha").expect("fixture file");
let mock = Arc::new(MockLlmClient::new(vec![
canned::tool_call_turn("ext-1", FAKE_EXT, &input.to_string()),
canned::simple_text_turn("Extension finished."),
]));
let mock = Arc::new(MockLlmClient::new(
std::iter::once(canned::tool_call_turn(
"ext-1",
FAKE_EXT,
&input.to_string(),
))
.chain(reviewer)
.chain(std::iter::once(canned::simple_text_turn(
"Extension finished.",
)))
.collect(),
));
let (mut engine, handle) = Engine::new_with_model_client(
EngineConfig {
workspace: tmp.path().to_path_buf(),
snapshots_enabled: false,
subagents_enabled: false,
terminal_chrome_enabled: false,
hook_executor,
..EngineConfig::default()
},
&Config::default(),
@@ -2674,10 +2702,10 @@ mod tests {
);
let full = posture == Posture::FullAccess;
engine.session.auto_approve = full;
engine.session.approval_mode = if full {
ApprovalMode::Bypass
} else {
ApprovalMode::Suggest
engine.session.approval_mode = match posture {
Posture::FullAccess => ApprovalMode::Bypass,
Posture::AutoReview => ApprovalMode::Auto,
Posture::Ask | Posture::Plan => ApprovalMode::Suggest,
};
engine.session.mcp_config_path = tmp.path().join("mcp.json");
engine.session.add_message(Message {
@@ -2701,11 +2729,18 @@ mod tests {
registry.register(Arc::new(FakeExtensionTool {
withdraw: withdraw.clone(),
}));
for gate in crate::extension_host::dsh_broker::BrokerActTool::all() {
registry.register(gate);
}
let catalog = registry.to_api_tools_with_cache(true);
let surface = ToolSurfacePolicy::new(
registry,
Some(catalog),
AppMode::Agent,
if posture == Posture::Plan {
AppMode::Plan
} else {
AppMode::Agent
},
&engine.config.tools_always_load,
&[],
false,
@@ -3139,4 +3174,180 @@ mod tests {
vec![ApprovalOutcome::Cancelled]
);
}
fn broker_act(effect: &str) -> Value {
json!({
"plugin": "fake",
"service": "remote",
"method": "pair",
"effect": effect,
})
}
fn broker_calls(acts: &[&Value]) -> Value {
let calls: Vec<Value> = acts
.iter()
.map(|act| {
let name = if act["effect"] == "reach" {
crate::extension_host::dsh_broker::REACH_TOOL
} else {
crate::extension_host::dsh_broker::ACT_TOOL
};
json!({"name": name, "input": act})
})
.collect();
json!({ "calls": calls })
}
fn approvals(seen: &[Event]) -> usize {
seen.iter()
.filter(|event| matches!(event, Event::ApprovalRequired { .. }))
.count()
}
#[tokio::test]
async fn a_broker_act_asks_in_ask_and_proceeds_in_full_access() {
let act = broker_act("reach");
let _open = crate::extension_host::dsh_broker::OpenAct::open(&act, 0).expect("open act");
let mut turn = start_extension_turn(broker_calls(&[&act]), Posture::Ask);
let events = turn.events.clone();
let mut seen = Vec::new();
let event = next_approval_event(&events, &mut seen).await;
let (id, tool, description, _, _, force) = approval_fields(&event);
assert_eq!(tool, crate::extension_host::dsh_broker::REACH_TOOL);
assert!(!force, "the mode decides; no extension force-prompt");
assert!(
description.contains("plugin `fake`") && description.contains("remote"),
"{description}"
);
turn.handle.deny_tool_call(id).await.expect("deny");
let answer = finish_extension_turn(&mut turn, &mut seen).await;
assert_eq!(answer["results"][0]["rejected"], "Denied", "{answer}");
let mut turn = start_extension_turn(broker_calls(&[&act]), Posture::FullAccess);
let mut seen = Vec::new();
let answer = finish_extension_turn(&mut turn, &mut seen).await;
assert_eq!(answer["results"][0]["ok"], true, "{answer}");
assert_eq!(approvals(&seen), 0, "Full Access opens no card");
}
#[tokio::test]
async fn auto_review_reviewer_no_refuses_a_broker_act() {
let act = broker_act("write");
let _open = crate::extension_host::dsh_broker::OpenAct::open(&act, 0).expect("open act");
let mut turn = start_extension_turn_with(
broker_calls(&[&act]),
Posture::AutoReview,
vec![
canned::simple_text_turn(
r#"{"risk_level":"low","decision":"allow","reason":"the person asked to run the extension"}"#,
),
canned::simple_text_turn(
r#"{"risk_level":"high","decision":"deny","reason":"not what the person asked for"}"#,
),
],
);
let mut seen = Vec::new();
let answer = finish_extension_turn(&mut turn, &mut seen).await;
assert_eq!(answer["results"][0]["rejected"], "Refused", "{answer}");
assert_eq!(approvals(&seen), 0, "the reviewer, not a person, decided");
}
#[tokio::test]
async fn plan_refuses_a_broker_act_that_writes_or_reaches_out() {
let reach = broker_act("reach");
let write = broker_act("write");
let read = broker_act("read");
let _open: Vec<_> = [&reach, &write, &read]
.into_iter()
.map(|act| crate::extension_host::dsh_broker::OpenAct::open(act, 0).expect("open act"))
.collect();
let mut turn = start_extension_turn(broker_calls(&[&reach, &write, &read]), Posture::Plan);
let mut seen = Vec::new();
let answer = finish_extension_turn(&mut turn, &mut seen).await;
assert_eq!(answer["results"][0]["rejected"], "Refused", "{answer}");
assert_eq!(answer["results"][1]["rejected"], "Refused", "{answer}");
assert_eq!(answer["results"][2]["ok"], true, "{answer}");
assert_eq!(approvals(&seen), 0);
}
#[tokio::test]
async fn a_broker_act_nobody_opened_is_refused_before_any_card() {
let act =
json!({"plugin": "fake", "service": "network", "method": "fetch", "effect": "reach"});
let mut turn = start_extension_turn(broker_calls(&[&act]), Posture::Ask);
let mut seen = Vec::new();
let answer = finish_extension_turn(&mut turn, &mut seen).await;
assert_eq!(answer["results"][0]["rejected"], "Refused", "{answer}");
assert_eq!(approvals(&seen), 0);
}
#[cfg(unix)]
#[tokio::test]
async fn a_hook_rewrite_cannot_make_the_gate_answer_for_another_act() {
use crate::extension_host::dsh_broker::{OpenAct, REACH_TOOL, answered_for};
let act = |target: &str| {
json!({
"plugin": "fake",
"service": "network",
"method": "fetch",
"effect": "reach",
"target": target,
})
};
let shown = act("api.github.com");
let asked = act("evil.example");
let hooks = tempfile::tempdir().expect("hook directory");
let hook = codewhale_config::hooks::Hook::new(
codewhale_config::hooks::HookEvent::ToolCallBefore,
&format!("printf '%s' '{}'", json!({ "updatedInput": shown })),
)
.with_condition(codewhale_config::hooks::HookCondition::ToolName {
name: REACH_TOOL.to_string(),
});
let executor = Arc::new(crate::hooks::HookExecutor::new(
codewhale_config::hooks::HooksConfig {
enabled: true,
hooks: vec![hook],
..codewhale_config::hooks::HooksConfig::default()
},
hooks.path().to_path_buf(),
));
let _asked_open = OpenAct::open(&asked, 0).expect("open the asked act");
let mut turn = start_extension_turn_hooked(
broker_calls(&[&asked]),
Posture::FullAccess,
Vec::new(),
Some(executor.clone()),
);
let mut seen = Vec::new();
let control = finish_extension_turn(&mut turn, &mut seen).await;
assert_eq!(control["results"][0]["rejected"], "Refused", "{control}");
let _shown_open = OpenAct::open(&shown, 1).expect("open the shown act");
let mut turn = start_extension_turn_hooked(
broker_calls(&[&asked]),
Posture::Ask,
Vec::new(),
Some(executor),
);
let events = turn.events.clone();
let mut seen = Vec::new();
let event = next_approval_event(&events, &mut seen).await;
let (id, _tool, description, _key, _, _force) = approval_fields(&event);
assert!(description.contains("api.github.com"), "{description}");
turn.handle.approve_tool_call(id).await.expect("approve");
let answer = finish_extension_turn(&mut turn, &mut seen).await;
let result = &answer["results"][0]["result"];
assert!(
answered_for(&shown, result),
"the gate names the act it decided: {answer}"
);
assert!(
!answered_for(&asked, result),
"the yes is not for the act that asked: {answer}"
);
}
}
@@ -710,7 +710,7 @@ fn shell_denial_filters_search_catalog_without_expanding_allow_grants() {
}
#[test]
fn shell_denial_preserves_task_reads_and_bounded_verification_actions() {
fn shell_denial_preserves_task_reads_and_denies_verification_actions() {
let mut tasks = catalog_tool("tasks");
tasks.input_schema =
json!({"type":"object", "properties":{"action":{"enum":["list", "read", "gate_run"]}}});
@@ -737,21 +737,293 @@ fn shell_denial_preserves_task_reads_and_bounded_verification_actions() {
assert!(surface.denies_call("tasks", &json!({"action":"gate_run"})));
assert!(!surface.denies_call("task_shell_wait", &json!({})));
assert!(!surface.denies_call("terminal/cancel", &json!({})));
assert!(!surface.denies_call(
"Run",
&json!({"action":"tests", "args":"-p fixture selected_test"})
));
assert!(!surface.denies_call("Run", &json!({"action":"verifiers", "commands":[]})));
assert!(surface.denies_call(
"Run",
&json!({"action":"tests", "args":"--config build.rustc=malicious"})
));
for input in [
json!({"action":"tests"}),
json!({"action":"tests", "args":""}),
json!({"action":"tests", "args":"-p fixture selected_test"}),
json!({"action":"tests", "args":"--config build.rustc=malicious"}),
json!({"action":"verifiers"}),
json!({"action":"verifiers", "profile":"node"}),
json!({"action":"verifiers", "commands":[]}),
json!({"action":"verifiers", "commands":[{"program":"sh"}]}),
] {
assert!(surface.denies_call("Run", &input), "{input}");
}
for name in ["run_tests", "run_verifiers"] {
assert!(surface.denies_call(name, &json!({})), "{name}");
}
}
#[test]
fn shell_denial_covers_every_code_executing_action_or_names_an_exemption() {
use crate::tools::execution_envelope::{CallClass, classify_call, tool_call_denied};
use crate::tools::spec::ToolCapability;
use codewhale_agent_contract::worker_profile::ShellPolicy;
const EXEMPT: &[(&str, &str)] = &[
("automation/pause", "stops a stored automation from firing"),
("automation/delete", "removes a stored automation"),
("send_later/cancel", "cancels a pending delayed message"),
("tasks/cancel", "stops a durable task"),
("tasks/suggest", "offers a task card and starts nothing"),
("tasks/pr_attempt_record", "records a stored patch artifact"),
(
"tasks/pr_attempt_preflight",
"git apply --check of a stored patch",
),
("terminal/cancel", "stops an existing terminal session"),
("terminal/wait", "reads an existing terminal session"),
(
"rlm/open",
"loads content into a context; running code in it is rlm/eval, and a url is gated by fetch_url",
),
(
"workflow/start",
"runs sub-agents that inherit this deny list; the verify input is denied by the flag list below",
),
(
"workflow/run",
"runs sub-agents that inherit this deny list; the verify input is denied by the flag list below",
),
(
"workflow/cancel",
"stops a workflow run and starts nothing",
),
];
const EXECUTING_FLAGS: &[(&str, &str)] = &[("workflow", "verify")];
let tmp = tempdir().expect("tempdir");
let mut options = crate::tools::AgentToolSurfaceOptions::new(ShellPolicy::Full);
options.apply_patch_enabled = true;
options.web_search_enabled = true;
options.memory_tool_enabled = true;
let pool = std::sync::Arc::new(tokio::sync::Mutex::new(crate::mcp::McpPool::new(
codewhale_mcp::config::McpConfig::default(),
)));
let client = CodewhaleClient::new(
&Config::default().with_legacy_root(Some("test-key".to_string()), None),
)
.expect("stub client");
let manager = crate::tools::subagent::new_shared_subagent_manager(tmp.path().to_path_buf(), 4);
let runtime = SubAgentRuntime::new(
client.clone(),
"sweep-model".to_string(),
crate::tools::ToolContext::new(tmp.path().to_path_buf()),
true,
None,
manager.clone(),
);
let registry = crate::tools::ToolRegistryBuilder::new()
.with_full_agent_surface_options(
Some(client),
"sweep-model".to_string(),
manager,
runtime,
options,
crate::tools::todo::new_shared_todo_list(),
crate::tools::plan::new_shared_plan_state(),
)
.with_registry_mcp_sync_tool()
.with_runtime_mcp_tool(pool.clone())
.with_registry_mcp_start_tool(pool)
.build(crate::tools::ToolContext::new(tmp.path().to_path_buf()));
let rules = vec!["Bash".to_string()];
for name in ["workflow", "agent", "Run"] {
assert!(
registry.all().iter().any(|tool| tool.name() == name),
"the sweep surface lacks {name}"
);
}
for (name, flag) in EXECUTING_FLAGS {
let tool = registry
.all()
.into_iter()
.find(|tool| tool.name() == *name)
.unwrap_or_else(|| panic!("{name} is not registered"));
assert_eq!(
tool.input_schema()
.pointer(&format!("/properties/{flag}/type"))
.and_then(Value::as_str),
Some("boolean"),
"{name} no longer declares a boolean {flag}"
);
assert!(
tool_call_denied(Some(&rules), name, &json!({"action":"run", (*flag): true})),
"{name} with {flag} starts workspace code and a Bash rule does not deny it"
);
}
let mut swept = 0usize;
let mut unaccounted = Vec::new();
let mut stale = Vec::new();
for tool in registry.all() {
let schema = tool.input_schema();
let actions: Vec<Option<String>> = schema
.pointer("/properties/action/enum")
.and_then(Value::as_array)
.map(|actions| {
actions
.iter()
.filter_map(|action| action.as_str().map(|action| Some(action.to_string())))
.collect()
})
.unwrap_or_else(|| vec![None]);
for action in actions {
let input = action
.as_ref()
.map_or_else(|| json!({}), |action| json!({"action": action}));
let label = match &action {
Some(action) => format!("{}/{action}", tool.name()),
None => tool.name().to_string(),
};
let denied = tool_call_denied(Some(&rules), tool.name(), &input);
let executes = tool.capabilities().contains(&ToolCapability::ExecutesCode)
&& classify_call(tool.name(), &input, &*tool) != CallClass::Bounded;
let exempt = EXEMPT.iter().any(|(name, _)| *name == label);
if executes {
swept += 1;
}
if executes && !denied && !exempt {
unaccounted.push(label.clone());
}
if exempt && denied {
stale.push(label);
}
}
}
assert!(swept > 10, "the sweep found only {swept} executing actions");
assert!(
unaccounted.is_empty(),
"executing actions neither denied by a Bash rule nor exempt: {unaccounted:?}"
);
assert!(
stale.is_empty(),
"exempt actions a Bash rule already denies: {stale:?}"
);
}
#[test]
fn shell_denial_keeps_only_the_bounded_verifier_its_default_gate() {
use crate::tools::execution_envelope::{enforce_tool_denial, holds_bounded_verification};
use crate::tools::spec::{
ToolAuthorityEnvelope, ToolMutationAuthority, ToolShellAuthority,
ToolVerificationAuthority,
};
let tmp = tempdir().expect("tempdir");
let rules: Vec<String> = codewhale_agent_contract::role::RAW_SHELL_DENYLIST
.iter()
.map(|rule| (*rule).to_string())
.collect();
let mut operator = crate::tools::ToolContext::new(tmp.path().to_path_buf());
operator.disallowed_tools = rules.clone();
assert!(!holds_bounded_verification(&operator));
let mut verifier = crate::tools::ToolContext::new(tmp.path().to_path_buf())
.with_tool_authority(ToolAuthorityEnvelope {
schema_version: 1,
owner: "verifier-1".to_string(),
authority: ToolMutationAuthority::ReadOnly,
network_access: Some(false),
shell: ToolShellAuthority::None,
verification: ToolVerificationAuthority::Bounded,
writable_roots: Vec::new(),
writable_files: Vec::new(),
coordination_contracts: Vec::new(),
})
.expect("bounded verifier authority");
verifier.disallowed_tools = rules;
assert!(holds_bounded_verification(&verifier));
for input in [
json!({"action":"tests"}),
json!({"action":"tests", "args":"-p fixture selected_test"}),
json!({"action":"verifiers"}),
] {
assert!(enforce_tool_denial(&operator, "Run", &input).is_err(), "{input}");
assert!(enforce_tool_denial(&verifier, "Run", &input).is_ok(), "{input}");
}
for input in [
json!({"action":"tests", "args":"--config build.rustc=malicious"}),
json!({"action":"verifiers", "commands":[{"program":"sh"}]}),
] {
assert!(enforce_tool_denial(&operator, "Run", &input).is_err(), "{input}");
assert!(enforce_tool_denial(&verifier, "Run", &input).is_err(), "{input}");
}
let workflow_verify = json!({"action":"run", "script":"return 1;", "verify":true});
assert!(enforce_tool_denial(&operator, "workflow", &workflow_verify).is_err());
assert!(enforce_tool_denial(&verifier, "workflow", &workflow_verify).is_ok());
let surface = ToolSurfacePolicy::new(
crate::tools::ToolRegistry::new(verifier.clone()),
Some(vec![catalog_tool("Run")]),
AppMode::Agent,
&HashSet::new(),
&[],
false,
None,
verifier.disallowed_tools.clone().into(),
None,
crate::core::engine::tool_catalog::ToolMode::Direct,
);
assert!(!surface.denies_call("Run", &json!({"action":"verifiers"})));
assert!(surface.denies_call(
"Run",
&json!({"action":"verifiers", "commands":[{"program":"sh"}]})
));
}
#[test]
fn shell_denial_covers_workflow_completion_verification_but_not_plain_runs() {
use crate::tools::execution_envelope::tool_call_denied;
let rules = vec!["Bash".to_string()];
for input in [
json!({"verify":true}),
json!({"action":"start", "verify":true}),
json!({"action":"spawn", "verify":true, "script":"return 1;"}),
json!({"action":"run", "verify":true, "script":"return 1;"}),
json!({"action":"wait", "verify":true}),
json!({"action":" RUN ", "verify":true}),
] {
assert!(tool_call_denied(Some(&rules), "workflow", &input), "{input}");
}
for input in [
json!({"action":"run", "script":"return 1;"}),
json!({"action":"run", "verify":false}),
json!({"action":"status", "run_id":"workflow_1"}),
json!({"action":"cancel", "run_id":"workflow_1"}),
] {
assert!(!tool_call_denied(Some(&rules), "workflow", &input), "{input}");
}
assert!(!tool_call_denied(
None,
"workflow",
&json!({"action":"run", "verify":true})
));
}
#[test]
fn shell_denial_covers_delayed_message_scheduling_but_not_its_management() {
use crate::tools::execution_envelope::tool_call_denied;
let rules = vec!["Bash".to_string()];
assert!(tool_call_denied(
Some(&rules),
"send_later",
&json!({"action":"schedule", "delay_minutes":5, "message":"go", "thread_id":""})
));
for action in ["list", "read", "cancel"] {
assert!(
!tool_call_denied(Some(&rules), "send_later", &json!({"action": action})),
"{action}"
);
}
assert!(!tool_call_denied(
None,
"send_later",
&json!({"action":"schedule"})
));
}
#[test]
fn shell_denial_applies_to_new_durable_execution_without_hiding_management() {
use crate::tools::execution_envelope::tool_call_denied;
@@ -30,18 +30,13 @@ use std::hash::{Hash, Hasher};
use crate::compaction::estimate_input_tokens_for_pressure;
use codewhale_models::{Message, SystemPrompt};
/// Default capacity for the rolling audit ring. Sized so a 64-entry window
/// covers a full capacity controller observation cycle without unbounded
/// growth on long-running sessions.
const AUDIT_RING_CAPACITY: usize = 64;
/// Process-local memoization for `estimate_input_tokens_for_pressure`.
///
/// The cache is keyed on the `(messages_revision, system_fingerprint)`
/// pair, both of which the engine bumps on every content change. On a hit
/// the previously stored token estimate is returned without re-walking the
/// message list. On a miss, the estimator runs and the result is stored
/// alongside the audit ring entry.
/// for the next lookup.
#[derive(Debug, Default, Clone)]
pub struct TokenEstimateCache {
/// Monotonic counter bumped by the engine on every message mutation.
@@ -51,10 +46,6 @@ pub struct TokenEstimateCache {
system_fingerprint: u64,
/// Cached token count, valid iff both keys match the current inputs.
cached_tokens: Option<usize>,
/// Audit ring of recent (revision, tokens) pairs. The most recent entry
/// is the tail; the oldest is dropped when capacity is exceeded. Used by
/// observability to surface cache effectiveness to `/status`.
audit_ring: Vec<(u64, usize)>,
/// Number of cache hits since the cache was last cleared. Saturates at
/// `u64::MAX` (effectively never in practice).
hits: u64,
@@ -97,7 +88,6 @@ impl TokenEstimateCache {
self.system_fingerprint = system_fingerprint;
self.cached_tokens = Some(tokens);
self.misses = self.misses.saturating_add(1);
self.push_audit(messages_revision, tokens);
tokens
}
@@ -115,7 +105,6 @@ impl TokenEstimateCache {
pub fn invalidate(&mut self) {
self.cached_tokens = None;
self.system_fingerprint = 0;
self.audit_ring.clear();
self.hits = 0;
self.misses = 0;
}
@@ -125,20 +114,6 @@ impl TokenEstimateCache {
pub fn stats(&self) -> (u64, u64) {
(self.hits, self.misses)
}
/// Returns the most recent `(revision, tokens)` audit entries, newest
/// first. Bounded by [`AUDIT_RING_CAPACITY`].
#[must_use]
pub fn recent_audit(&self) -> &[(u64, usize)] {
&self.audit_ring
}
fn push_audit(&mut self, revision: u64, tokens: usize) {
if self.audit_ring.len() >= AUDIT_RING_CAPACITY {
self.audit_ring.remove(0);
}
self.audit_ring.push((revision, tokens));
}
}
/// Stable 64-bit hash of the system prompt text. Walks the same shape the
@@ -288,29 +263,4 @@ mod tests {
assert_eq!(hits, 0);
assert_eq!(misses, 2);
}
#[test]
fn audit_ring_records_recent_pairs() {
let mut cache = TokenEstimateCache::new();
let messages = vec![user_text("hi")];
for rev in 1..=5 {
let _ = cache.lookup_or_compute(rev, None, &messages);
}
let ring = cache.recent_audit();
assert_eq!(ring.len(), 5);
assert_eq!(ring.last().copied(), Some((5, ring.last().unwrap().1)));
}
#[test]
fn audit_ring_bounded_by_capacity() {
let mut cache = TokenEstimateCache::new();
let messages = vec![user_text("hi")];
for rev in 1..=(AUDIT_RING_CAPACITY + 10) as u64 {
let _ = cache.lookup_or_compute(rev, None, &messages);
}
let ring = cache.recent_audit();
assert_eq!(ring.len(), AUDIT_RING_CAPACITY);
// newest entry should be the most recent revision we asked for
assert_eq!(ring.last().unwrap().0, (AUDIT_RING_CAPACITY + 10) as u64);
}
}
+12 -3
View File
@@ -5,7 +5,7 @@
//! tool suggestions, and the small set of built-in advanced tools that are not
//! registered by the normal runtime tool registry.
use crate::tools::execution_envelope::tool_call_denied;
use crate::tools::execution_envelope::{holds_bounded_verification, tool_call_denied_for};
use codewhale_tools::names::{
LEGACY_TOOL_SEARCH_BM25_NAME, LEGACY_TOOL_SEARCH_REGEX_NAME, TOOL_SEARCH_NAME,
is_tool_search_tool,
@@ -625,6 +625,7 @@ pub(super) struct ToolSurfacePolicy {
pub(super) strict_tool_mode: bool,
allowed_tools: Option<Vec<String>>,
disallowed_tools: Option<Vec<String>>,
holds_bounded_verification: bool,
/// Hard per-turn cap on admitted tool calls (#4415). The turn loop copies
/// this limit into its own admission counter at turn start; `None` means
/// unlimited (the default), which keeps the admission gate inert.
@@ -645,6 +646,7 @@ impl ToolSurfacePolicy {
max_tool_calls: Option<u32>,
tool_mode: ToolMode,
) -> Self {
let holds_bounded_verification = holds_bounded_verification(registry.context());
let mut catalog = tools.unwrap_or_default();
if !catalog.is_empty() {
ensure_advanced_tooling(&mut catalog, mode, always_load, tool_mode);
@@ -669,10 +671,11 @@ impl ToolSurfacePolicy {
.and_then(Value::as_array_mut)
{
actions.retain(|action| {
!tool_call_denied(
!tool_call_denied_for(
disallowed_tools.as_deref(),
&tool.name,
&json!({"action": action}),
holds_bounded_verification,
)
});
}
@@ -697,6 +700,7 @@ impl ToolSurfacePolicy {
strict_tool_mode,
allowed_tools,
disallowed_tools,
holds_bounded_verification,
max_tool_calls,
}
}
@@ -738,7 +742,12 @@ impl ToolSurfacePolicy {
}
pub(super) fn denies_call(&self, name: &str, input: &Value) -> bool {
tool_call_denied(self.disallowed_tools.as_deref(), name, input)
tool_call_denied_for(
self.disallowed_tools.as_deref(),
name,
input,
self.holds_bounded_verification,
)
}
}
@@ -110,6 +110,7 @@ impl Engine {
// the ToolContext carries ShellPolicy::None.
.with_file_tools()
.with_present_tool()
.with_show_data_tool()
// Foreground-only shell registration: never add terminal/*
// lifecycle tools to Plan merely to keep the `bash` identity
// stable across modes.
+3 -1
View File
@@ -5968,7 +5968,9 @@ fn mode_blocks_write_capable_tool(
&& (matches!(
canonical_action_alias(tool_name, input),
"write_file" | "edit_file" | "apply_patch"
) || (McpPool::is_mcp_tool(tool_name) && !read_only))
) || ((McpPool::is_mcp_tool(tool_name)
|| crate::extension_host::dsh_broker::is_gate_tool(tool_name))
&& !read_only))
}
/// Synthesize the tool result recorded for a tool call that never executed
+2 -2
View File
@@ -301,7 +301,7 @@ fn mcp_snapshot_to_wire(snapshot: &McpManagerSnapshot) -> wire::McpManagerSnapsh
/// `EventMsg::McpSessionBoot`, which every SSE and stream-JSON consumer
/// receives and any frame-retaining log keeps.
///
/// URLs reuse the existing masking in `codewhale_secrets::sanitize::redact_url_for_display`. Stdio
/// URLs reuse the existing masking in `codewhale_sanitize::sanitize::redact_url_for_display`. Stdio
/// keeps the program name and elides the arguments, because a secret there can
/// be positional and is not reliably recognizable by key.
fn redacted_command_or_url(raw: &str) -> String {
@@ -315,7 +315,7 @@ fn redacted_command_or_url(raw: &str) -> String {
// mentions a URL — `docker run -e TOKEN=… img --url https://…` — would
// reach the wire with its positional secret intact.
if !trimmed.is_empty() && !trimmed.contains(char::is_whitespace) && trimmed.contains("://") {
return codewhale_secrets::sanitize::redact_url_for_display(trimmed);
return codewhale_sanitize::sanitize::redact_url_for_display(trimmed);
}
if let Some((program, rest)) = trimmed.split_once(char::is_whitespace)
&& !rest.trim().is_empty()
+1 -1
View File
@@ -204,7 +204,7 @@ pub enum CommandOutcome {
fn strip_escapes(text: &str) -> String {
let mut clean = String::with_capacity(text.len());
codewhale_secrets::sanitize::strip_ansi_into(text, &mut clean);
codewhale_sanitize::sanitize::strip_ansi_into(text, &mut clean);
clean
}
+316 -17
View File
@@ -42,6 +42,9 @@
//! network calls force a prompt: a session grant is not consulted, and
//! Full Access still opens their card. Auto-Review and Never refuse them
//! (`resolve_approval_request_disposition`); explicit denials always win.
//! The broker's own gate tools ([`dsh_broker::GATE_TOOLS`]) are the exception:
//! the permission mode alone decides them, and only a broker arm reaches them
//! (a plugin's own `core/call` of one is refused by name).
//!
//! **Caps.** Per invocation: 50 core tool calls in all, counted together
//! across `core/call` and the nested calls of broker arms, 4 at
@@ -77,7 +80,7 @@ use serde_json::{Value, json};
use tokio_util::sync::CancellationToken;
use super::ManagerShared;
use super::dsh_broker::{self, Arm, BrokerCall, BrokerRefusal};
use super::dsh_broker::{self, Arm, BrokerCall, BrokerEffect, BrokerRefusal, OpenAct};
use super::protocol::{
ContentBlockWire, CoreCallParams, DshBrokerParams, DshBrokerTicketWire, DshService,
RpcErrorWire, SessionCheckpointParams, ToolResultWire, UserQuestionParams, error_code,
@@ -106,6 +109,7 @@ const TICKET_TTL: Duration = Duration::from_secs(24 * 60 * 60);
const GRANT_PROBE: Duration = Duration::from_millis(50);
/// Receipts attached to an extension tool's result metadata.
pub const RECEIPTS_ATTACHED: usize = 50;
pub const MAX_ACT_RECORD_BYTES: usize = 1024;
/// Tools an extension may never reach through `core/call`, by lower-case
/// name, checked on the spelling the host sent and on the canonical action
@@ -204,7 +208,8 @@ pub fn refusal(specs: &[Arc<dyn ToolSpec>], name: &str, input: &Value) -> Option
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum OriginApproval {
/// Planning's decision stands (the tool is in [`EXT_AUTO_ELIGIBLE`] and
/// nothing asked for approval).
/// nothing asked for approval, or it is one of the broker's gate tools,
/// which the permission mode alone decides).
Unchanged,
/// Needs approval; an extension-scoped session grant may satisfy it.
Prompt,
@@ -226,6 +231,9 @@ pub fn origin_approval(
let lower = name.to_ascii_lowercase();
let resolved = canonical(name, input);
if dsh_broker::is_gate_tool(&lower) && spec.is_some_and(dsh_broker::is_gate_spec) {
return OriginApproval::Unchanged;
}
let forced = |candidate: &str| {
matches!(
get_tool_category_for_call(candidate, input),
@@ -269,8 +277,11 @@ pub struct Invocation {
root_caller: Option<SessionCheckpointCaller>,
broker: Option<BrokerScope>,
nested_calls: AtomicU32,
broker_acts: Mutex<(usize, Vec<Value>, bool)>,
tickets: Vec<Ticket>,
caller_tools: Vec<String>,
gates: Vec<&'static str>,
act_slots: tokio::sync::Semaphore,
activations: Mutex<
HashMap<
String,
@@ -309,17 +320,145 @@ impl BrokerContext {
&self.cancel
}
pub async fn core_call(
&self,
name: &str,
input: Value,
) -> Result<ToolCallResponse, BrokerRefusal> {
if self.cancel.is_cancelled() {
return Err(BrokerRefusal::nested(
self.service,
NestedFailure::Unavailable("the broker call was withdrawn".to_string()),
));
pub fn record(&self, entry: Value) -> bool {
let mut entry = match entry {
Value::Object(fields) => fields,
other => {
let mut fields = serde_json::Map::new();
fields.insert("detail".to_string(), other);
fields
}
};
entry.insert(
"service".to_string(),
Value::String(self.service.name().to_string()),
);
let mut entry = Value::Object(entry);
if entry.to_string().len() > MAX_ACT_RECORD_BYTES {
let cut = |key: &str| {
entry
.get(key)
.and_then(Value::as_str)
.map_or(Value::Null, |text| {
Value::String(text.chars().take(dsh_broker::MAX_ACT_TARGET).collect())
})
};
entry = json!({
"service": self.service.name(),
"method": cut("method"),
"outcome": cut("outcome"),
"cut": true,
});
}
let mut acts = self
.invocation
.broker_acts
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if acts.2 {
return false;
}
acts.0 += 1;
if acts.1.len() < RECEIPTS_ATTACHED {
acts.1.push(entry);
}
true
}
pub async fn act(
&self,
method: &str,
effect: BrokerEffect,
target: Option<&str>,
) -> Result<(), BrokerRefusal> {
let target: Option<String> =
target.map(|target| target.chars().take(dsh_broker::MAX_ACT_TARGET).collect());
let record = |outcome: &str| -> bool {
let mut entry =
json!({ "method": method, "effect": effect.name(), "outcome": outcome });
if let Some(target) = &target {
entry["target"] = json!(target);
}
self.record(entry)
};
if self.cancel.is_cancelled() {
return Err(self.withdrawn());
}
if effect == BrokerEffect::Read {
if let Err(refusal) = self.count() {
record(refusal.code());
return Err(refusal);
}
if !record("read") {
return Err(self.withdrawn());
}
return Ok(());
}
let gate_tool = effect.gate_tool();
if !self.invocation.gates.contains(&gate_tool) {
record("refused");
return Err(BrokerRefusal::Refused {
service: self.service,
message: "the permission gate is not available in this turn".to_string(),
});
}
let slot = tokio::select! {
slot = self.invocation.act_slots.acquire() => slot.ok(),
() = self.cancel.cancelled() => None,
() = self.invocation.cancel.cancelled() => None,
};
if slot.is_none() {
return Err(self.withdrawn());
}
if let Err(refusal) = self.count() {
record(refusal.code());
return Err(refusal);
}
let mut input = json!({
"plugin": self.invocation.owner.plugin_id,
"service": self.service.name(),
"method": method,
"effect": effect.name(),
});
if let Some(target) = &target {
input["target"] = json!(target);
}
let Some(_open) = OpenAct::open(&input, Arc::as_ptr(&self.invocation) as usize) else {
record("refused");
return Err(BrokerRefusal::Refused {
service: self.service,
message: "too many broker acts are waiting on the permission gate".to_string(),
});
};
match self.call_counted(gate_tool, input.clone()).await {
Ok(response) if response.ok && dsh_broker::answered_for(&input, &response.result) => {
if !record("allowed") {
return Err(self.withdrawn());
}
Ok(())
}
Ok(_) => {
record("refused");
Err(BrokerRefusal::Refused {
service: self.service,
message: "the permission gate refused this act".to_string(),
})
}
Err(refusal) => {
record(refusal.code());
Err(refusal)
}
}
}
fn withdrawn(&self) -> BrokerRefusal {
BrokerRefusal::nested(
self.service,
NestedFailure::Unavailable("the broker call was withdrawn".to_string()),
)
}
fn count(&self) -> Result<(), BrokerRefusal> {
if self.invocation.nested_calls.fetch_add(1, Ordering::SeqCst) >= MAX_CALLS_PER_INVOCATION {
return Err(BrokerRefusal::nested(
self.service,
@@ -331,6 +470,26 @@ impl BrokerContext {
},
));
}
Ok(())
}
pub async fn core_call(
&self,
name: &str,
input: Value,
) -> Result<ToolCallResponse, BrokerRefusal> {
if self.cancel.is_cancelled() {
return Err(self.withdrawn());
}
self.count()?;
self.call_counted(name, input).await
}
async fn call_counted(
&self,
name: &str,
input: Value,
) -> Result<ToolCallResponse, BrokerRefusal> {
self.invocation
.calls
.call(name.to_string(), input, Some(&self.cancel))
@@ -345,7 +504,7 @@ impl BrokerContext {
let tool = ToolContext::new(std::path::PathBuf::from("."));
let gate = NestedCallGate::admitting_for_test();
let calls = CodemodeInvoker::for_extension(
Vec::new(),
dsh_broker::BrokerActTool::all().to_vec(),
tool.clone(),
gate.clone(),
String::new(),
@@ -363,8 +522,11 @@ impl BrokerContext {
root_caller: None,
broker: None,
nested_calls: AtomicU32::new(0),
broker_acts: Mutex::new((0, Vec::new(), false)),
tickets: Vec::new(),
caller_tools: Vec::new(),
gates: dsh_broker::GATE_TOOLS.to_vec(),
act_slots: tokio::sync::Semaphore::new(dsh_broker::MAX_OPEN_ACTS_PER_INVOCATION),
activations: Mutex::new(HashMap::new()),
session_call_lock: tokio::sync::Mutex::new(()),
cancel: CancellationToken::new(),
@@ -511,12 +673,16 @@ impl CoreCalls {
.as_ref()
.map(|session| session.session_id.clone())
.filter(|id| !id.is_empty());
let root_agent = context.execution.owner_agent_id.is_none();
let broker_tickets: Vec<(DshService, Ticket)> = broker_session
.iter()
.flat_map(|session| {
DshService::ALL
DshService::EVERY
.into_iter()
.filter(|service| tier == HostTier::Plugin && broker_services.contains(service))
.filter(|service| {
broker_services.contains(service)
&& dsh_broker::mintable(tier, owner, *service, root_agent)
})
.map(move |service| {
(
service,
@@ -544,9 +710,19 @@ impl CoreCalls {
context: Arc::new(context),
}
});
let gates: Vec<&'static str> = dsh_broker::GATE_TOOLS
.into_iter()
.filter(|gate| {
specs.iter().any(|spec| {
spec.name().eq_ignore_ascii_case(gate)
&& dsh_broker::is_gate_spec(spec.as_ref())
})
})
.collect();
let caller_tools = specs
.iter()
.map(|spec| spec.name().to_string())
.filter(|name| !dsh_broker::is_gate_tool(name))
.filter(|name| {
!context
.disallowed_tools
@@ -566,6 +742,7 @@ impl CoreCalls {
root_caller,
broker,
nested_calls: AtomicU32::new(0),
broker_acts: Mutex::new((0, Vec::new(), false)),
tickets: std::iter::once(ticket.clone())
.chain(checkpoint_ticket.iter().cloned())
.chain(question_ticket.iter().cloned())
@@ -573,6 +750,8 @@ impl CoreCalls {
.chain(broker_tickets.iter().map(|(_, ticket)| ticket.clone()))
.collect(),
caller_tools,
gates,
act_slots: tokio::sync::Semaphore::new(dsh_broker::MAX_OPEN_ACTS_PER_INVOCATION),
activations: Mutex::new(HashMap::new()),
session_call_lock: tokio::sync::Mutex::new(()),
cancel: CancellationToken::new(),
@@ -1299,6 +1478,9 @@ impl CoreCalls {
.as_ref()
.map(|scope| (scope.session_id.clone(), Arc::clone(&scope.context)))
.ok_or_else(|| refused("dsh/broker requires an invocation in a session".into()))?;
if service.builtin_only() && tool.execution.owner_agent_id.is_some() {
return Err(BrokerRefusal::NotGranted(service).wire());
}
if !shared.broker_granted(&params.owner, service).await {
return Err(BrokerRefusal::NotGranted(service).wire());
}
@@ -1386,6 +1568,15 @@ impl CoreCalls {
METHOD,
&cx,
)?;
if dsh_broker::is_gate_tool(&params.name) {
return Err(refuse(
error_code::REFUSED,
format!(
"`{}` is the host broker's permission gate; only the broker reaches it",
params.name
),
));
}
if invocation.nested_calls.fetch_add(1, Ordering::SeqCst) >= MAX_CALLS_PER_INVOCATION {
return Err(refuse(
error_code::REFUSED,
@@ -1544,8 +1735,33 @@ impl InvocationGuard {
/// The receipts of the core calls made so far, bounded, for the tool's
/// result metadata; `None` when it made none.
pub fn receipts(&self) -> Option<Value> {
let receipts = self.invocation.calls.receipts_json(RECEIPTS_ATTACHED);
(receipts["total"].as_u64().unwrap_or(0) > 0).then_some(receipts)
self.snapshot(false)
}
pub fn seal_receipts(&self) -> Option<Value> {
self.snapshot(true)
}
fn snapshot(&self, seal: bool) -> Option<Value> {
let (total, acts) = {
let mut guard = self
.invocation
.broker_acts
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if seal {
guard.2 = true;
}
(guard.0, guard.1.clone())
};
if seal {
self.invocation.cancel.cancel();
}
let mut receipts = self.invocation.calls.receipts_json(RECEIPTS_ATTACHED);
if total > 0 {
receipts["broker_acts"] = json!({ "total": total, "acts": acts });
}
(receipts["total"].as_u64().unwrap_or(0) > 0 || total > 0).then_some(receipts)
}
}
@@ -1554,3 +1770,86 @@ impl Drop for InvocationGuard {
self.core_calls.end(&self.ticket);
}
}
#[cfg(test)]
mod seal_tests {
use super::*;
use crate::tools::codemode::{NestedCallRequest, NestedCallVerdict};
use crate::tools::spec::ToolResult;
#[tokio::test]
async fn a_successful_gate_answer_after_sealing_cannot_record_or_perform_an_act() {
let owner = OwnerRef {
plugin_id: "sealed-gate".into(),
generation: 1,
owner_token: "fixture".into(),
};
let caller = ExtensionCaller {
origin: "extension:sealed-gate".into(),
tool: "sealed_tool".into(),
scope: "ext:sealed-gate@hash".into(),
};
let context = ToolContext::new(std::path::PathBuf::from("."));
let (events, _events) = tokio::sync::mpsc::channel(8);
let (gate, mut requests) = NestedCallGate::new(None, events, Duration::from_secs(5));
let gate = gate.for_extension(caller.clone(), dsh_broker::BrokerActTool::all().to_vec());
let calls = Arc::new(CoreCalls::default());
let guard = calls
.begin(
HostTier::Plugin,
1,
&owner,
"call",
&caller,
&context,
&gate,
)
.expect("invocation");
let broker = BrokerContext {
invocation: Arc::clone(&guard.invocation),
tool: Arc::new(context),
service: DshService::Remote,
cancel: CancellationToken::new(),
};
assert!(broker.record(json!({"method": "before", "outcome": "read"})));
let performed = std::sync::atomic::AtomicBool::new(false);
let (result, sealed) = tokio::join!(
async {
broker
.act("write", BrokerEffect::Write, Some("late"))
.await?;
performed.store(true, Ordering::SeqCst);
Ok::<(), BrokerRefusal>(())
},
async {
let Some(NestedCallRequest::Tool(request)) = requests.recv().await else {
panic!("the act must reach its gate before sealing");
};
let sealed = guard.seal_receipts().expect("the earlier record survives");
assert!(!broker.cancel.is_cancelled());
assert!(
request
.reply
.send(NestedCallVerdict::Answered {
result: ToolResult::success(request.input.to_string()),
hook_context: None,
})
.is_ok(),
"the gate response is still delivered"
);
sealed
}
);
assert!(
result.is_err(),
"a successful late answer must be refused: {result:?}"
);
assert!(!performed.load(Ordering::SeqCst));
assert!(!broker.record(json!({"method": "late", "outcome": "allowed"})));
assert_eq!(sealed["broker_acts"]["total"], 1);
assert_eq!(
guard.receipts().unwrap()["broker_acts"],
sealed["broker_acts"]
);
}
}
File diff suppressed because it is too large. Load diff
+294 -2
View File
@@ -7,10 +7,15 @@ use serde_json::{Value, json};
use super::protocol::{DshService, RpcErrorWire, error_code};
use super::registry::OwnerRegistry;
use super::tier::{HOST_OWNER_PREFIX, HostTier};
use super::{
dsh_catalog, dsh_llm, dsh_network, dsh_remote, dsh_settings, dsh_terminal, dsh_workspace,
dsh_catalog, dsh_llm, dsh_network, dsh_plugins, dsh_remote, dsh_settings, dsh_terminal,
dsh_workspace,
};
use crate::tools::codemode::{NestedDecision, NestedFailure};
use crate::tools::spec::{
ApprovalRequirement, ToolCapability, ToolContext, ToolError, ToolResult, ToolSpec,
};
pub use super::core_call::BrokerContext;
@@ -149,8 +154,52 @@ pub const TABLE: &[(DshService, Arm)] = &[
(DshService::Catalog, dsh_catalog::serve as Arm),
(DshService::Terminal, dsh_terminal::serve as Arm),
(DshService::Remote, dsh_remote::serve as Arm),
(DshService::Plugins, dsh_plugins::serve as Arm),
];
pub const BUILTIN_SERVICES: &[(&str, &[DshService])] = &[("creator", &[DshService::Plugins])];
#[must_use]
pub fn builtin_services(owner_id: &str) -> &'static [DshService] {
owner_id
.strip_prefix(HOST_OWNER_PREFIX)
.and_then(|module| {
BUILTIN_SERVICES
.iter()
.find(|(id, _)| *id == module)
.map(|(_, services)| *services)
})
.unwrap_or(&[])
}
fn builtin_route(owner: &OwnerRef, service: DshService) -> bool {
owner.plugin_id.starts_with(HOST_OWNER_PREFIX) || service.builtin_only()
}
#[must_use]
pub fn builtin_granted(
registry: &OwnerRegistry,
owner: &OwnerRef,
service: DshService,
) -> Option<bool> {
builtin_route(owner, service).then(|| {
registry.tier_of(owner) == Some(HostTier::Builtin)
&& builtin_services(&owner.plugin_id).contains(&service)
})
}
#[must_use]
pub fn mintable(tier: HostTier, owner: &OwnerRef, service: DshService, root_agent: bool) -> bool {
if builtin_route(owner, service) || tier == HostTier::Builtin {
tier == HostTier::Builtin
&& owner.plugin_id.starts_with(HOST_OWNER_PREFIX)
&& builtin_services(&owner.plugin_id).contains(&service)
&& (root_agent || !service.builtin_only())
} else {
tier == HostTier::Plugin
}
}
#[cfg(test)]
thread_local! {
static TEST_GRANTS: std::cell::RefCell<Option<Vec<DshService>>> =
@@ -193,6 +242,9 @@ pub fn offered(service: DshService) -> bool {
#[must_use]
pub fn granted(registry: &OwnerRegistry, owner: &OwnerRef, service: DshService) -> bool {
if let Some(granted) = builtin_granted(registry, owner, service) {
return granted;
}
match test_grants() {
Some(granted) => granted.contains(&service),
None => offered(service) && registry.host_service_granted(owner, service),
@@ -201,7 +253,7 @@ pub fn granted(registry: &OwnerRegistry, owner: &OwnerRef, service: DshService)
#[must_use]
pub fn granted_services(registry: &OwnerRegistry, owner: &OwnerRef) -> Vec<DshService> {
DshService::ALL
DshService::EVERY
.into_iter()
.filter(|service| granted(registry, owner, *service))
.collect()
@@ -225,6 +277,7 @@ pub fn ticket_method(service: DshService) -> &'static str {
DshService::Catalog => "dsh/broker:catalog",
DshService::Terminal => "dsh/broker:terminal",
DshService::Remote => "dsh/broker:remote",
DshService::Plugins => "dsh/broker:plugins",
}
}
@@ -252,3 +305,242 @@ pub fn not_available(service: DshService) -> BrokerFuture {
service,
))))
}
pub const ACT_TOOL: &str = "dsh_broker_act";
pub const REACH_TOOL: &str = "dsh_broker_reach";
pub const GATE_TOOLS: [&str; 2] = [ACT_TOOL, REACH_TOOL];
pub const MAX_ACT_TARGET: usize = 256;
const MAX_OPEN_ACTS: usize = 256;
pub const MAX_OPEN_ACTS_PER_INVOCATION: usize = 4;
#[must_use]
pub fn is_gate_tool(name: &str) -> bool {
GATE_TOOLS
.iter()
.any(|tool| name.eq_ignore_ascii_case(tool))
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum BrokerEffect {
Read,
Write,
Reach,
}
impl BrokerEffect {
#[must_use]
pub fn name(self) -> &'static str {
match self {
Self::Read => "read",
Self::Write => "write",
Self::Reach => "reach",
}
}
#[must_use]
pub fn gate_tool(self) -> &'static str {
match self {
Self::Reach => REACH_TOOL,
Self::Read | Self::Write => ACT_TOOL,
}
}
fn of(input: &Value) -> Option<Self> {
serde_json::from_value(input.get("effect")?.clone()).ok()
}
}
static OPEN_ACTS: std::sync::Mutex<Vec<(usize, Value)>> = std::sync::Mutex::new(Vec::new());
fn open_acts() -> std::sync::MutexGuard<'static, Vec<(usize, Value)>> {
OPEN_ACTS
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
}
pub struct OpenAct(usize, Value);
impl OpenAct {
#[must_use]
pub fn open(input: &Value, holder: usize) -> Option<Self> {
let mut open = open_acts();
if open.len() >= MAX_OPEN_ACTS {
return None;
}
open.push((holder, input.clone()));
Some(Self(holder, input.clone()))
}
}
impl Drop for OpenAct {
fn drop(&mut self) {
let mut open = open_acts();
if let Some(at) = open
.iter()
.position(|(holder, act)| *holder == self.0 && *act == self.1)
{
open.swap_remove(at);
}
}
}
fn act_is_open(input: &Value) -> bool {
open_acts().iter().any(|(_, open)| open == input)
}
#[must_use]
pub fn answered_for(input: &Value, result: &Value) -> bool {
match result.get("content") {
Some(Value::String(text)) => {
serde_json::from_str::<Value>(text).is_ok_and(|answered| answered == *input)
}
Some(other) => other == input,
None => false,
}
}
#[must_use]
pub fn is_gate_spec(spec: &dyn ToolSpec) -> bool {
is_gate_tool(spec.name())
&& spec.registration_origin() == std::any::type_name::<BrokerActTool>()
}
#[must_use]
pub fn act_description(input: &Value) -> String {
let field = |key: &str| input.get(key).and_then(Value::as_str).unwrap_or("?");
let verb = match BrokerEffect::of(input) {
Some(BrokerEffect::Write) => "change something",
Some(BrokerEffect::Reach) => "reach outside this session",
_ => "read",
};
let target = input
.get("target")
.and_then(Value::as_str)
.map(|target| format!(" ({target})"))
.unwrap_or_default();
format!(
"plugin `{}` asks to {verb} through the {} service: `{}`{target}",
field("plugin"),
field("service"),
field("method"),
)
}
pub struct BrokerActTool {
reach: bool,
}
impl BrokerActTool {
pub const ACT: Self = Self { reach: false };
pub const REACH: Self = Self { reach: true };
#[must_use]
pub fn all() -> [std::sync::Arc<dyn ToolSpec>; 2] {
[
std::sync::Arc::new(Self::ACT),
std::sync::Arc::new(Self::REACH),
]
}
fn effects(&self) -> &'static [&'static str] {
if self.reach {
&["reach"]
} else {
&["read", "write"]
}
}
fn admits(&self, input: &Value) -> Result<(), ToolError> {
let fits = BrokerEffect::of(input)
.is_some_and(|effect| (effect == BrokerEffect::Reach) == self.reach);
if fits && act_is_open(input) {
Ok(())
} else {
Err(ToolError::permission_denied(format!(
"{} answers only for an act a plugin broker has open",
self.name()
)))
}
}
}
#[async_trait::async_trait]
impl ToolSpec for BrokerActTool {
fn name(&self) -> &str {
if self.reach { REACH_TOOL } else { ACT_TOOL }
}
fn description(&self) -> &str {
"The session's permission gate for one act a plugin asked the host broker for."
}
fn input_schema(&self) -> Value {
json!({
"type": "object",
"properties": {
"plugin": {"type": "string"},
"service": {"type": "string"},
"method": {"type": "string"},
"effect": {"type": "string", "enum": self.effects()},
"target": {"type": "string"},
},
"required": ["plugin", "service", "method", "effect"],
"additionalProperties": false,
})
}
fn capabilities(&self) -> Vec<ToolCapability> {
if self.reach {
vec![ToolCapability::Network]
} else {
Vec::new()
}
}
fn approval_requirement(&self) -> ApprovalRequirement {
ApprovalRequirement::Required
}
fn approval_requirement_for(&self, input: &Value) -> ApprovalRequirement {
match BrokerEffect::of(input) {
Some(BrokerEffect::Read) => ApprovalRequirement::Auto,
_ => ApprovalRequirement::Required,
}
}
fn is_read_only(&self) -> bool {
false
}
fn is_read_only_for(&self, input: &Value) -> bool {
BrokerEffect::of(input) == Some(BrokerEffect::Read)
}
fn model_visible(&self) -> bool {
false
}
fn prepare(
&self,
input: Value,
_context: &ToolContext,
) -> Result<crate::tools::spec::PreparedToolCall, ToolError> {
self.admits(&input)?;
Ok(crate::tools::spec::PreparedToolCall {
name: self.name().to_string(),
description: act_description(&input),
read_only: self.is_read_only_for(&input),
supports_parallel: false,
starts_detached: false,
approval: self.approval_requirement_for(&input),
resources: vec![crate::tools::spec::ResourceClaim::GlobalExclusive],
input,
})
}
async fn execute(&self, input: Value, _context: &ToolContext) -> Result<ToolResult, ToolError> {
self.admits(&input)?;
Ok(ToolResult::success(input.to_string()))
}
}
+335 -4
View File
@@ -1,6 +1,337 @@
use super::dsh_broker::{BrokerCall, BrokerFuture, not_available};
use super::protocol::DshService;
use std::sync::Arc;
pub(super) fn serve(_call: BrokerCall) -> BrokerFuture {
not_available(DshService::Catalog)
use codewhale_config::PluginAuthority;
use codewhale_plugins::PluginRegistry;
use codewhale_plugins::activation::{
PluginActivationCapability, PolicyScope, extension_host_policy_enabled,
};
use codewhale_plugins::install::dsh::{plan_npm_install, refusal_text};
use codewhale_plugins::providers::{PluginProviderDeclaration, validate_declarations};
use codewhale_plugins::recommend::{
PluginMatchSource, PluginNextStep, PluginTaskRecommendation, lookup_reviewable_plugin,
};
use codewhale_plugins::registry::verify_plugin_component_authority;
use serde::Deserialize;
use serde_json::{Value, json};
use super::dsh_broker::{
BrokerCall, BrokerEffect, BrokerFuture, BrokerMethods, BrokerRefusal, decode,
};
use super::protocol::DshService;
use crate::pricing::BalanceInfo;
#[derive(Deserialize)]
#[serde(tag = "method", content = "input", deny_unknown_fields)]
enum CatalogMethod {
#[serde(rename = "install.plan")]
InstallPlan(InstallRequest),
#[serde(rename = "routes.list")]
RoutesList(NoInput),
#[serde(rename = "routes.resolve")]
RoutesResolve(RouteRequest),
#[serde(rename = "balance")]
Balance(NoInput),
}
impl BrokerMethods for CatalogMethod {
const NAMES: &'static [&'static str] =
&["install.plan", "routes.list", "routes.resolve", "balance"];
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct NoInput {}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct InstallRequest {
package: String,
version: String,
#[serde(default)]
registry: Option<String>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct RouteRequest {
provider: String,
#[serde(default)]
model: Option<String>,
}
pub(super) struct AccountBalance {
pub available: bool,
pub reading: Option<BalanceInfo>,
}
type ApprovedRoutes = Vec<(String, PluginProviderDeclaration)>;
pub(super) fn serve(call: BrokerCall) -> BrokerFuture {
serve_with(call, None)
}
pub(super) fn serve_with(call: BrokerCall, account: Option<AccountBalance>) -> BrokerFuture {
Box::pin(async move {
let caller = call.owner.plugin_id.clone();
match decode::<CatalogMethod>(&call)? {
CatalogMethod::InstallPlan(request) => {
let plugins = plugins(&call)?;
call.context
.act("install.plan", BrokerEffect::Read, Some(&request.package))
.await?;
install_plan(&plugins, &caller, request)
}
CatalogMethod::RoutesList(NoInput {}) => {
let plugins = plugins(&call)?;
call.context
.act("routes.list", BrokerEffect::Read, None)
.await?;
let routes = approved_routes(plugins, &caller).await?;
let routes: Vec<Value> = routes
.iter()
.map(|(name, declaration)| route(name, declaration))
.collect();
Ok(json!({ "routes": routes }))
}
CatalogMethod::RoutesResolve(request) => {
let plugins = plugins(&call)?;
call.context
.act(
"routes.resolve",
BrokerEffect::Read,
Some(&request.provider),
)
.await?;
let routes = approved_routes(plugins, &caller).await?;
resolve_route(&caller, routes, request)
}
CatalogMethod::Balance(NoInput {}) => {
call.context
.act("balance", BrokerEffect::Read, None)
.await?;
Ok(balance(account))
}
}
})
}
fn plugins(call: &BrokerCall) -> Result<Arc<PluginRegistry>, BrokerRefusal> {
call.context
.tool()
.plugin_registry
.clone()
.ok_or(BrokerRefusal::NotAvailable(DshService::Catalog))
}
fn refused(message: impl Into<String>) -> BrokerRefusal {
BrokerRefusal::Refused {
service: DshService::Catalog,
message: message.into(),
}
}
fn review_command(name: &str, next_step: PluginNextStep) -> String {
PluginTaskRecommendation {
name: name.to_string(),
source: PluginMatchSource::Installed {
id: name.to_string(),
},
matched_terms: Vec::new(),
score: 0,
next_step,
}
.command()
}
fn install_plan(
plugins: &PluginRegistry,
caller: &str,
request: InstallRequest,
) -> Result<Value, BrokerRefusal> {
let plan = plan_npm_install(
&request.package,
&request.version,
request.registry.as_deref(),
)
.map_err(|reason| {
tracing::info!(target: "codewhale::dsh_catalog", plugin = %caller, "install plan refused");
refused(reason)
})?;
let existing = plugins.get(&plan.plugin_name);
let (status, command, message) = match existing {
Some(existing) => {
let command = lookup_reviewable_plugin(existing.name(), plugins, &[])
.map(|reviewable| reviewable.command())
.unwrap_or_else(|| review_command(existing.name(), PluginNextStep::Inspect));
let message = format!(
"A plugin named `{}` is already present; nothing was installed. Review it with `{command}`. A market listing does not establish that it is the same bundle.",
existing.name()
);
("already_present", command, message)
}
None => {
let command = review_command(
&plan.plugin_name,
PluginNextStep::SourceInstall {
spec: plan.spec.clone(),
},
);
let message = format!(
"Codewhale installs a plugin only through its reviewed install flow; nothing was installed. Review and install {}@{} with `{command}`.",
request.package, request.version
);
("review_required", command, message)
}
};
tracing::info!(
target: "codewhale::dsh_catalog",
plugin = %caller,
package = %request.package,
version = %request.version,
status,
"install planned for review; nothing installed"
);
let mut answer = json!({
"status": status,
"package": request.package,
"version": request.version,
"plugin": plan.plugin_name,
"command": command,
"message": message,
});
if existing.is_none() {
answer["spec"] = Value::String(plan.spec);
}
Ok(answer)
}
fn declared_routes(
plugins: &PluginRegistry,
caller: &str,
) -> Result<Option<(ApprovedRoutes, PluginAuthority)>, String> {
let Some(plugin) = plugins.get(caller).filter(|plugin| plugin.active()) else {
return Ok(None);
};
if plugin.manifest.providers.is_empty() {
return Ok(None);
}
validate_declarations(&plugin.manifest.providers)?;
let authority = plugins
.authority_for(caller)
.ok_or("the plugin has no review receipt")?;
let routes = plugin
.manifest
.providers
.iter()
.map(|(name, declaration)| (name.clone(), declaration.clone()))
.collect();
Ok(Some((routes, authority)))
}
async fn approved_routes(
plugins: Arc<PluginRegistry>,
caller: &str,
) -> Result<ApprovedRoutes, BrokerRefusal> {
let receipt = |reason: String| {
tracing::info!(
target: "codewhale::dsh_catalog",
plugin = %caller,
reason = %reason,
"provider routes refused: the review receipt does not hold"
);
refused(
"the review that approved this plugin's provider routes no longer holds; review the plugin again in Codewhale",
)
};
let Some((routes, authority)) = declared_routes(&plugins, caller).map_err(receipt)? else {
return Ok(Vec::new());
};
let policy = extension_host_policy_enabled();
#[cfg(any(test, feature = "test-support"))]
let env_scope = crate::test_support::env_scope_ticket();
let verified = tokio::task::spawn_blocking(move || {
#[cfg(any(test, feature = "test-support"))]
let _env_scope = crate::test_support::join_env_scope(env_scope);
let _policy = PolicyScope::propagate(policy);
verify_plugin_component_authority(&authority, PluginActivationCapability::Providers)
})
.await;
match verified {
Ok(Ok(())) => Ok(routes),
Ok(Err(reason)) => Err(receipt(reason)),
Err(error) => Err(receipt(error.to_string())),
}
}
fn resolve_route(
caller: &str,
routes: ApprovedRoutes,
request: RouteRequest,
) -> Result<Value, BrokerRefusal> {
let Some((name, declaration)) = routes
.into_iter()
.find(|(name, _)| *name == request.provider)
else {
tracing::info!(
target: "codewhale::dsh_catalog",
plugin = %caller,
"provider route refused: not approved for this plugin"
);
return Err(refused(format!(
"`{}` is not a provider route approved for this plugin. A plugin cannot register a model adapter in Codewhale: a model route is a provider route you approve in the plugin's review",
refusal_text(&request.provider)
)));
};
let model = match request.model {
None => declaration.model.clone(),
Some(model) => {
let offered = if declaration.models.is_empty() {
declaration
.model
.as_deref()
.is_none_or(|default| default == model)
} else {
declaration.models.contains(&model)
};
if !offered {
return Err(refused(format!(
"model `{}` is not on the approved provider route `{name}`",
refusal_text(&model)
)));
}
Some(model)
}
};
Ok(json!({ "route": route(&name, &declaration), "model": model }))
}
fn route(name: &str, declaration: &PluginProviderDeclaration) -> Value {
json!({
"provider": name,
"base_url": declaration.base_url,
"default_model": declaration.model,
"models": declaration.models,
})
}
fn balance(account: Option<AccountBalance>) -> Value {
let Some(account) = account else {
return json!({ "status": "unknown" });
};
if !account.available {
return json!({ "status": "absent" });
}
match account
.reading
.filter(|reading| !reading.total_balance.trim().is_empty())
{
Some(reading) => json!({
"status": "known",
"currency": reading.currency,
"total": reading.total_balance,
"topped_up": reading.topped_up_balance,
"granted": reading.granted_balance,
}),
None => json!({ "status": "unknown" }),
}
}
+97 -16
View File
@@ -6,10 +6,13 @@ use codewhale_secrets::Secrets;
use serde::Deserialize;
use serde_json::{Value, json};
use super::dsh_broker::{BrokerCall, BrokerFuture, BrokerMethods, BrokerRefusal, decode};
use super::dsh_broker::{
BrokerCall, BrokerEffect, BrokerFuture, BrokerMethods, BrokerRefusal, decode,
};
use super::protocol::DshService;
use crate::tools::spec::ToolContext;
use crate::tools::web::guard::validate_fetch_target;
use codewhale_net::guard::PrivateTargets;
pub const CREDENTIAL_HANDLE_PREFIX: &str = "codewhale-credential:";
const MAX_REF_LEN: usize = 128;
@@ -90,6 +93,8 @@ pub(super) struct NetworkAuthority {
pub hosts: BTreeSet<String>,
pub tool: Option<ToolContext>,
pub vault: Option<Secrets>,
#[cfg(test)]
pub test_dns_pin: Option<(String, std::net::IpAddr)>,
}
impl NetworkAuthority {
@@ -98,6 +103,8 @@ impl NetworkAuthority {
hosts: BTreeSet::new(),
tool: None,
vault: None,
#[cfg(test)]
test_dns_pin: None,
}
}
}
@@ -115,11 +122,19 @@ pub(super) fn serve_with(call: BrokerCall, authority: NetworkAuthority) -> Broke
let Some(tool) = authority.tool.as_ref() else {
return Err(BrokerRefusal::NotAvailable(DshService::Network));
};
let host = reqwest::Url::parse(&input.url)
.ok()
.and_then(|url| url.host_str().map(normalized_host));
call.context
.act("fetch", BrokerEffect::Reach, host.as_deref())
.await?;
fetch(
&plugin,
&authority.hosts,
tool,
authority.vault.as_ref(),
#[cfg(test)]
authority.test_dns_pin.as_ref(),
input,
)
.await
@@ -127,6 +142,9 @@ pub(super) fn serve_with(call: BrokerCall, authority: NetworkAuthority) -> Broke
NetworkMethod::Describe(CredentialRef { name }) => {
let vault = vault(&authority)?;
let name = checked_ref(&name)?;
call.context
.act("credentials.describe", BrokerEffect::Read, Some(name))
.await?;
let set = read_secret(vault, &plugin, name)?.is_some();
tracing::info!(target: "codewhale::dsh_network", plugin = %plugin, credential = %name, set, "credential described");
Ok(json!({ "ref": name, "set": set, "handle": handle(name) }))
@@ -134,6 +152,9 @@ pub(super) fn serve_with(call: BrokerCall, authority: NetworkAuthority) -> Broke
NetworkMethod::Resolve(CredentialRef { name }) => {
let vault = vault(&authority)?;
let name = checked_ref(&name)?;
call.context
.act("credentials.resolve", BrokerEffect::Read, Some(name))
.await?;
let set = read_secret(vault, &plugin, name)?.is_some();
tracing::info!(target: "codewhale::dsh_network", plugin = %plugin, credential = %name, set, "credential resolved to its handle");
Ok(json!({ "ref": name, "value": set.then(|| handle(name)) }))
@@ -146,6 +167,9 @@ pub(super) fn serve_with(call: BrokerCall, authority: NetworkAuthority) -> Broke
"a credential value must be a non-empty secret, not a handle",
));
}
call.context
.act("credentials.set", BrokerEffect::Write, Some(name))
.await?;
vault
.set(&vault_slot(&plugin, name), &value)
.map_err(|_| invalid("the credential vault refused the write"))?;
@@ -315,6 +339,7 @@ async fn fetch(
hosts: &BTreeSet<String>,
tool: &ToolContext,
vault: Option<&Secrets>,
#[cfg(test)] test_dns_pin: Option<&(String, std::net::IpAddr)>,
input: FetchInput,
) -> Result<Value, BrokerRefusal> {
let mut url = reqwest::Url::parse(&input.url).map_err(|_| invalid("the URL is not valid"))?;
@@ -376,9 +401,26 @@ async fn fetch(
let mut substituted: Option<Substituted> = None;
let response = loop {
egress_host(&url, hosts)?;
let pin = validate_fetch_target(&url, tool, AUDIT_LABEL)
.await
.map_err(|error| invalid(format!("egress refused: {error}")))?;
let pin = {
#[cfg(test)]
if let Some((pinned_host, pinned_ip)) = test_dns_pin
&& url
.host_str()
.is_some_and(|candidate| candidate.eq_ignore_ascii_case(pinned_host))
{
crate::tools::web::guard::validate_network_policy(&host, tool, AUDIT_LABEL)
.map_err(|error| invalid(format!("egress refused: {error}")))?;
Some((pinned_host.clone(), *pinned_ip))
} else {
validate_fetch_target(&url, tool, AUDIT_LABEL, PrivateTargets::Refuse)
.await
.map_err(|error| invalid(format!("egress refused: {error}")))?
}
#[cfg(not(test))]
validate_fetch_target(&url, tool, AUDIT_LABEL, PrivateTargets::Refuse)
.await
.map_err(|error| invalid(format!("egress refused: {error}")))?
};
let substitution = match substituted.take() {
Some(done) => done,
None => substitute_headers(plugin, vault, &input.headers)?,
@@ -539,6 +581,22 @@ mod tests {
hosts: hosts.iter().map(|h| (*h).to_string()).collect(),
tool: Some(tool),
vault: Some(vault.clone()),
test_dns_pin: None,
}
}
fn authority_with_pin(
hosts: &[&str],
tool: ToolContext,
vault: &Secrets,
pinned_host: &str,
) -> NetworkAuthority {
NetworkAuthority {
test_dns_pin: Some((
pinned_host.to_string(),
std::net::IpAddr::V4(std::net::Ipv4Addr::LOCALHOST),
)),
..authority(hosts, tool, vault)
}
}
@@ -648,11 +706,16 @@ mod tests {
"a",
"fetch",
json!({
"url": format!("{}/v1/models", server.uri()),
"url": format!("http://api.example.com:{}/v1/models", server.address().port()),
"headers": [["authorization", "Bearer codewhale-credential:KEY"]],
}),
),
authority(&["127.0.0.1"], policy(&["127.0.0.1"], &[]), &vault),
authority_with_pin(
&["api.example.com"],
policy(&["api.example.com"], &[]),
&vault,
"api.example.com",
),
)
.await
.unwrap();
@@ -708,6 +771,16 @@ mod tests {
)
.await;
assert!(message(&unapproved).contains("egress refused"));
let local_allowed = serve_with(
call("a", "fetch", json!({"url": url})),
authority(&["127.0.0.1"], policy(&["127.0.0.1"], &[]), &vault),
)
.await;
assert!(
message(&local_allowed).contains("restricted"),
"{}",
message(&local_allowed)
);
let metadata = serve_with(
call(
"a",
@@ -798,16 +871,18 @@ mod tests {
let vault = vault();
vault.set(&vault_slot("a", "KEY"), SECRET).unwrap();
let headers = json!([["authorization", "codewhale-credential:KEY"]]);
let both = authority(
&["127.0.0.1", "localhost"],
policy(&["127.0.0.1", "localhost"], &[]),
let base = format!("http://api.example.com:{}", server.address().port());
let both = authority_with_pin(
&["api.example.com", "localhost"],
policy(&["api.example.com", "localhost"], &[]),
&vault,
"api.example.com",
);
let away = serve_with(
call(
"a",
"fetch",
json!({"url": format!("{}/away", server.uri()), "headers": headers}),
json!({"url": format!("{base}/away"), "headers": headers}),
),
both,
)
@@ -817,12 +892,13 @@ mod tests {
call(
"a",
"fetch",
json!({"url": format!("{}/rename", server.uri()), "headers": headers}),
json!({"url": format!("{base}/rename"), "headers": headers}),
),
authority(
&["127.0.0.1", "localhost"],
policy(&["127.0.0.1", "localhost"], &[]),
authority_with_pin(
&["api.example.com", "localhost"],
policy(&["api.example.com", "localhost"], &[]),
&vault,
"api.example.com",
),
)
.await;
@@ -831,9 +907,14 @@ mod tests {
call(
"a",
"fetch",
json!({"url": format!("{}/hop", server.uri()), "headers": headers}),
json!({"url": format!("{base}/hop"), "headers": headers}),
),
authority_with_pin(
&["api.example.com"],
policy(&["api.example.com"], &[]),
&vault,
"api.example.com",
),
authority(&["127.0.0.1"], policy(&["127.0.0.1"], &[]), &vault),
)
.await
.unwrap();
@@ -0,0 +1,85 @@
use serde::Deserialize;
use serde_json::{Map, Value};
use super::dsh_broker::{
BrokerCall, BrokerFuture, BrokerMethods, BrokerRefusal, builtin_services, decode,
};
use super::protocol::DshService;
use crate::tools::spec::ToolContext;
const LOG: &str = "codewhale::dsh_plugins";
const MAX_INPUT_FIELDS: usize = 32;
#[derive(Deserialize)]
#[serde(tag = "method", content = "input", deny_unknown_fields)]
enum PluginsMethod {
#[serde(rename = "list")]
List(Map<String, Value>),
#[serde(rename = "status")]
Status(Map<String, Value>),
#[serde(rename = "posture")]
Posture(Map<String, Value>),
#[serde(rename = "install")]
Install(Map<String, Value>),
#[serde(rename = "remove")]
Remove(Map<String, Value>),
#[serde(rename = "enable")]
Enable(Map<String, Value>),
#[serde(rename = "disable")]
Disable(Map<String, Value>),
}
impl BrokerMethods for PluginsMethod {
const NAMES: &'static [&'static str] = &[
"list", "status", "posture", "install", "remove", "enable", "disable",
];
}
pub(super) fn serve(call: BrokerCall) -> BrokerFuture {
Box::pin(async move {
let method = decode::<PluginsMethod>(&call)?;
if !builtin_services(&call.owner.plugin_id).contains(&DshService::Plugins) {
tracing::info!(target: LOG, plugin = %call.owner.plugin_id, "plugins refused: not the creator module");
return Err(refused());
}
let tool = live(&call);
if tool.execution.owner_agent_id.is_some() {
tracing::info!(target: LOG, "plugins refused: a sub-agent caller");
return Err(refused());
}
if method.input().len() > MAX_INPUT_FIELDS {
return Err(BrokerRefusal::InvalidInput {
service: DshService::Plugins,
message: format!("a plugins input has at most {MAX_INPUT_FIELDS} fields"),
});
}
Err(BrokerRefusal::NotAvailable(DshService::Plugins))
})
}
impl PluginsMethod {
fn input(&self) -> &Map<String, Value> {
match self {
Self::List(input)
| Self::Status(input)
| Self::Posture(input)
| Self::Install(input)
| Self::Remove(input)
| Self::Enable(input)
| Self::Disable(input) => input,
}
}
}
fn live(call: &BrokerCall) -> ToolContext {
let mut tool = call.context.tool().clone();
let _ = tool.refresh_live_posture();
tool
}
fn refused() -> BrokerRefusal {
BrokerRefusal::Refused {
service: DshService::Plugins,
message: "the plugins service is not available to this caller".to_string(),
}
}
+13 -2
View File
@@ -6,7 +6,9 @@ use serde::Deserialize;
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
use super::dsh_broker::{BrokerCall, BrokerFuture, BrokerMethods, BrokerRefusal, decode};
use super::dsh_broker::{
BrokerCall, BrokerEffect, BrokerFuture, BrokerMethods, BrokerRefusal, decode,
};
use super::protocol::DshService;
pub const PAIRING_TTL: Duration = Duration::from_secs(10 * 60);
@@ -142,7 +144,16 @@ pub fn remote_grants() -> &'static RemoteGrants {
}
pub(super) fn serve(call: BrokerCall) -> BrokerFuture {
Box::pin(std::future::ready(remote_grants().serve(&call)))
Box::pin(async move {
let (method, effect) = match decode::<RemoteMethod>(&call)? {
RemoteMethod::Status {} => ("status", BrokerEffect::Read),
RemoteMethod::Pair { .. } => ("pair", BrokerEffect::Reach),
RemoteMethod::Clients {} => ("clients", BrokerEffect::Read),
RemoteMethod::Revoke { .. } => ("revoke", BrokerEffect::Write),
};
call.context.act(method, effect, None).await?;
remote_grants().serve(&call)
})
}
impl RemoteGrants {
File diff suppressed because it is too large. Load diff
@@ -5,7 +5,9 @@ use codewhale_workflow_js::ToolCallResponse;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use super::dsh_broker::{BrokerCall, BrokerFuture, BrokerMethods, BrokerRefusal, decode};
use super::dsh_broker::{
BrokerCall, BrokerEffect, BrokerFuture, BrokerMethods, BrokerRefusal, decode,
};
use super::protocol::DshService;
use crate::tools::execution_envelope::enforce_tool_denial;
@@ -121,6 +123,7 @@ pub(super) fn serve(call: BrokerCall) -> BrokerFuture {
match method {
TerminalMethod::List(NoInput {}) => {
let live = live(&call, "terminal/wait").await?;
call.context.act("list", BrokerEffect::Read, None).await?;
tracing::info!(target: LOG, plugin = %call.owner.plugin_id, terminals = live.terminals.len(), "terminals listed");
Ok(json!({
"instance": live.instance,
+16
View File
@@ -126,6 +126,7 @@ pub mod dsh_broker;
mod dsh_catalog;
mod dsh_llm;
mod dsh_network;
mod dsh_plugins;
mod dsh_remote;
pub use dsh_remote::{RemoteClient, RemoteDenial, RemoteGrants, RemotePairing, remote_grants};
mod dsh_settings;
@@ -1933,6 +1934,11 @@ impl ExtensionHostManager {
if tools.is_empty() {
return Vec::new();
}
for gate in dsh_broker::BrokerActTool::all() {
if tool_registry.get(gate.name()).is_none() {
tool_registry.register(gate);
}
}
let taken: HashSet<String> = tool_registry
.names()
.into_iter()
@@ -3629,6 +3635,9 @@ impl ManagerShared {
})
}
pub(super) fn broker_services(&self, owner: &OwnerRef) -> Vec<protocol::DshService> {
if let Some(granted) = dsh_broker::test_grants() {
return granted;
}
dsh_broker::granted_services(&self.registry.lock().expect("registry lock"), owner)
}
@@ -3637,6 +3646,13 @@ impl ManagerShared {
owner: &OwnerRef,
service: protocol::DshService,
) -> bool {
if let Some(granted) = dsh_broker::builtin_granted(
&self.registry.lock().expect("registry lock"),
owner,
service,
) {
return granted;
}
if let Some(granted) = dsh_broker::test_grants() {
return granted.contains(&service);
}
@@ -38,7 +38,7 @@ pub const MAX_PLUGIN_CONFIG_BYTES: usize = 16 * 1024;
/// Deepest nesting of tables and arrays accepted.
const MAX_CONFIG_DEPTH: usize = 16;
/// How the `toml` crate serializes a date-time (a one-key table).
const TOML_DATETIME_KEY: &str = "$__toml_private_datetime";
pub(super) const TOML_DATETIME_KEY: &str = "$__toml_private_datetime";
/// The config one activation is given, and its digest.
#[derive(Debug, Clone, PartialEq)]
@@ -89,7 +89,7 @@ fn has_datetime(value: &Value) -> bool {
}
/// One plugin's `[plugins."<name>".config]`, checked.
fn convert(name: &str, settings: &PluginSettings) -> Result<PluginConfig, String> {
pub(super) fn convert(name: &str, settings: &PluginSettings) -> Result<PluginConfig, String> {
let Some(table) = &settings.config else {
return Ok(PluginConfig::empty());
};
+23 -1
View File
@@ -540,6 +540,7 @@ pub enum DshService {
Catalog,
Terminal,
Remote,
Plugins,
}
impl DshService {
@@ -553,6 +554,19 @@ impl DshService {
Self::Remote,
];
pub const BUILTIN_ONLY: [Self; 1] = [Self::Plugins];
pub const EVERY: [Self; 8] = [
Self::Workspace,
Self::Llm,
Self::Network,
Self::Settings,
Self::Catalog,
Self::Terminal,
Self::Remote,
Self::Plugins,
];
#[must_use]
pub fn name(self) -> &'static str {
match self {
@@ -563,12 +577,20 @@ impl DshService {
Self::Catalog => "catalog",
Self::Terminal => "terminal",
Self::Remote => "remote",
Self::Plugins => "plugins",
}
}
#[must_use]
pub fn parse(name: &str) -> Option<Self> {
Self::ALL.into_iter().find(|service| service.name() == name)
Self::EVERY
.into_iter()
.find(|service| service.name() == name)
}
#[must_use]
pub fn builtin_only(self) -> bool {
Self::BUILTIN_ONLY.contains(&self)
}
}
@@ -196,7 +196,7 @@ fn render() -> String {
let _ = writeln!(out, " {name}: {code},");
}
out.push_str("} as const\n\nexport type Direction = 'core_to_host' | 'host_to_core'\n");
let services: Vec<String> = DshService::ALL
let services: Vec<String> = DshService::EVERY
.iter()
.map(|service| quote(service.name()))
.collect();
@@ -676,7 +676,7 @@ fn dsh_broker_is_one_strict_envelope_and_names_only_the_closed_service_list() {
unknown["params"]["service"] = json!("vault");
assert!(parse_host_message(unknown, tier).is_ok());
}
let names: Vec<&str> = DshService::ALL
let names: Vec<&str> = DshService::EVERY
.iter()
.map(|service| service.name())
.collect();
@@ -689,10 +689,17 @@ fn dsh_broker_is_one_strict_envelope_and_names_only_the_closed_service_list() {
"settings",
"catalog",
"terminal",
"remote"
"remote",
"plugins"
]
);
for service in DshService::ALL {
assert_eq!(DshService::BUILTIN_ONLY, [DshService::Plugins]);
assert!(
DshService::ALL
.iter()
.all(|service| !service.builtin_only())
);
for service in DshService::EVERY {
assert_eq!(DshService::parse(service.name()), Some(service));
}
assert_eq!(DshService::parse("Workspace"), None);
@@ -51,6 +51,8 @@ const RESERVED_NAMES: &[&str] = &[
"js_execution",
"request_user_input",
"multi_tool_use.parallel",
"dsh_broker_act",
"dsh_broker_reach",
];
#[derive(Debug, Clone, PartialEq, Eq)]
+53 -19
View File
@@ -226,6 +226,23 @@ fn map_call_error(tool: &str, error: HostCallError) -> ToolError {
}
}
pub(super) fn failed_with_acts(
error: ToolError,
receipts: Option<&Value>,
) -> Result<ToolResultWire, ToolError> {
let holds_acts = receipts.is_some_and(|receipts| receipts.get("broker_acts").is_some());
if !holds_acts {
return Err(error);
}
Ok(ToolResultWire {
content: vec![ContentBlockWire::Text {
text: error.to_string(),
}],
is_error: true,
structured: None,
})
}
pub fn wire_to_result(wire: ToolResultWire, origin: &str) -> ToolResult {
let content = wire
.content
@@ -396,26 +413,44 @@ impl ToolSpec for HostToolSpec {
});
// The deadline stops while one of this call's `core/call`s waits on
// the gate (an approval card), so a person's time is not the tool's.
let value = host
let outcome = host
.call_with_clock(
request,
Some(registration.owner.plugin_id.clone()),
invocation.as_ref().map(InvocationGuard::clock),
)
.await
.map_err(|error| map_call_error(&registration.name, error))?;
self.check_caller(context)?;
self.manager
.live_host_for(registration)
.await
.map_err(ToolError::not_available)?;
self.check_caller(context)?;
let wire: ToolResultWire = serde_json::from_value(value).map_err(|error| {
ToolError::execution_failed(format!(
"extension tool `{}` returned a malformed result: {error}",
registration.name
))
})?;
.await;
let receipts = invocation.as_ref().and_then(InvocationGuard::seal_receipts);
let settled: Result<ToolResultWire, ToolError> = async {
match outcome {
Ok(value) => {
self.check_caller(context)?;
self.manager
.live_host_for(registration)
.await
.map_err(ToolError::not_available)?;
self.check_caller(context)?;
serde_json::from_value(value).map_err(|error| {
ToolError::execution_failed(format!(
"extension tool `{}` returned a malformed result: {error}",
registration.name
))
})
}
Err(error) => Err(map_call_error(&registration.name, error)),
}
}
.await;
let wire = match settled {
Ok(wire) => wire,
Err(error) => {
let wire = failed_with_acts(error, receipts.as_ref())?;
match self.check_caller(context) {
Ok(()) => wire,
Err(stale) => failed_with_acts(stale, receipts.as_ref())?,
}
}
};
let mut result = wire_to_result(wire, &self.origin());
if let Some(Value::Object(metadata)) = result.metadata.as_mut() {
metadata.insert(
@@ -433,10 +468,9 @@ impl ToolSpec for HostToolSpec {
);
}
// What the tool asked the core to run, so the persisted record shows it.
if let (Some(receipts), Some(Value::Object(metadata))) = (
invocation.as_ref().and_then(InvocationGuard::receipts),
result.metadata.as_mut(),
) {
if let (Some(receipts), Some(Value::Object(metadata))) =
(receipts, result.metadata.as_mut())
{
metadata.insert("core_calls".to_string(), receipts);
}
Ok(result)
+1 -3
View File
@@ -18,8 +18,6 @@
//! never render a child session, which is what keeps the orchestrator light at
//! high fanout.
#![allow(dead_code)]
use anyhow::Result;
use codewhale_config::FleetExecConfig;
use codewhale_protocol::fleet::{
@@ -529,7 +527,7 @@ fn parse_exec_terminal_route(value: &serde_json::Value) -> ParsedTerminalRoute {
route.map_or(ParsedTerminalRoute::Invalid, ParsedTerminalRoute::Valid)
}
#[cfg(any(test, feature = "test-support"))]
#[cfg(test)]
fn map_exec_terminal_route(line: &str) -> Option<FleetWorkerReportedRoute> {
let value: serde_json::Value = serde_json::from_str(line).ok()?;
match parse_exec_terminal_route(&value) {
+3 -5
View File
@@ -4,8 +4,6 @@
//! observe work through this trait without knowing whether the worker is a
//! local child process or an SSH-backed remote command.
#![allow(dead_code)]
use std::collections::{BTreeMap, BTreeSet};
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
@@ -218,7 +216,7 @@ struct LocalWorkerProcess {
#[cfg(unix)]
session_id: libc::pid_t,
#[cfg(unix)]
parent_death_writer: Option<std::io::PipeWriter>,
_parent_death_writer: Option<std::io::PipeWriter>,
#[cfg(windows)]
windows_job: FleetWindowsJob,
host_kind: FleetHostKind,
@@ -387,7 +385,7 @@ impl LocalProcessFleetHostAdapter {
#[cfg(unix)]
session_id: pid as libc::pid_t,
#[cfg(unix)]
parent_death_writer,
_parent_death_writer: parent_death_writer,
#[cfg(windows)]
windows_job,
host_kind,
@@ -1520,7 +1518,7 @@ fn signal_unix_session(
Ok(errors)
}
#[cfg(unix)]
#[cfg(all(unix, test))]
fn unix_pid_is_running(pid: libc::pid_t) -> bool {
if !unix_pid_exists(pid) {
return false;
-2
View File
@@ -5,8 +5,6 @@
//! Artifacts are referenced by bounded metadata; large payloads live on disk
//! and are never embedded in the ledger.
#![allow(dead_code)]
use std::collections::BTreeMap;
use std::collections::HashMap;
#[cfg(test)]
+1 -2
View File
@@ -4,8 +4,6 @@
//! foreground and coordinate logical local workers while later host adapters
//! add real process and SSH execution behind the same records.
#![allow(dead_code)]
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
use std::time::Duration;
@@ -2360,6 +2358,7 @@ impl FleetManager {
}
}
#[cfg(test)]
fn append_worker_event(
&self,
run_id: &FleetRunId,
-3
View File
@@ -1,7 +1,5 @@
//! Fleet profile vocabulary, local profile discovery, and config-facing aliases.
#![allow(dead_code)]
use std::collections::BTreeSet;
use std::path::{Path, PathBuf};
@@ -10,7 +8,6 @@ use serde::{Deserialize, Serialize};
use crate::reasoning_preference::ReasoningEffort;
#[allow(unused_imports)]
pub use codewhale_config::{
FleetDelegationHints, FleetLoadout, FleetProfile, FleetProfilePermissions, FleetRole, FleetSlot,
};
+16 -5
View File
@@ -1,14 +1,16 @@
//! Fleet scheduler policy: leases, heartbeats, backpressure, and recovery.
#![allow(dead_code)]
#[cfg(test)]
use std::collections::{BTreeMap, BTreeSet};
use std::path::Path;
use std::time::Duration;
use anyhow::{Context, Result, anyhow};
#[cfg(test)]
use anyhow::anyhow;
use anyhow::{Context, Result};
use chrono::{DateTime, SecondsFormat, Utc};
use codewhale_protocol::fleet::*;
#[cfg(test)]
use serde_json::Value;
use super::ledger::{FleetLedger, FleetLedgerState, FleetTaskLedgerStatus, FleetTaskState};
@@ -65,7 +67,7 @@ impl FleetScheduler {
self.now = now;
}
#[cfg(any(test, feature = "test-support"))]
#[cfg(test)]
pub fn tick_run(&self, run_id: &FleetRunId) -> Result<FleetSchedulerReport> {
let mut report = FleetSchedulerReport::default();
self.recover_unhealthy_work(run_id, &mut report, &mut |_, _, _, _| Ok(()))?;
@@ -83,7 +85,7 @@ impl FleetScheduler {
/// and idempotent to call on a fresh process: a task re-leased by an
/// earlier resume is no longer stale at the same instant, and a terminally
/// failed task is never failed or escalated twice.
#[cfg(any(test, feature = "test-support"))]
#[cfg(test)]
pub fn resume_run(&self, run_id: &FleetRunId) -> Result<FleetSchedulerReport> {
self.resume_run_with_restart_callback(run_id, |_, _, _, _| Ok(()))
}
@@ -134,6 +136,7 @@ impl FleetScheduler {
Ok(report)
}
#[cfg(test)]
fn recover_unhealthy_work(
&self,
run_id: &FleetRunId,
@@ -394,6 +397,7 @@ impl FleetScheduler {
Ok(())
}
#[cfg(test)]
fn launch_queued_work(
&self,
run_id: &FleetRunId,
@@ -450,6 +454,7 @@ impl FleetScheduler {
}
}
#[cfg(test)]
fn next_launch(
&self,
run: &FleetRun,
@@ -594,6 +599,7 @@ impl FleetScheduler {
.update_run_status(run_id, status, &self.timestamp())
}
#[cfg(test)]
fn append_worker_event(
&self,
run_id: &FleetRunId,
@@ -615,12 +621,14 @@ impl FleetScheduler {
}
}
#[cfg(test)]
#[derive(Debug, Default)]
struct ActiveCounts {
by_host: BTreeMap<String, usize>,
by_task_class: BTreeMap<String, usize>,
}
#[cfg(test)]
fn active_counts(state: &FleetLedgerState, run: &FleetRun) -> ActiveCounts {
let mut counts = ActiveCounts::default();
for task in active_tasks_for_run(state, &run.id) {
@@ -646,6 +654,7 @@ fn active_counts(state: &FleetLedgerState, run: &FleetRun) -> ActiveCounts {
counts
}
#[cfg(test)]
fn active_tasks_for_run(state: &FleetLedgerState, run_id: &FleetRunId) -> Vec<FleetTaskState> {
state
.tasks
@@ -667,6 +676,7 @@ fn task_spec_for(state: &FleetLedgerState, task: &FleetTaskState) -> Option<Flee
.cloned()
}
#[cfg(test)]
fn host_key(worker: &FleetWorkerSpec) -> String {
match &worker.host {
FleetHostSpec::Local => "local".to_string(),
@@ -676,6 +686,7 @@ fn host_key(worker: &FleetWorkerSpec) -> String {
}
}
#[cfg(test)]
fn task_class(task: &FleetTaskSpec) -> String {
task.metadata
.get("class")
@@ -11,8 +11,6 @@
//! - Mailbox events stream into the Fleet ledger as `FleetWorkerEventPayload`
//! - `FleetWorkerInspection` reads both ledger state and sub-agent worker records
#![allow(dead_code)]
use std::borrow::Cow;
use std::path::PathBuf;
+30 -3
View File
@@ -25,6 +25,7 @@
pub mod rate_limit;
use anyhow::Result;
use codewhale_command_contract::metrics::format_duration;
use codewhale_models::{MessageRequest, MessageResponse, StreamEvent};
use codewhale_runtime::config::{RateLimitRetryPolicy, RetryPolicy};
use serde_json::Value;
@@ -457,6 +458,18 @@ pub enum LlmError {
Other(String),
}
fn opencode_go_hint(message: &str) -> &'static str {
let lowered = message.to_lowercase();
if lowered.contains("codewhale auth set") {
return "";
}
if lowered.contains("opencode go") || lowered.contains("opencode.ai/go") {
" OpenCode Go is a separate provider with its own key; run: codewhale auth set --provider opencode-go"
} else {
""
}
}
impl std::fmt::Display for LlmError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
@@ -464,11 +477,25 @@ impl std::fmt::Display for LlmError {
message,
retry_after,
..
} => write!(
} => {
let hint = opencode_go_hint(message);
match retry_after {
Some(d) => write!(
f,
"Rate limit exceeded: {message}; quota window resets in {}; automatic retry stopped{hint}",
format_duration(*d)
),
None => write!(
f,
"Rate limit exceeded: {message}; automatic retry stopped{hint}"
),
}
}
LlmError::RateLimited { message, .. } => write!(
f,
"Rate limit exceeded: {message}; quota window reset in {retry_after:?}; automatic retry stopped"
"Rate limit exceeded: {message}{}",
opencode_go_hint(message)
),
LlmError::RateLimited { message, .. } => write!(f, "Rate limit exceeded: {message}"),
LlmError::QuotaExhausted(error) => {
write!(f, "Provider plan quota exhausted: {}", error.message)
}
+70
View File
@@ -119,6 +119,76 @@ fn retryability_distinguishes_transient_failures_from_durable_failures() {
}
}
#[test]
fn quota_window_display_prints_reset_time_as_plain_text() {
let windowed = LlmError::QuotaWindow {
message: "Free usage limit reached".into(),
retry_after: Some(Duration::from_secs(70168)),
transmission: RequestTransmission::Rejected,
}
.to_string();
assert!(windowed.contains("resets in 19h29m"), "{windowed}");
assert!(windowed.contains("automatic retry stopped"), "{windowed}");
let open_ended = LlmError::QuotaWindow {
message: "Free usage limit reached".into(),
retry_after: None,
transmission: RequestTransmission::Rejected,
}
.to_string();
assert!(!open_ended.contains("reset"), "{open_ended}");
assert!(
open_ended.contains("automatic retry stopped"),
"{open_ended}"
);
for text in [&windowed, &open_ended] {
assert!(!text.contains("Some("), "{text}");
assert!(!text.contains("None"), "{text}");
}
let hinted = LlmError::QuotaWindow {
message: "OpenCode Go free usage limit reached".into(),
retry_after: Some(Duration::from_secs(70168)),
transmission: RequestTransmission::Rejected,
}
.to_string();
assert_eq!(
hinted
.matches("codewhale auth set --provider opencode-go")
.count(),
1,
"{hinted}"
);
let rate_limited = LlmError::RateLimited {
message: "OpenCode Go free usage limit reached".into(),
retry_after: None,
}
.to_string();
assert_eq!(
rate_limited
.matches("codewhale auth set --provider opencode-go")
.count(),
1,
"{rate_limited}"
);
let already_hinted = LlmError::QuotaWindow {
message: hinted.clone(),
retry_after: Some(Duration::from_secs(70168)),
transmission: RequestTransmission::Rejected,
}
.to_string();
assert_eq!(
already_hinted
.matches("codewhale auth set --provider opencode-go")
.count(),
1,
"{already_hinted}"
);
}
#[test]
fn http_response_boundary_classifies_status_contract() {
for status in [408, 409, 499, 500, 502, 503, 504] {
-17
View File
@@ -3736,23 +3736,6 @@ impl McpPool {
Ok(pool)
}
/// Construct a source-aware empty pool after the initial config load
/// failed. Keeping the source paths means a later edit or explicit
/// `/mcp reload` can recover in-process instead of pinning the session to
/// an ad-hoc pool that has no files to re-read.
pub fn empty_with_workspace_config_sources(
path: &std::path::Path,
workspace: &Path,
plugins: Arc<codewhale_plugins::PluginRegistry>,
) -> Result<Self> {
Self::empty_with_workspace_config_sources_and_auth_context(
path,
workspace,
plugins,
process_mcp_auth_context(),
)
}
pub fn empty_with_workspace_config_sources_and_auth_context(
path: &std::path::Path,
workspace: &Path,
+1 -68
View File
@@ -404,30 +404,6 @@ pub fn persist_decisions(
})
}
/// Filter candidates that still need a user decision for this content hash.
#[allow(dead_code)] // used by future selector UI + unit tests
pub fn candidates_needing_consent(
candidates: &[ImportCandidate],
store: &ImportConsentStore,
) -> Vec<ImportCandidate> {
candidates
.iter()
.filter(|c| {
let key = consent_key(&c.source_path, &c.content_hash);
match store.entries.get(&key) {
Some(entry) if entry.decision == ImportDecision::Decline => false,
Some(entry) if entry.decision == ImportDecision::Approve => {
// Re-prompt only when the specific server was not part of
// the prior approval list (partial approval).
!entry.servers.iter().any(|s| s == &c.name)
}
_ => true,
}
})
.cloned()
.collect()
}
/// Apply approvals: returns servers to merge into user mcp.json.
/// Hard-blocked candidates are never returned even if decision is Approve.
#[cfg(any(test, feature = "test-support"))]
@@ -1093,7 +1069,7 @@ mod tests {
}
#[test]
fn unrelated_source_rewrites_keep_review_tokens_and_declines() {
fn unrelated_source_rewrites_keep_review_tokens() {
with_import_context(|context| {
let source = write_claude_json(
&context.home,
@@ -1116,23 +1092,6 @@ mod tests {
)
.unwrap();
assert!(receipt.imported);
let candidates = discover_from_json_file(&source, ExternalMcpSourceKind::ClaudeJson);
let mut store = ImportConsentStore::default();
let decisions = HashMap::from([("y".to_string(), ImportDecision::Decline)]);
record_decisions(&mut store, &candidates, &decisions, 1);
fs::write(
&source,
r#"{"numStartups":3,"mcpServers":{"x":{"command":"echo"},"y":{"command":"echo"}}}"#,
)
.unwrap();
let refreshed = discover_from_json_file(&source, ExternalMcpSourceKind::ClaudeJson);
let needing: Vec<_> = candidates_needing_consent(&refreshed, &store)
.into_iter()
.map(|c| c.name)
.collect();
// `y` stays declined; `x` was never decided in this store.
assert_eq!(needing, ["x"]);
});
}
@@ -1208,32 +1167,6 @@ mod tests {
assert!(!config.servers["ok"].is_enabled());
}
#[test]
fn declined_consent_skips_reprompt_until_hash_changes() {
let dir = tempdir().unwrap();
let path = write_claude_json(
dir.path(),
r#"{"mcpServers":{"x":{"command":"echo","enabled":true}}}"#,
);
let candidates = discover_from_json_file(&path, ExternalMcpSourceKind::ClaudeJson);
let mut store = ImportConsentStore::default();
let mut decisions = HashMap::new();
decisions.insert("x".into(), ImportDecision::Decline);
record_decisions(&mut store, &candidates, &decisions, 1);
let needing = candidates_needing_consent(&candidates, &store);
assert!(needing.is_empty(), "declined should not re-prompt");
// Content change → new hash → re-prompt.
fs::write(
&path,
r#"{"mcpServers":{"x":{"command":"echo","args":["changed"],"enabled":true}}}"#,
)
.unwrap();
let refreshed = discover_from_json_file(&path, ExternalMcpSourceKind::ClaudeJson);
let needing = candidates_needing_consent(&refreshed, &store);
assert_eq!(needing.len(), 1);
}
#[test]
fn provenance_display_includes_source_and_hash() {
let dir = tempdir().unwrap();
-17
View File
@@ -1607,23 +1607,6 @@ pub fn server_supports_oauth_login(server: &McpServerConfig) -> bool {
&& !server_has_manual_authorization(server)
}
/// Whether the shared token store already holds a usable credential for this
/// server, with its binding revalidated against discovery. Used by the
/// authenticate tool's already-authorized branch.
pub async fn has_usable_stored_tokens(
name: &str,
server: &McpServerConfig,
network_policy: Option<&NetworkPolicyDecider>,
) -> Result<bool> {
has_usable_stored_tokens_with_auth_context(
name,
server,
network_policy,
super::process_mcp_auth_context(),
)
.await
}
pub async fn has_usable_stored_tokens_with_auth_context(
name: &str,
server: &McpServerConfig,
+23 -7
View File
@@ -244,22 +244,38 @@ pub async fn capture(
if let Some(snapshot) = supplied {
return snapshot.render().map(Some);
}
// Unit tests never read an operator's account credentials. Transport tests
// exercise load_from with an explicitly isolated store instead.
#[cfg(not(any(test, feature = "test-support")))]
if let Some(snapshot) = load(profile).await? {
return snapshot.render().map(Some);
}
#[cfg(any(test, feature = "test-support"))]
{
let _ = profile;
if let Some(message) = TEST_ACCOUNT_LOAD_FAILURE.get() {
anyhow::bail!(message);
}
if !account_api_base_is_loopback() {
return Ok(crate::prompts::load_user_constitution_block());
}
}
if let Some(snapshot) = load(profile).await? {
return snapshot.render().map(Some);
}
Ok(crate::prompts::load_user_constitution_block())
}
#[cfg(any(test, feature = "test-support"))]
fn account_api_base_is_loopback() -> bool {
std::env::var(codewhale_secrets::account::ACCOUNT_API_BASE_ENV)
.ok()
.and_then(|value| normalize_runtime_account_api_base(&value))
.and_then(|base| reqwest::Url::parse(&base).ok())
.and_then(|url| url.host_str().map(str::to_owned))
.is_some_and(|host| {
host.eq_ignore_ascii_case("localhost")
|| host
.trim_start_matches('[')
.trim_end_matches(']')
.parse::<std::net::IpAddr>()
.is_ok_and(|address| address.is_loopback())
})
}
#[cfg(any(test, feature = "test-support"))]
thread_local! {
static TEST_ACCOUNT_LOAD_FAILURE: std::cell::Cell<Option<&'static str>> =
-1
View File
@@ -1,4 +1,3 @@
#![allow(dead_code)]
//! System prompt composition.
//!
//! Prompts are assembled from composable layers loaded at compile time from
+1 -1
View File
@@ -544,7 +544,7 @@ pub enum ResolvedProviderReadiness {
impl ResolvedProviderReadiness {
pub fn label(&self) -> Cow<'static, str> {
match self {
Self::MissingKey => Cow::Borrowed("missing key"),
Self::MissingKey => Cow::Borrowed("needs key"),
Self::MissingLogin => Cow::Borrowed("missing login"),
Self::ExternalConsentPendingSelection => {
Cow::Borrowed("external consent · select to check")
+1 -1
View File
@@ -1715,7 +1715,7 @@ fn number(value: Option<&Value>, keys: &[&str]) -> Option<i64> {
}
fn redact(text: &str) -> String {
codewhale_secrets::redact::redact_secrets(text)
codewhale_sanitize::redact::redact_secrets(text)
}
use codewhale_commands::diagnostics_reports::receipts::{bounded, plural};
+112 -11
View File
@@ -401,24 +401,84 @@ pub fn resolve_model(
)?
.is_some_and(|entry| entry.fetched_at > 0)
{
return Err(
codewhale_agent::ModelResolutionError::ModelNotAvailableForProvider {
provider: identity.provider,
requested: model.unwrap_or("auto").to_string(),
}
.into(),
);
return Err(model_not_available(
identity.provider,
model.unwrap_or("auto"),
));
}
if !crate::provider_catalog_live::provider_owns_live_catalog(identity.provider, base_url) {
codewhale_agent::ModelRegistry::default().resolve(
model.filter(|model| *model != "auto"),
Some(identity.provider),
)?;
admit_reviewed_selection(identity.provider, model.filter(|model| *model != "auto"))?;
}
}
Ok(route?.candidate)
}
fn model_not_available(provider: ProviderKind, requested: &str) -> anyhow::Error {
anyhow::anyhow!(
"model '{requested}' is not available from provider '{}'",
provider.as_str()
)
}
fn admit_reviewed_selection(provider: ProviderKind, requested: Option<&str>) -> Result<()> {
let selections = &codewhale_config::catalog::reviewed::bundled_reviewed().selections;
let rows = || {
selections
.iter()
.filter(move |row| ProviderKind::parse_config_identity(&row.provider) == Some(provider))
};
let normalize = |value: &str| value.trim().to_ascii_lowercase();
let row_matches = |row: &codewhale_config::catalog::reviewed::ModelSelection, wanted: &str| {
let wanted = normalize(wanted);
normalize(&row.id) == wanted || row.aliases.iter().any(|alias| normalize(alias) == wanted)
};
let default_model = provider.provider().default_model();
let requested = requested.filter(|name| !name.trim().is_empty());
let Some(name) = requested else {
if rows().any(|row| row_matches(row, default_model)) {
return Ok(());
}
if !default_model.trim().is_empty() && provider != ProviderKind::Ollama {
return Ok(());
}
if rows().next().is_none() {
anyhow::bail!(
"provider '{}' has no registered default model",
provider.as_str()
);
}
anyhow::bail!(
"provider '{}' declares default model '{default_model}', but that model is not registered for the provider",
provider.as_str()
);
};
let trimmed = name.trim();
let admitted = match provider {
ProviderKind::Ollama | ProviderKind::OllamaCloud => true,
ProviderKind::OpencodeGo => codewhale_config::opencode_go_model_id(name)
.is_some_and(|canonical| rows().any(|row| row.id.eq_ignore_ascii_case(canonical))),
_ => rows().any(|row| row_matches(row, name)),
} || match provider {
ProviderKind::Atlascloud => !trimmed.is_empty() && trimmed.contains('/'),
ProviderKind::Arcee => !trimmed.is_empty(),
ProviderKind::XiaomiMimo => !trimmed.is_empty() && !trimmed.chars().any(char::is_control),
_ => false,
} || (!default_model.trim().is_empty()
&& trimmed.eq_ignore_ascii_case(default_model.trim()));
if admitted {
return Ok(());
}
if rows().next().is_none() {
anyhow::bail!(
"provider '{}' has no registered model for '{name}'",
provider.as_str()
);
}
Err(model_not_available(provider, name))
}
/// Save one explicit route preference after atomically adopting legacy choices.
pub fn set(path: &Path, key: &str, value: &str) -> Result<()> {
persistence::mutate_config_document(path, |doc| set_document(path, doc, key, value))
@@ -1082,4 +1142,45 @@ model = "Other-X"
assert_eq!(loaded.default_model(), selected_route(&path)?.1);
Ok(())
}
#[test]
fn reviewed_admission_keeps_passthrough_and_default_contracts() {
for (provider, model) in [
(ProviderKind::Atlascloud, Some("Qwen/Qwen3-Coder")),
(ProviderKind::Arcee, Some("trinity-future-thinking")),
(ProviderKind::XiaomiMimo, Some("mimo-custom-id")),
(ProviderKind::Ollama, Some("qwen2.5-coder:7b")),
(ProviderKind::OpencodeGo, Some("deepseek-v4-pro")),
(ProviderKind::OllamaCloud, None),
(ProviderKind::Deepseek, None),
] {
admit_reviewed_selection(provider, model)
.unwrap_or_else(|error| panic!("{provider:?} {model:?}: {error}"));
}
for kind in ProviderKind::all() {
let default = kind.provider().default_model();
if default.trim().is_empty() {
continue;
}
admit_reviewed_selection(*kind, Some(default))
.unwrap_or_else(|error| panic!("default of {kind:?}: {error}"));
}
}
#[test]
fn reviewed_admission_fails_closed_without_borrowing_another_provider() {
let error = admit_reviewed_selection(ProviderKind::Atlascloud, Some("not-in-atlas"))
.expect_err("unknown model must not become the provider default");
assert_eq!(
error.to_string(),
"model 'not-in-atlas' is not available from provider 'atlascloud'"
);
let error = admit_reviewed_selection(ProviderKind::Ollama, None)
.expect_err("the placeholder default must not resolve");
assert!(
error.to_string().contains("default model 'unknown'"),
"{error}"
);
assert!(admit_reviewed_selection(ProviderKind::Moonshot, Some("glm-5.2")).is_err());
}
}
+2 -2
View File
@@ -11,7 +11,7 @@
//! - the provider enum and the non-secret configured route key,
//! - the canonical wire model id,
//! - a **normalized, redacted** endpoint identity (URL userinfo and sensitive
//! query values masked by [`codewhale_secrets::sanitize::redact_url_for_display`]),
//! query values masked by [`codewhale_sanitize::sanitize::redact_url_for_display`]),
//! - a one-way credential *generation* digest that is never rendered.
//!
//! The credential generation is deliberately taken over the endpoint **and** the
@@ -47,7 +47,7 @@ pub fn endpoint_identity(base_url: &str) -> String {
if reqwest::Url::parse(trimmed).is_err() {
return OPAQUE_ENDPOINT.to_string();
}
codewhale_secrets::sanitize::redact_url_for_display(trimmed)
codewhale_sanitize::sanitize::redact_url_for_display(trimmed)
.trim_end_matches('/')
.to_string()
}
+775 -2
View File
@@ -1835,6 +1835,7 @@ pub(super) enum RuntimeTurnInputSource {
Scheduled {
task_id: String,
},
AsyncCompletionWake,
}
impl RuntimeTurnInputSource {
@@ -1850,7 +1851,7 @@ impl RuntimeTurnInputSource {
Self::GoalContinuation {
continuation_index, ..
} if *continuation_index > 0 => crate::core::ops::UserInputProvenance::GoalContinuation,
Self::GoalContinuation { .. } | Self::Scheduled { .. } => {
Self::GoalContinuation { .. } | Self::Scheduled { .. } | Self::AsyncCompletionWake => {
crate::core::ops::UserInputProvenance::Runtime
}
}
@@ -1863,7 +1864,9 @@ impl RuntimeTurnInputSource {
| Self::ExtensionCommand { .. }
| Self::WatchEvent(_) => None,
Self::AgentMail { message_id, .. } => Some(message_id),
Self::GoalContinuation { .. } | Self::Scheduled { .. } => None,
Self::GoalContinuation { .. } | Self::Scheduled { .. } | Self::AsyncCompletionWake => {
None
}
}
}
@@ -1874,6 +1877,7 @@ impl RuntimeTurnInputSource {
| Self::Review(_)
| Self::ExtensionCommand { .. }
| Self::Scheduled { .. } => Some(prompt.to_string()),
Self::AsyncCompletionWake => Some("[runtime] Background work finished".to_string()),
// The provider projection contains runtime-only framing. Persist
// the bounded canonical mail summary instead, so history and app
// clients never mistake that projection for typed user input.
@@ -1938,6 +1942,9 @@ impl RuntimeTurnInputSource {
"input_provenance": "scheduled",
"task_id": task_id,
})),
Self::AsyncCompletionWake => Some(json!({
"input_provenance": "async_completion_wake",
})),
}
}
}
@@ -6599,6 +6606,695 @@ impl RuntimeThreadManager {
self.track_receipt_worker(worker);
}
const ASYNC_COMPLETION_WAKE_DEBOUNCE: std::time::Duration =
std::time::Duration::from_millis(75);
const IDLE_SHELL_WAKE_POLL_INTERVAL: std::time::Duration =
std::time::Duration::from_millis(150);
const ASYNC_COMPLETION_WAKE_PROMPT: &'static str = "[runtime] Background work finished";
pub(super) fn parse_resumed_subagent_completion_count(message: &str) -> Option<usize> {
let rest = message.strip_prefix("Resuming turn with ")?;
if !rest.contains("sub-agent completion(s)") {
return None;
}
rest.split_whitespace().next()?.parse::<usize>().ok()
}
fn async_wake_arm_op(
is_shell: bool,
manager_key: usize,
owner: Option<std::sync::Weak<Mutex<ActiveThreads>>>,
thread_id: &str,
arm: bool,
) -> bool {
type WakeOwnerMap = HashMap<(usize, String), std::sync::Weak<Mutex<ActiveThreads>>>;
static ARMED_WAKES: std::sync::OnceLock<parking_lot::Mutex<WakeOwnerMap>> =
std::sync::OnceLock::new();
static ARMED_SHELL_MONITORS: std::sync::OnceLock<parking_lot::Mutex<WakeOwnerMap>> =
std::sync::OnceLock::new();
let table = if is_shell {
ARMED_SHELL_MONITORS.get_or_init(|| parking_lot::Mutex::new(HashMap::new()))
} else {
ARMED_WAKES.get_or_init(|| parking_lot::Mutex::new(HashMap::new()))
};
let mut guard = table.lock();
guard.retain(|_, existing| existing.upgrade().is_some());
let key = (manager_key, thread_id.to_string());
if arm {
let Some(owner) = owner else {
return false;
};
if guard.contains_key(&key) {
return false;
}
guard.insert(key, owner);
true
} else {
guard.remove(&key).is_some()
}
}
fn async_completion_wake_arm(&self, thread_id: &str) -> bool {
Self::async_wake_arm_op(
false,
Arc::as_ptr(&self.active) as usize,
Some(Arc::downgrade(&self.active)),
thread_id,
true,
)
}
fn async_completion_wake_release_key(manager_key: usize, thread_id: &str) {
let _ = Self::async_wake_arm_op(false, manager_key, None, thread_id, false);
}
fn async_shell_monitor_arm(&self, thread_id: &str) -> bool {
Self::async_wake_arm_op(
true,
Arc::as_ptr(&self.active) as usize,
Some(Arc::downgrade(&self.active)),
thread_id,
true,
)
}
fn async_shell_monitor_release_key(manager_key: usize, thread_id: &str) {
let _ = Self::async_wake_arm_op(true, manager_key, None, thread_id, false);
}
fn async_subagent_wake_op(
&self,
thread_id: &str,
op: u8,
agent_id: Option<&str>,
flag: bool,
count: usize,
ids: &[String],
) -> (bool, Vec<String>, HashSet<String>) {
#[derive(Default)]
struct SubAgentWakeEntry {
owner: std::sync::Weak<Mutex<ActiveThreads>>,
pending: Vec<String>,
late_after_turn_complete: Vec<String>,
delivered: HashSet<String>,
active_turn_ids: Vec<String>,
active_turn_drained: usize,
saw_turn_complete: bool,
}
static STATES: std::sync::OnceLock<
parking_lot::Mutex<HashMap<(usize, String), SubAgentWakeEntry>>,
> = std::sync::OnceLock::new();
let table = STATES.get_or_init(|| parking_lot::Mutex::new(HashMap::new()));
let manager_key = Arc::as_ptr(&self.active) as usize;
let mut guard = table.lock();
guard.retain(|_, entry| entry.owner.upgrade().is_some());
let entry = guard
.entry((manager_key, thread_id.to_string()))
.or_insert_with(|| SubAgentWakeEntry {
owner: Arc::downgrade(&self.active),
..Default::default()
});
entry.owner = Arc::downgrade(&self.active);
match op {
0 => {
entry.active_turn_ids.clear();
for pending_id in entry.pending.drain(..) {
if !entry.delivered.contains(&pending_id)
&& !entry.active_turn_ids.contains(&pending_id)
{
entry.active_turn_ids.push(pending_id);
}
}
for late in entry.late_after_turn_complete.drain(..) {
if !entry.delivered.contains(&late) && !entry.active_turn_ids.contains(&late) {
entry.active_turn_ids.push(late);
}
}
entry.active_turn_drained = 0;
entry.saw_turn_complete = false;
(false, Vec::new(), HashSet::new())
}
1 => {
entry.saw_turn_complete = true;
(false, Vec::new(), HashSet::new())
}
2 => {
entry.active_turn_drained = entry.active_turn_drained.saturating_add(count);
(false, Vec::new(), HashSet::new())
}
3 => {
let Some(id) = agent_id else {
return (false, Vec::new(), HashSet::new());
};
if entry.delivered.contains(id) {
return (false, Vec::new(), HashSet::new());
}
if flag {
if !entry.pending.iter().any(|existing| existing == id) {
entry.pending.push(id.to_string());
}
(true, Vec::new(), HashSet::new())
} else if entry.saw_turn_complete {
if !entry
.late_after_turn_complete
.iter()
.any(|existing| existing == id)
{
entry.late_after_turn_complete.push(id.to_string());
}
(false, Vec::new(), HashSet::new())
} else {
if !entry.active_turn_ids.iter().any(|existing| existing == id) {
entry.active_turn_ids.push(id.to_string());
}
(false, Vec::new(), HashSet::new())
}
}
4 => {
let mut remaining_drain = entry.active_turn_drained;
let active_ids = std::mem::take(&mut entry.active_turn_ids);
for id in active_ids {
if entry.delivered.contains(&id) {
continue;
}
if remaining_drain > 0 {
remaining_drain -= 1;
entry.delivered.insert(id);
} else if !entry.pending.iter().any(|existing| existing == &id) {
entry.pending.push(id);
}
}
for late in entry.late_after_turn_complete.drain(..) {
if !entry.delivered.contains(&late)
&& !entry.pending.iter().any(|existing| existing == &late)
{
entry.pending.push(late);
}
}
entry.active_turn_drained = 0;
entry.saw_turn_complete = false;
(!entry.pending.is_empty(), Vec::new(), HashSet::new())
}
5 => (false, Vec::new(), entry.delivered.clone()),
6 => (false, entry.pending.clone(), entry.delivered.clone()),
7 => {
let keep: HashSet<&str> = ids.iter().map(String::as_str).collect();
entry.pending.retain(|id| keep.contains(id.as_str()));
(false, Vec::new(), HashSet::new())
}
8 => {
let mark: HashSet<&str> = ids.iter().map(String::as_str).collect();
entry.pending.retain(|id| !mark.contains(id.as_str()));
for id in ids {
entry.delivered.insert(id.clone());
}
(false, Vec::new(), HashSet::new())
}
_ => (false, Vec::new(), HashSet::new()),
}
}
pub(super) fn async_wake_on_turn_started(&self, thread_id: &str) {
let _ = self.async_subagent_wake_op(thread_id, 0, None, false, 0, &[]);
}
pub(super) fn async_wake_on_turn_complete(&self, thread_id: &str) {
let _ = self.async_subagent_wake_op(thread_id, 1, None, false, 0, &[]);
}
pub(super) fn async_wake_on_resumed_status(&self, thread_id: &str, count: usize) {
if count > 0 {
let _ = self.async_subagent_wake_op(thread_id, 2, None, false, count, &[]);
}
}
fn async_wake_on_agent_complete(
&self,
thread_id: &str,
agent_id: &str,
idle_now: bool,
) -> bool {
self.async_subagent_wake_op(thread_id, 3, Some(agent_id), idle_now, 0, &[])
.0
}
fn async_wake_settle_turn(&self, thread_id: &str, turn_completed: bool) -> bool {
self.async_subagent_wake_op(thread_id, 4, None, turn_completed, 0, &[])
.0
}
fn async_wake_delivered_ids(&self, thread_id: &str) -> HashSet<String> {
self.async_subagent_wake_op(thread_id, 5, None, false, 0, &[])
.2
}
fn async_wake_pending_snapshot(&self, thread_id: &str) -> (Vec<String>, HashSet<String>) {
let (_, pending, delivered) =
self.async_subagent_wake_op(thread_id, 6, None, false, 0, &[]);
(pending, delivered)
}
fn async_wake_prune_pending(&self, thread_id: &str, keep_ids: &[String]) {
let _ = self.async_subagent_wake_op(thread_id, 7, None, false, 0, keep_ids);
}
fn async_wake_mark_delivered(&self, thread_id: &str, ids: &[String]) {
if !ids.is_empty() {
let _ = self.async_subagent_wake_op(thread_id, 8, None, false, 0, ids);
}
}
pub(super) fn weak_wake_manager(
&self,
) -> Arc<dyn Fn() -> Option<RuntimeThreadManager> + Send + Sync> {
let host_profile = self.host_profile;
let config = self.config.clone();
let workspace = self.workspace.clone();
let plugin_registry = self.plugin_registry.clone();
let store = self.store.clone();
let weak_process_owner_lock = Arc::downgrade(&self._process_owner_lock);
let config_admission = self.config_admission.clone();
let engine_load = self.engine_load.clone();
let shutdown_drain = self.shutdown_drain.clone();
let engine_workers = self.engine_workers.clone();
let subagent_manager = self.subagent_manager.clone();
let turn_monitors = self.turn_monitors.clone();
let event_monitors = self.event_monitors.clone();
let event_reader_wake = self.event_reader_wake.clone();
let event_pump_stop = self.event_pump_stop.clone();
let weak_event_pump_owner = Arc::downgrade(&self._event_pump_owner);
let weak_active = Arc::downgrade(&self.active);
let event_emit = self.event_emit.clone();
let projection_locks = self.projection_locks.clone();
let event_tx = self.event_tx.clone();
let manager_cfg = self.manager_cfg.clone();
let runtime_mcp_servers = self.runtime_mcp_servers.clone();
let cancel_token = self.cancel_token.clone();
let task_manager = self.task_manager.clone();
let task_execution_lease = self.task_execution_lease.clone();
let automations = self.automations.clone();
let pending_approvals = self.pending_approvals.clone();
let grants = self.grants.clone();
let pending_user_inputs = self.pending_user_inputs.clone();
let pending_dynamic_tools = self.pending_dynamic_tools.clone();
let pending_commands = self.pending_commands.clone();
let recovery_receipts = self.recovery_receipts.clone();
let notices = self.notices.clone();
let recovery_flush = self.recovery_flush.clone();
let hook_base = self.hook_base.clone();
#[cfg(any(test, feature = "test-support"))]
let snapshot_test_hook = self.snapshot_test_hook.clone();
#[cfg(any(test, feature = "test-support"))]
let replay_test_hook = self.replay_test_hook.clone();
#[cfg(any(test, feature = "test-support"))]
let test_model_client = self.test_model_client.clone();
#[cfg(any(test, feature = "test-support"))]
let test_approval_decision_timeout_ms = self.test_approval_decision_timeout_ms.clone();
#[cfg(any(test, feature = "test-support"))]
let test_dynamic_tool_result_timeout_ms = self.test_dynamic_tool_result_timeout_ms.clone();
Arc::new(move || {
if cancel_token.is_cancelled() || event_pump_stop.is_cancelled() {
return None;
}
let process_owner_lock = weak_process_owner_lock.upgrade()?;
let event_pump_owner = weak_event_pump_owner.upgrade()?;
let active = weak_active.upgrade()?;
Some(RuntimeThreadManager {
host_profile,
config: config.clone(),
workspace: workspace.clone(),
plugin_registry: plugin_registry.clone(),
store: store.clone(),
_process_owner_lock: process_owner_lock,
config_admission: config_admission.clone(),
engine_load: engine_load.clone(),
shutdown_drain: shutdown_drain.clone(),
engine_workers: engine_workers.clone(),
subagent_manager: subagent_manager.clone(),
turn_monitors: turn_monitors.clone(),
event_monitors: event_monitors.clone(),
event_reader_wake: event_reader_wake.clone(),
event_pump_stop: event_pump_stop.clone(),
_event_pump_owner: event_pump_owner,
active,
event_emit: event_emit.clone(),
projection_locks: projection_locks.clone(),
event_tx: event_tx.clone(),
manager_cfg: manager_cfg.clone(),
runtime_mcp_servers: runtime_mcp_servers.clone(),
cancel_token: cancel_token.clone(),
task_manager: task_manager.clone(),
task_execution_lease: task_execution_lease.clone(),
automations: automations.clone(),
pending_approvals: pending_approvals.clone(),
grants: grants.clone(),
pending_user_inputs: pending_user_inputs.clone(),
pending_dynamic_tools: pending_dynamic_tools.clone(),
pending_commands: pending_commands.clone(),
recovery_receipts: recovery_receipts.clone(),
notices: notices.clone(),
recovery_flush: recovery_flush.clone(),
hook_base: hook_base.clone(),
#[cfg(any(test, feature = "test-support"))]
snapshot_test_hook: snapshot_test_hook.clone(),
#[cfg(any(test, feature = "test-support"))]
replay_test_hook: replay_test_hook.clone(),
#[cfg(any(test, feature = "test-support"))]
test_model_client: test_model_client.clone(),
#[cfg(any(test, feature = "test-support"))]
test_approval_decision_timeout_ms: test_approval_decision_timeout_ms.clone(),
#[cfg(any(test, feature = "test-support"))]
test_dynamic_tool_result_timeout_ms: test_dynamic_tool_result_timeout_ms.clone(),
})
})
}
pub(super) async fn handle_agent_complete_async_wake(
&self,
thread_id: &str,
agent_id: &str,
subagent_manager: &SharedSubAgentManager,
active_weak: &std::sync::Weak<Mutex<ActiveThreads>>,
identity: &std::sync::Weak<AsyncRwLock<mpsc::Receiver<EngineEvent>>>,
) {
if self.cancel_token.is_cancelled()
|| self.event_pump_stop.is_cancelled()
|| self.is_acp_host()
{
return;
}
let delivered = self.async_wake_delivered_ids(thread_id);
let eligible = {
let guard = subagent_manager.read().await;
guard
.terminal_results_excluding_for_session(thread_id, &delivered)
.into_iter()
.any(|result| result.agent_id == agent_id)
};
if !eligible {
return;
}
let Some(active) = active_weak.upgrade() else {
return;
};
let (owns_receiver, engine_active, turn_is_active) = {
let guard = active.lock().await;
guard
.engines
.get(thread_id)
.map(|state| {
let owns_receiver = identity
.upgrade()
.is_some_and(|receiver| Arc::ptr_eq(&receiver, &state.engine.rx_event));
(
owns_receiver,
!state.engine.is_cancelled(),
state.active_turn.is_some(),
)
})
.unwrap_or((false, false, false))
};
if !owns_receiver || !engine_active {
return;
}
if self.async_wake_on_agent_complete(thread_id, agent_id, !turn_is_active) {
self.spawn_async_completion_wake(thread_id.to_string());
}
}
pub(super) async fn handle_non_acp_turn_settled_async_completions(
&self,
thread_id: &str,
turn_completed: bool,
) {
let has_pending_subagent = self.async_wake_settle_turn(thread_id, turn_completed);
if !turn_completed
|| self.cancel_token.is_cancelled()
|| self.event_pump_stop.is_cancelled()
|| self.is_acp_host()
{
return;
}
let Ok(thread) = self.get_thread(thread_id).await else {
return;
};
if thread.archived || thread.executor.is_some() {
return;
}
let (engine_ready, shell_manager) = {
let active = self.active.lock().await;
let engine_ready = active
.engines
.get(thread_id)
.is_some_and(|state| !state.engine.is_cancelled() && state.active_turn.is_none());
let shell_manager = active.shell_managers.get(thread_id).cloned();
(engine_ready, shell_manager)
};
if !engine_ready {
return;
}
let (has_finished_shell, may_have_shell) = shell_manager
.and_then(|manager| {
manager.lock().ok().map(|mut guard| {
(
guard.has_finished_unreported_jobs_for_session(thread_id),
guard.may_have_undelivered_completion_for_session(thread_id),
)
})
})
.unwrap_or((false, false));
if has_pending_subagent || has_finished_shell {
self.spawn_async_completion_wake(thread_id.to_string());
}
if may_have_shell {
self.spawn_idle_shell_completion_monitor(thread_id.to_string());
}
}
pub(super) async fn deliver_async_completion_wake(&self, thread_id: &str) -> Result<()> {
if self.cancel_token.is_cancelled()
|| self.event_pump_stop.is_cancelled()
|| self.is_acp_host()
{
return Ok(());
}
self.ensure_accepting_execution()?;
let thread = match self.get_thread(thread_id).await {
Ok(thread) => thread,
Err(_) => return Ok(()),
};
if thread.archived || thread.executor.is_some() {
return Ok(());
}
let (subagent_manager, shell_manager) = {
let active = self.active.lock().await;
let Some(state) = active.engines.get(thread_id) else {
return Ok(());
};
if state.engine.is_cancelled() || state.active_turn.is_some() {
return Ok(());
}
(
state.engine.subagent_manager(),
active.shell_managers.get(thread_id).cloned(),
)
};
let has_finished_shell = shell_manager.as_ref().is_some_and(|manager| {
manager
.lock()
.is_ok_and(|mut guard| guard.has_finished_unreported_jobs_for_session(thread_id))
});
let (pending_ids, delivered_ids) = self.async_wake_pending_snapshot(thread_id);
let claimed_subagent_ids = if pending_ids.is_empty() {
Vec::new()
} else {
let eligible_ids: HashSet<String> = {
let guard = subagent_manager.read().await;
guard
.terminal_results_excluding_for_session(thread_id, &delivered_ids)
.into_iter()
.map(|result| result.agent_id)
.collect()
};
let claimed: Vec<String> = pending_ids
.into_iter()
.filter(|id| eligible_ids.contains(id))
.collect();
self.async_wake_prune_pending(thread_id, &claimed);
claimed
};
if !has_finished_shell && claimed_subagent_ids.is_empty() {
return Ok(());
}
let request = StartTurnRequest {
prompt: Self::ASYNC_COMPLETION_WAKE_PROMPT.to_string(),
..Default::default()
};
match Box::pin(self.start_turn_with_source(
thread_id,
request,
RuntimeTurnInputSource::AsyncCompletionWake,
None,
false,
))
.await
{
Ok((_turn, _replayed)) => {
self.async_wake_mark_delivered(thread_id, &claimed_subagent_ids);
Ok(())
}
Err(error) if error.to_string().contains("already has an active") => Ok(()),
Err(error) => Err(error),
}
}
pub(super) fn spawn_async_completion_wake(&self, thread_id: String) {
if self.cancel_token.is_cancelled()
|| self.event_pump_stop.is_cancelled()
|| self.is_acp_host()
{
return;
}
if !self.async_completion_wake_arm(&thread_id) {
return;
}
let manager_key = Arc::as_ptr(&self.active) as usize;
let weak = self.weak_wake_manager();
let cancel = self.cancel_token.clone();
let stop = self.event_pump_stop.clone();
let worker = tokio::spawn(async move {
struct WakeArmGuard {
manager_key: usize,
thread_id: String,
armed: bool,
}
impl Drop for WakeArmGuard {
fn drop(&mut self) {
if self.armed {
RuntimeThreadManager::async_completion_wake_release_key(
self.manager_key,
&self.thread_id,
);
}
}
}
let mut guard = WakeArmGuard {
manager_key,
thread_id: thread_id.clone(),
armed: true,
};
tokio::select! {
biased;
_ = cancel.cancelled() => return,
_ = stop.cancelled() => return,
_ = tokio::time::sleep(Self::ASYNC_COMPLETION_WAKE_DEBOUNCE) => {}
}
guard.armed = false;
Self::async_completion_wake_release_key(manager_key, &thread_id);
let Some(manager) = weak() else {
return;
};
if let Err(error) = Box::pin(manager.deliver_async_completion_wake(&thread_id)).await {
tracing::warn!(
thread_id,
%error,
"Failed to deliver coalesced async completion wake"
);
}
});
self.track_receipt_worker(worker);
}
pub(super) fn spawn_idle_shell_completion_monitor(&self, thread_id: String) {
if self.cancel_token.is_cancelled()
|| self.event_pump_stop.is_cancelled()
|| self.is_acp_host()
{
return;
}
if !self.async_shell_monitor_arm(&thread_id) {
return;
}
let manager_key = Arc::as_ptr(&self.active) as usize;
let weak = self.weak_wake_manager();
let cancel = self.cancel_token.clone();
let stop = self.event_pump_stop.clone();
let worker = tokio::spawn(async move {
struct ShellMonitorGuard {
manager_key: usize,
thread_id: String,
}
impl Drop for ShellMonitorGuard {
fn drop(&mut self) {
RuntimeThreadManager::async_shell_monitor_release_key(
self.manager_key,
&self.thread_id,
);
}
}
let _guard = ShellMonitorGuard {
manager_key,
thread_id: thread_id.clone(),
};
loop {
tokio::select! {
biased;
_ = cancel.cancelled() => break,
_ = stop.cancelled() => break,
_ = tokio::time::sleep(Self::IDLE_SHELL_WAKE_POLL_INTERVAL) => {}
}
let Some(manager) = weak() else {
break;
};
if manager.ensure_accepting_execution().is_err() || manager.is_acp_host() {
break;
}
let Ok(thread) = manager.get_thread(&thread_id).await else {
break;
};
if thread.archived || thread.executor.is_some() {
break;
}
let (engine_alive, is_idle, shell_manager) = {
let active = manager.active.lock().await;
let engine_alive = active
.engines
.get(&thread_id)
.is_some_and(|state| !state.engine.is_cancelled());
let is_idle = active
.engines
.get(&thread_id)
.is_some_and(|state| state.active_turn.is_none());
let shell_manager = active.shell_managers.get(&thread_id).cloned();
(engine_alive, is_idle, shell_manager)
};
if !engine_alive {
break;
}
let Some(shell_manager) = shell_manager else {
break;
};
let (has_finished, may_have_pending) = match shell_manager.lock() {
Ok(mut guard) => (
guard.has_finished_unreported_jobs_for_session(&thread_id),
guard.may_have_undelivered_completion_for_session(&thread_id),
),
Err(_) => break,
};
if !may_have_pending {
break;
}
if has_finished && is_idle {
manager.spawn_async_completion_wake(thread_id.clone());
}
}
});
self.track_receipt_worker(worker);
}
pub(super) fn projection_lock(&self, thread_id: &str) -> Arc<Mutex<()>> {
let mut locks = self.projection_locks.lock();
Arc::clone(
@@ -15006,6 +15702,9 @@ source_usage_records: Default::default(),
owner: Arc::downgrade(&self._process_owner_lock),
};
let input_settlement = self.user_input_settlement(&thread.id);
let allow_reactive_wake = !self.is_acp_host() && thread.executor.is_none();
let weak_wake_manager = self.weak_wake_manager();
let pump_cancel = self.cancel_token.clone();
let handle = engine.clone();
let thread_id = thread.id.clone();
let pump = tokio::spawn(async move {
@@ -15015,6 +15714,51 @@ source_usage_records: Default::default(),
event = async { original.write().await.recv().await } => match event { Some(event) => event, None => break },
_ = stop.cancelled() => break,
};
if allow_reactive_wake
&& !stop.is_cancelled()
&& !pump_cancel.is_cancelled()
&& !handle.is_cancelled()
{
enum WakeTurnSignal {
Started,
Complete,
Resumed(usize),
}
let signal = match &event {
EngineEvent::TurnStarted { .. } => Some(WakeTurnSignal::Started),
EngineEvent::TurnComplete { .. } => Some(WakeTurnSignal::Complete),
EngineEvent::Status { message } => {
Self::parse_resumed_subagent_completion_count(message)
.map(WakeTurnSignal::Resumed)
}
_ => None,
};
if let Some(signal) = signal
&& let Some(active_arc) = active.upgrade()
{
let owns_receiver = {
let guard = active_arc.lock().await;
guard.engines.get(&thread_id).is_some_and(|state| {
identity.upgrade().is_some_and(|receiver| {
Arc::ptr_eq(&receiver, &state.engine.rx_event)
})
})
};
if owns_receiver && let Some(manager) = weak_wake_manager() {
match signal {
WakeTurnSignal::Started => {
manager.async_wake_on_turn_started(&thread_id);
}
WakeTurnSignal::Complete => {
manager.async_wake_on_turn_complete(&thread_id);
}
WakeTurnSignal::Resumed(count) => {
manager.async_wake_on_resumed_status(&thread_id, count);
}
}
}
}
}
if matches!(
&event,
EngineEvent::AgentSpawned { .. }
@@ -15057,6 +15801,30 @@ source_usage_records: Default::default(),
} else {
false
};
if allow_reactive_wake
&& !stop.is_cancelled()
&& !pump_cancel.is_cancelled()
&& !handle.is_cancelled()
&& let EngineEvent::AgentComplete {
owner_session_id,
id,
parent_run_id,
..
} = &event
&& owner_session_id == &thread_id
&& parent_run_id.is_none()
&& let Some(manager) = weak_wake_manager()
{
manager
.handle_agent_complete_async_wake(
&thread_id,
id,
&sink.manager,
&active,
&identity,
)
.await;
}
if !owns_origin {
continue;
}
@@ -18145,6 +18913,11 @@ source_usage_records: Default::default(),
// next one, keeping every wake explicit and bounded to one turn.
if !admitted_acp {
self.spawn_agent_mail_safe_boundary_delivery(thread_id.clone());
self.handle_non_acp_turn_settled_async_completions(
&thread_id,
turn.status == RuntimeTurnStatus::Completed,
)
.await;
}
Ok(())
+616
View File
@@ -26146,3 +26146,619 @@ async fn profile_constitution_runtime_admission_binds_the_complete_snapshot_to_r
wait_for_terminal_turn(&manager, &turn.id).await?;
Ok(())
}
#[tokio::test]
async fn async_completion_wake_coalesces_rapid_idle_subagent_completions() -> Result<()> {
use crate::tools::subagent::{SubAgentManager, SubAgentStatus};
let manager = test_manager(test_runtime_dir())?;
let thread = manager
.create_thread(CreateThreadRequest::default())
.await?;
let mut harness = mock_engine_handle();
let subagent_manager = Arc::new(tokio::sync::RwLock::new(SubAgentManager::new(
thread.workspace.clone(),
2,
)));
harness
.handle
.set_subagent_manager(subagent_manager.clone());
manager
.install_test_engine(&thread.id, harness.handle.clone())
.await?;
let mut rx_op = harness.rx_op;
let tx_event = harness.tx_event;
let (sub_a, sub_b) = {
let mut guard = subagent_manager.write().await;
let sub_a = guard.insert_test_terminal_direct_child("sub_a", &thread.workspace);
guard.assign_test_session_owner(&sub_a, &thread.id);
let sub_b = guard.insert_test_terminal_direct_child("sub_b", &thread.workspace);
guard.assign_test_session_owner(&sub_b, &thread.id);
(sub_a, sub_b)
};
for (index, agent_id) in [sub_a, sub_b].into_iter().enumerate() {
if index > 0 {
tokio::time::sleep(Duration::from_millis(10)).await;
}
tx_event
.send(EngineEvent::AgentComplete {
display_name: Some("worker".into()),
owner_session_id: thread.id.clone(),
id: agent_id,
result: "done".into(),
outcome: Some(SubAgentStatus::Completed),
parent_run_id: None,
spawn_depth: Some(1),
continuable: Some(false),
usage: None,
})
.await?;
}
let op = tokio::time::timeout(Duration::from_secs(2), rx_op.recv())
.await
.context("expected coalesced wake turn")?;
let Some(Op::SendMessage(spec)) = op else {
bail!("expected Op::SendMessage wake operation");
};
assert_eq!(
spec.provenance,
crate::core::ops::UserInputProvenance::Runtime
);
assert_eq!(spec.content, "[runtime] Background work finished");
tx_event
.send(EngineEvent::TurnStarted {
previous_source: None,
turn_id: "engine_wake_coalesced".into(),
created_at: Utc::now(),
route: None,
submission_id: None,
})
.await?;
tx_event
.send(EngineEvent::MessageStarted { index: 0 })
.await?;
tx_event
.send(EngineEvent::MessageDelta {
index: 0,
content: "processed coalesced subagents".to_string(),
})
.await?;
tx_event
.send(EngineEvent::MessageComplete { index: 0 })
.await?;
tx_event
.send(EngineEvent::TurnComplete {
source_accounting: false,
turn_id: None,
submission_id: None,
compaction_id: None,
usage: Usage::default(),
parent_route_usage: Usage::default(),
routed_usage_dropped_records: 0,
status: TurnOutcomeStatus::Completed,
error: None,
tool_catalog: None,
base_url: None,
})
.await?;
let turns =
wait_for_terminal_turn_count(&manager, &thread.id, 1, Duration::from_secs(2)).await?;
assert_eq!(turns.len(), 1);
assert_eq!(turns[0].status, RuntimeTurnStatus::Completed);
let items = manager.store.list_items_for_turn(&turns[0].id)?;
let input_item = items
.iter()
.find(|item| item.kind == TurnItemKind::UserMessage)
.context("missing wake input item")?;
assert_eq!(
input_item.detail.as_deref(),
Some("[runtime] Background work finished")
);
assert_eq!(
input_item
.metadata
.as_ref()
.and_then(|meta| meta.get("input_provenance"))
.and_then(Value::as_str),
Some("async_completion_wake")
);
tokio::time::sleep(Duration::from_millis(120)).await;
assert!(rx_op.try_recv().is_err());
assert_eq!(manager.store.list_turns_for_thread(&thread.id)?.len(), 1);
Ok(())
}
#[tokio::test]
async fn async_completion_wake_skips_cancelled_and_archived_threads() -> Result<()> {
use crate::tools::subagent::{SubAgentManager, SubAgentStatus};
let manager = test_manager(test_runtime_dir())?;
let archived_thread = manager
.create_thread(CreateThreadRequest::default())
.await?;
let mut archived_harness = mock_engine_handle();
let shared_subagents = Arc::new(tokio::sync::RwLock::new(SubAgentManager::new(
archived_thread.workspace.clone(),
2,
)));
archived_harness
.handle
.set_subagent_manager(shared_subagents.clone());
manager
.install_test_engine(&archived_thread.id, archived_harness.handle.clone())
.await?;
let cancelled_thread = manager
.create_thread(CreateThreadRequest::default())
.await?;
let mut cancelled_harness = mock_engine_handle();
cancelled_harness
.handle
.set_subagent_manager(shared_subagents.clone());
manager
.install_test_engine(&cancelled_thread.id, cancelled_harness.handle.clone())
.await?;
let (sub_archived, sub_cancelled) = {
let mut guard = shared_subagents.write().await;
let a = guard.insert_test_terminal_direct_child("sub_arch", &archived_thread.workspace);
guard.assign_test_session_owner(&a, &archived_thread.id);
let b = guard.insert_test_terminal_direct_child("sub_canc", &cancelled_thread.workspace);
guard.assign_test_session_owner(&b, &cancelled_thread.id);
(a, b)
};
manager
.update_thread(
&archived_thread.id,
UpdateThreadRequest {
archived: Some(true),
..Default::default()
},
)
.await?;
cancelled_harness
.handle
.cancel_with_reason(crate::core::engine::CancelReason::User);
archived_harness
.tx_event
.send(EngineEvent::AgentComplete {
display_name: None,
owner_session_id: archived_thread.id.clone(),
id: sub_archived,
result: "done".into(),
outcome: Some(SubAgentStatus::Completed),
parent_run_id: None,
spawn_depth: Some(1),
continuable: Some(false),
usage: None,
})
.await?;
cancelled_harness
.tx_event
.send(EngineEvent::AgentComplete {
display_name: None,
owner_session_id: cancelled_thread.id.clone(),
id: sub_cancelled,
result: "done".into(),
outcome: Some(SubAgentStatus::Completed),
parent_run_id: None,
spawn_depth: Some(1),
continuable: Some(false),
usage: None,
})
.await?;
tokio::time::sleep(Duration::from_millis(140)).await;
assert!(archived_harness.rx_op.try_recv().is_err());
assert!(cancelled_harness.rx_op.try_recv().is_err());
assert!(
manager
.store
.list_turns_for_thread(&archived_thread.id)?
.is_empty()
);
assert!(
manager
.store
.list_turns_for_thread(&cancelled_thread.id)?
.is_empty()
);
Ok(())
}
#[tokio::test]
async fn async_completion_wake_waits_for_busy_turn_settlement_and_wakes_once() -> Result<()> {
use crate::tools::subagent::{SubAgentManager, SubAgentStatus};
let manager = test_manager(test_runtime_dir())?;
let thread = manager
.create_thread(CreateThreadRequest::default())
.await?;
let mut harness = mock_engine_handle();
let subagent_manager = Arc::new(tokio::sync::RwLock::new(SubAgentManager::new(
thread.workspace.clone(),
1,
)));
harness
.handle
.set_subagent_manager(subagent_manager.clone());
manager
.install_test_engine(&thread.id, harness.handle.clone())
.await?;
let mut rx_op = harness.rx_op;
let tx_event = harness.tx_event;
let first_turn = manager
.start_turn(
&thread.id,
StartTurnRequest {
prompt: "primary user turn".into(),
..Default::default()
},
)
.await?;
let Some(Op::SendMessage(first_spec)) =
tokio::time::timeout(Duration::from_secs(2), rx_op.recv()).await?
else {
bail!("missing Turn 1 operation");
};
assert_eq!(first_spec.content, "primary user turn");
tx_event
.send(EngineEvent::TurnStarted {
previous_source: None,
turn_id: "engine_busy_1".into(),
created_at: Utc::now(),
route: None,
submission_id: None,
})
.await?;
let sub_busy = {
let mut guard = subagent_manager.write().await;
let id = guard.insert_test_terminal_direct_child("sub_busy", &thread.workspace);
guard.assign_test_session_owner(&id, &thread.id);
id
};
tx_event
.send(EngineEvent::AgentComplete {
display_name: None,
owner_session_id: thread.id.clone(),
id: sub_busy,
result: "done while busy".into(),
outcome: Some(SubAgentStatus::Completed),
parent_run_id: None,
spawn_depth: Some(1),
continuable: Some(false),
usage: None,
})
.await?;
manager.spawn_async_completion_wake(thread.id.clone());
tokio::time::sleep(Duration::from_millis(120)).await;
assert!(rx_op.try_recv().is_err());
tx_event
.send(EngineEvent::MessageStarted { index: 0 })
.await?;
tx_event
.send(EngineEvent::MessageDelta {
index: 0,
content: "finished primary user turn".to_string(),
})
.await?;
tx_event
.send(EngineEvent::MessageComplete { index: 0 })
.await?;
tx_event
.send(EngineEvent::TurnComplete {
source_accounting: false,
turn_id: None,
submission_id: None,
compaction_id: None,
usage: Usage::default(),
parent_route_usage: Usage::default(),
routed_usage_dropped_records: 0,
status: TurnOutcomeStatus::Completed,
error: None,
tool_catalog: None,
base_url: None,
})
.await?;
let settled_first = wait_for_terminal_turn(&manager, &first_turn.id).await?;
assert_eq!(settled_first.status, RuntimeTurnStatus::Completed);
let Some(Op::SendMessage(wake_spec)) =
tokio::time::timeout(Duration::from_secs(2), rx_op.recv()).await?
else {
bail!("missing deferred wake operation");
};
assert_eq!(
wake_spec.provenance,
crate::core::ops::UserInputProvenance::Runtime
);
assert_eq!(wake_spec.content, "[runtime] Background work finished");
tx_event
.send(EngineEvent::TurnStarted {
previous_source: None,
turn_id: "engine_busy_2".into(),
created_at: Utc::now(),
route: None,
submission_id: None,
})
.await?;
tx_event
.send(EngineEvent::MessageStarted { index: 0 })
.await?;
tx_event
.send(EngineEvent::MessageDelta {
index: 0,
content: "handled deferred wake".to_string(),
})
.await?;
tx_event
.send(EngineEvent::MessageComplete { index: 0 })
.await?;
tx_event
.send(EngineEvent::TurnComplete {
source_accounting: false,
turn_id: None,
submission_id: None,
compaction_id: None,
usage: Usage::default(),
parent_route_usage: Usage::default(),
routed_usage_dropped_records: 0,
status: TurnOutcomeStatus::Completed,
error: None,
tool_catalog: None,
base_url: None,
})
.await?;
let turns =
wait_for_terminal_turn_count(&manager, &thread.id, 2, Duration::from_secs(2)).await?;
assert_eq!(turns.len(), 2);
assert_eq!(turns[1].status, RuntimeTurnStatus::Completed);
let second_items = manager.store.list_items_for_turn(&turns[1].id)?;
assert!(second_items.iter().any(|item| {
item.metadata
.as_ref()
.and_then(|meta| meta.get("input_provenance"))
.and_then(Value::as_str)
== Some("async_completion_wake")
}));
tokio::time::sleep(Duration::from_millis(120)).await;
assert!(rx_op.try_recv().is_err());
Ok(())
}
#[tokio::test]
async fn async_completion_wake_skips_turn_that_already_drained_subagent() -> Result<()> {
use crate::tools::subagent::{SubAgentManager, SubAgentStatus};
let manager = test_manager(test_runtime_dir())?;
let thread = manager
.create_thread(CreateThreadRequest::default())
.await?;
let mut harness = mock_engine_handle();
let subagent_manager = Arc::new(tokio::sync::RwLock::new(SubAgentManager::new(
thread.workspace.clone(),
1,
)));
harness
.handle
.set_subagent_manager(subagent_manager.clone());
manager
.install_test_engine(&thread.id, harness.handle.clone())
.await?;
let mut rx_op = harness.rx_op;
let tx_event = harness.tx_event;
let turn = manager
.start_turn(
&thread.id,
StartTurnRequest {
prompt: "draining turn".into(),
..Default::default()
},
)
.await?;
assert!(matches!(
tokio::time::timeout(Duration::from_secs(2), rx_op.recv()).await?,
Some(Op::SendMessage(_))
));
tx_event
.send(EngineEvent::TurnStarted {
previous_source: None,
turn_id: "engine_drain_1".into(),
created_at: Utc::now(),
route: None,
submission_id: None,
})
.await?;
let sub_drained = {
let mut guard = subagent_manager.write().await;
let id = guard.insert_test_terminal_direct_child("sub_drained", &thread.workspace);
guard.assign_test_session_owner(&id, &thread.id);
id
};
tx_event
.send(EngineEvent::AgentComplete {
display_name: None,
owner_session_id: thread.id.clone(),
id: sub_drained,
result: "already absorbed".into(),
outcome: Some(SubAgentStatus::Completed),
parent_run_id: None,
spawn_depth: Some(1),
continuable: Some(false),
usage: None,
})
.await?;
tx_event
.send(EngineEvent::status(
"Resuming turn with 1 late sub-agent completion(s)",
))
.await?;
tx_event
.send(EngineEvent::MessageStarted { index: 0 })
.await?;
tx_event
.send(EngineEvent::MessageDelta {
index: 0,
content: "already consumed child completion".to_string(),
})
.await?;
tx_event
.send(EngineEvent::MessageComplete { index: 0 })
.await?;
tx_event
.send(EngineEvent::TurnComplete {
source_accounting: false,
turn_id: None,
submission_id: None,
compaction_id: None,
usage: Usage::default(),
parent_route_usage: Usage::default(),
routed_usage_dropped_records: 0,
status: TurnOutcomeStatus::Completed,
error: None,
tool_catalog: None,
base_url: None,
})
.await?;
let settled = wait_for_terminal_turn(&manager, &turn.id).await?;
assert_eq!(settled.status, RuntimeTurnStatus::Completed);
tokio::time::sleep(Duration::from_millis(140)).await;
assert!(rx_op.try_recv().is_err());
assert_eq!(manager.store.list_turns_for_thread(&thread.id)?.len(), 1);
Ok(())
}
#[tokio::test]
async fn async_completion_wake_monitors_background_shell_after_turn_settlement() -> Result<()> {
let manager = test_manager(test_runtime_dir())?;
let thread = manager
.create_thread(CreateThreadRequest::default())
.await?;
let harness = install_mock_engine(&manager, &thread.id).await;
let shell_manager = manager
.thread_shell_manager(&thread.id, true)
.await?
.context("expected thread shell manager")?;
{
let mut guard = shell_manager
.lock()
.map_err(|_| anyhow!("shell manager lock poisoned"))?;
guard.execute_with_options_env_for_session(
"sleep 0.12",
None,
5_000,
true,
None,
false,
None,
HashMap::new(),
&thread.id,
)?;
}
let mut rx_op = harness.rx_op;
let tx_event = harness.tx_event;
let thread_id = thread.id.clone();
let shell_mgr_for_mock = shell_manager.clone();
tokio::spawn(async move {
let mut index = 0usize;
while let Some(op) = rx_op.recv().await {
if let Op::SendMessage(spec) = op {
index += 1;
if index > 1 {
assert_eq!(
spec.provenance,
crate::core::ops::UserInputProvenance::Runtime
);
if let Ok(mut guard) = shell_mgr_for_mock.lock() {
let _ = guard.drain_finished_jobs_with_evidence_for_session(&thread_id);
}
}
let _ = tx_event
.send(EngineEvent::TurnStarted {
previous_source: None,
turn_id: format!("engine_shell_{index}"),
created_at: Utc::now(),
route: None,
submission_id: None,
})
.await;
let _ = tx_event
.send(EngineEvent::MessageStarted { index: 0 })
.await;
let _ = tx_event
.send(EngineEvent::MessageDelta {
index: 0,
content: format!("shell step {index}"),
})
.await;
let _ = tx_event
.send(EngineEvent::MessageComplete { index: 0 })
.await;
let _ = tx_event
.send(EngineEvent::TurnComplete {
source_accounting: false,
turn_id: None,
submission_id: None,
compaction_id: None,
usage: Usage::default(),
parent_route_usage: Usage::default(),
routed_usage_dropped_records: 0,
status: TurnOutcomeStatus::Completed,
error: None,
tool_catalog: None,
base_url: None,
})
.await;
}
}
});
manager
.start_turn(
&thread.id,
StartTurnRequest {
prompt: "launch background shell".into(),
..Default::default()
},
)
.await?;
let turns =
wait_for_terminal_turn_count(&manager, &thread.id, 2, Duration::from_secs(5)).await?;
assert_eq!(turns.len(), 2);
assert_eq!(turns[0].status, RuntimeTurnStatus::Completed);
assert_eq!(turns[1].status, RuntimeTurnStatus::Completed);
let second_items = manager.store.list_items_for_turn(&turns[1].id)?;
assert!(second_items.iter().any(|item| {
item.metadata
.as_ref()
.and_then(|meta| meta.get("input_provenance"))
.and_then(Value::as_str)
== Some("async_completion_wake")
}));
tokio::time::sleep(Duration::from_millis(250)).await;
assert_eq!(manager.store.list_turns_for_thread(&thread.id)?.len(), 2);
Ok(())
}
+1 -18
View File
@@ -2009,7 +2009,7 @@ impl SavedTurnOutcome {
/// Build the persisted record for a failed turn. Redacts before bounding
/// so a cut can never leave half a secret behind.
pub fn failed(error: &str, after_message_count: usize) -> Self {
let redacted = codewhale_secrets::redact::redact_secrets(error.trim());
let redacted = codewhale_sanitize::redact::redact_secrets(error.trim());
let error = if redacted.chars().count() > MAX_SAVED_TURN_OUTCOME_ERROR_CHARS {
let mut cut: String = redacted
.chars()
@@ -4309,23 +4309,6 @@ impl SessionManager {
self.save_session_inner(session, JournalPrune::Excess, true)
}
#[cfg(any(test, feature = "test-support"))]
pub fn commit_completed_session(&self, session: SavedSession) -> io::Result<io::Result<()>> {
let session_id = session.metadata.id.clone();
let exec_mode = session.metadata.mode.as_deref() == Some("exec");
let clear = self
.with_session_write_admission(&session_id, || {
let persisted = self.load_session_write_metadata(&session_id)?;
self.write_session_admitted(session, JournalPrune::Excess, true, &persisted)?;
Ok(self.clear_session_checkpoint_admitted(&session_id))
})?
.ok_or_else(Self::retired_session_write_error)?;
if let Err(error) = self.cleanup_old_sessions_for_mode(None, Some(exec_mode)) {
tracing::warn!(session_id = %session_id, %error, "completion was committed but retention could not be inspected");
}
Ok(clear)
}
/// Make a saved document whose file exceeds the full-history transport
/// bound readable under it. The bound counts what a canonical handoff
/// carries: compact JSON without the journal-derived `messages` copy
+2 -2
View File
@@ -203,7 +203,7 @@ pub const REDACTED_PLACEHOLDER: &str = "[redacted]";
/// Mask credential-shaped substrings.
///
/// The shared export sanitizer ([`codewhale_secrets::sanitize::sanitize_text`],
/// The shared export sanitizer ([`codewhale_sanitize::sanitize::sanitize_text`],
/// also used by `/export`) runs first: it covers `Bearer <token>`, JWTs,
/// URL credentials, PEM blocks, and quoted JSON/TOML keyed secrets. A
/// token-level pass then masks the extra bare prefixes (AWS, Google, Hugging
@@ -211,7 +211,7 @@ pub const REDACTED_PLACEHOLDER: &str = "[redacted]";
/// peek line is cheap; leaking a key over a LAN is not.
#[must_use]
pub fn redact(text: &str) -> (String, bool) {
let sanitized = codewhale_secrets::sanitize::sanitize_text(text);
let sanitized = codewhale_sanitize::sanitize::sanitize_text(text);
// The sanitizer also strips control bytes and normalises URLs, so compare
// placeholder counts rather than bytes to decide whether it redacted.
let mut redacted = redaction_marks(&sanitized) > redaction_marks(text);
+62 -7
View File
@@ -82,13 +82,7 @@ pub fn build_approval_key(tool_name: &str, input: &serde_json::Value) -> Approva
| "exec_shell_wait"
| "exec_shell_interact"
| "exec_wait"
| "exec_interact" => {
let mut execution_input = input.clone();
if let Some(fields) = execution_input.as_object_mut() {
fields.remove("description");
}
format!("shell:{tool_name}:{}", hash_json_value(&execution_input))
}
| "exec_interact" => exact_shell_call_key(tool_name, input),
"fetch_url" | "web.fetch" | "web_fetch" => {
let host = parse_host(input);
format!("net:{host}")
@@ -113,6 +107,9 @@ pub fn build_approval_grouping_key(tool_name: &str, input: &serde_json::Value) -
let paths_hash = hash_patch_paths(input);
format!("patch:{paths_hash}")
}
"exec_shell" | "task_shell_start" if shell_call_carries_stdin(input) => {
exact_shell_call_key(tool_name, input)
}
"exec_shell" | "task_shell_start" => shell_command_grant_scope(input),
// Interact and wait calls carry no command, only input for a live
// session. Keying them on the (empty) command prefix gave every one
@@ -447,6 +444,20 @@ pub fn computer_use_batch_hidden_gate(tool_name: &str, input: &Value) -> Option<
})
}
fn exact_shell_call_key(tool_name: &str, input: &serde_json::Value) -> String {
let mut execution_input = input.clone();
if let Some(fields) = execution_input.as_object_mut() {
fields.remove("description");
}
format!("shell:{tool_name}:{}", hash_json_value(&execution_input))
}
fn shell_call_carries_stdin(input: &serde_json::Value) -> bool {
["stdin", "input", "data"]
.iter()
.any(|key| input.get(*key).is_some_and(|value| !value.is_null()))
}
/// The session-grant scope for a shell command.
///
/// A simple command whose family is in the arity dictionary keeps the
@@ -1021,6 +1032,50 @@ mod tests {
}
}
#[test]
fn shell_run_grant_does_not_cover_a_different_stdin() {
for tool in ["Bash", "bash", "exec_shell", "task_shell_start"] {
let plain = json!({"command": "python3 -"});
let hello = json!({"command": "python3 -", "stdin": "print(1)"});
let other = json!({"command": "python3 -", "stdin": "print(2)"});
let granted = build_approval_grouping_key(tool, &plain);
let with_stdin = build_approval_grouping_key(tool, &hello);
assert_ne!(granted, with_stdin, "{tool}: stdin is part of what runs");
assert_ne!(
with_stdin,
build_approval_grouping_key(tool, &other),
"{tool}: different stdin must not share a grant"
);
assert_eq!(
with_stdin,
build_approval_grouping_key(tool, &hello),
"{tool}: the same call keeps its grant"
);
for alias in ["input", "data"] {
let aliased = json!({"command": "python3 -", alias: "print(2)"});
assert_ne!(
granted,
build_approval_grouping_key(tool, &aliased),
"{tool}: `{alias}` carries stdin too"
);
assert_ne!(
with_stdin,
build_approval_grouping_key(tool, &aliased),
"{tool}: `{alias}` input is not the `stdin` input of another call"
);
}
assert_eq!(
granted,
build_approval_grouping_key(tool, &json!({"command": "python3 -", "stdin": null})),
"{tool}: a null stdin is no stdin"
);
}
let family_plain = build_approval_grouping_key("Bash", &json!({"command": "git status"}));
let family_stdin =
build_approval_grouping_key("Bash", &json!({"command": "git status", "stdin": "x"}));
assert_ne!(family_plain, family_stdin);
}
#[test]
fn grouping_key_grants_mcp_tools_as_reviewed_kinds() {
// A session grant for a reviewed plugin MCP tool is the kind
+8 -1
View File
@@ -165,7 +165,7 @@ fn generic_summary(locale: Locale, name: &str, input: &Value, workspace: Option<
fn is_delegated_work_tool(tool_name: &str) -> bool {
matches!(
tool_name,
"task_create" | "automation_create" | "automation_update"
"task_create" | "automation_create" | "automation_update" | "send_later"
)
}
@@ -728,6 +728,13 @@ mod tests {
assert!(summary.contains("Auto-approve: on"), "{summary}");
assert!(summary.contains("Workspace: /x"), "{summary}");
let summary = approval_summary(
"send_later",
&json!({"action": "schedule", "delay_minutes": 5, "message": "m", "workspace": "/elsewhere"}),
None,
);
assert!(summary.contains("Workspace: /elsewhere"), "{summary}");
let zh = approval_summary_in(
Locale::ZhHans,
"automation",
+52 -7
View File
@@ -61,6 +61,7 @@
use serde_json::Value;
use crate::tools::spec::{ApprovalRequirement, ToolCapability, ToolSpec};
use codewhale_agent_contract::worker_profile::ShellGrant;
use codewhale_execpolicy::canonical_action::canonical_action_alias;
/// The execution authority a child actually holds, read off the runtime posture
@@ -477,6 +478,15 @@ pub fn enforce_execution_envelope(
}
pub fn tool_call_denied(rules: Option<&[String]>, name: &str, input: &Value) -> bool {
tool_call_denied_for(rules, name, input, false)
}
pub fn tool_call_denied_for(
rules: Option<&[String]>,
name: &str,
input: &Value,
holds_bounded_verification: bool,
) -> bool {
let action = canonical_action_alias(name, input);
codewhale_execpolicy::tool_rules::tool_denied(rules, name)
|| codewhale_execpolicy::tool_rules::tool_denied(rules, action)
@@ -486,12 +496,40 @@ pub fn tool_call_denied(rules: Option<&[String]>, name: &str, input: &Value) ->
.and_then(Value::as_str)
.is_some_and(|url| !url.trim().is_empty())
&& codewhale_execpolicy::tool_rules::tool_denied(rules, "fetch_url"))
|| (matches!(
classify_verification(action, input),
Some(VerificationBound::Unbounded)
) && rules.is_some_and(|rules| {
codewhale_execpolicy::tool_rules::tool_matches_any_rule(rules, "Bash")
}))
|| (starts_workspace_execution(name, action, input, holds_bounded_verification)
&& rules.is_some_and(|rules| {
codewhale_execpolicy::tool_rules::tool_matches_any_rule(rules, "Bash")
}))
}
fn starts_workspace_execution(
name: &str,
action: &str,
input: &Value,
holds_bounded_verification: bool,
) -> bool {
match classify_verification(action, input) {
Some(VerificationBound::Unbounded) => true,
Some(VerificationBound::Default | VerificationBound::Filter) => !holds_bounded_verification,
None => {
(name.eq_ignore_ascii_case("send_later")
&& input.get("action").and_then(Value::as_str) == Some("schedule"))
|| (name.eq_ignore_ascii_case("workflow")
&& input.get("verify").and_then(Value::as_bool) == Some(true)
&& !holds_bounded_verification)
}
}
}
#[must_use]
pub fn holds_bounded_verification(context: &crate::tools::spec::ToolContext) -> bool {
context
.child_host
.as_ref()
.is_some_and(|child| child.grant.shell == ShellGrant::Verify)
|| context.tool_authority.as_deref().is_some_and(|authority| {
authority.verification == crate::tools::spec::ToolVerificationAuthority::Bounded
})
}
/// Repeat the command ceiling at native dispatch and direct delegation sinks.
@@ -507,7 +545,14 @@ pub fn enforce_tool_denial(
&& context.owner_agent_id.is_some()
&& context.shell_policy == codewhale_agent_contract::worker_profile::ShellPolicy::ReadOnly
&& crate::tools::shell::agent_readonly_bash_input(input);
if !bounded_child_read && tool_call_denied(Some(&context.disallowed_tools), name, input) {
if !bounded_child_read
&& tool_call_denied_for(
Some(&context.disallowed_tools),
name,
input,
holds_bounded_verification(context),
)
{
return Err(crate::tools::spec::ToolError::permission_denied(format!(
"Tool '{name}' or its execution dependency is in the disallowed-tools list"
)));
Loaded 100 of 376 files, more files were not shown because too many files have changed in this diff. Show more