Compare commits

..
Author SHA1 Message Date
CodeWhale BotandClaude Sonnet 5.5 3f99cdab9e fix(config): show the telemetry setting default as on, matching the runtime
The settings schema declared telemetry default false while the resolver
(telemetry_consent_from_env) treats an unset value as on, so the settings
API reported default false for a default-on policy.

Landed in Engine push 18 (CW-7875) with the CHANGELOG line the review wrote
(CW-7987 REVIEW.md finding 2; the packet's line was not used). This is the
committed part only: CW-7987 stays open for the active-version readback and
the ingest bundle rebuilt from the frozen candidate. Review: farm runs
20261112T110021Z-gtelrev-config-37102e (codewhale-config, 21 run, 21 passed,
823 skipped) and 20261112T110024Z-gtelrev-telem-751b20 (codewhale-telemetry,
53 run, 53 passed).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 18:02:40 -08:00
CodeWhale BotandClaude Sonnet 5.5 ae18591d10 feat(router): preserve explicit per-question decision abstention
An explicit abstained:true on a System One answer now survives decoding and the
routing receipt, and can never authorize Auto (tier), the thinking effort, a
shadow noul read or a shadow chat intent, even at a zero confidence floor. A
non-boolean abstained value fails answer decoding and is refused like any other
malformed answer. Provider usage evidence and the declared fallback are
unchanged. The custom decision endpoint half of the original change is not
included.

Receipt field abstained_questions touches crates/models, cost_status and the
tui activity_detail fixture outside the slice owner list; it is additive with a
serde default.

Landed in Engine push 18 (CW-7875) with the CHANGELOG line from the review (CW-7992 REVIEW.md finding 1). Review: farm run 20261112T112213Z-leg01review-tip-faf9ea, 42 run, 42 passed, 7181 skipped; counterfactual run 20261112T112313Z-leg01review-cf-032c60 fails 2 of 3 with the guards removed.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 18:02:27 -08:00
CodeWhale BotandClaude Sonnet 5.5 4e20c2ba06 test(engine): expire the dropped-dispatch deadline only after the client call is entered (CW-8127)
The dropped-dispatch test gave the nested call a real 100 ms deadline. On a
slow runner that deadline could expire before bind_dispatch settled the
unknown-usage record, so the biased wall-clock select won with zero records
(left: 0, right: 1). Give the call a 600 s deadline, wait until the fixture
client is entered (the record is settled before the client is called), then
pause tokio time and advance 601 s so the deadline always fires after dispatch.
The wait selects on entered against the dispatch with a panic if the dispatch
ends first, inside a 10 s real-time timeout, so a regression that stops the
dispatch before the client fails at once instead of hanging. Every assertion is
unchanged. Product code is unchanged.

nextest stress 40/40 passed, 6 tests per iteration, target test 40/40 (farm run
20261112T113538Z-cw8127r1-new-33edc8); the old test under the same stress failed
1 of 20 at left: 0 (farm run 20261112T104528Z-cw8127-old-7569e0).
clippy -p codewhale-engine --all-targets --all-features -D warnings clean (farm
run 20261112T113538Z-cw8127r1-clippy-373b1e); cargo fmt --check and
scripts/preflight.sh --check OK.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 18:02:19 -08:00
CodeWhale BotandClaude Sonnet 5.5 b87bd535c2 fix(app-server): plugin index names plugins by id, labels tools, signals cut name lists (CW-8049 PL-02 repair 1)
/v1/commands rows carry plugin_id (not the name) and workspace_sourced; the
index projects an active plugin's commands from those rows, labels native and
language-server plugins as tool, sources project commands as project, and
reports a cut or control-character name list under problems. Doc comment
removed.

Checks (Boat farm, Linux x86_64, at c3bc310042): nextest -p codewhale-app-server
-p codewhale-tui -E 'test(/plugin_index|command_catalog|get_v1_commands|user_registry|commands::discovery/)':
51 run, 51 passed, 6713 skipped (run 20261112T080110Z-plidx4-test-26f07e);
clippy -D warnings for engine, app-server and tui: clean
(run 20261112T080110Z-plidx4-clippy-124ba8). fmt, runtime-routes and facts
were the worker's local checks; preflight was not run on that tip.

Landing (push 18): rebased onto push 17; agent-docs.generated.json conflicted
with push 17's regeneration and was regenerated with derive-facts at each of
the two picks. docs/RUNTIME_ROUTES.md: check-runtime-routes PASS, no change.
CHANGELOG line from the round-2 review wording.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 18:02:11 -08:00
CodeWhale BotandClaude Sonnet 5.5 969059fb83 feat(app-server): one read route lists every plugin kind (CW-8049 PL-02)
GET /v1/apps/plugin-index projects the plugin registry, the built-in
switch catalog, skills, connectors, hooks and hand-written commands into
one list with kind, source and state. Each item embeds the row its owner's
route returns by calling that owner's handler, so a field an owner adds
appears here without a second edit; an owner that fails is named under
unavailable and the rest still list. A plugin item carries the names of the
skills, connectors, hooks, commands and profiles it adds, and those are
not listed again as items. Replaces nothing; the owners keep their routes.

A user command row now says which plugin added it (UserCommand.plugin,
CommandCatalogEntry.plugin, filled from the registry's plugin authority),
so a plugin's markdown command is not shown as hand-written. The wiring
in runtime_api.rs (mod line, route line, that field) and the
command_catalog and tui hunks are separable from plugin_index.rs.

Checks: farm nextest 6 run, 6 passed; farm clippy -D warnings clean for
engine, app-server, tui; preflight OK.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 18:02:03 -08:00
CodeWhale BotandClaude Fable 5.2 09797fa316 refactor(catalog): one model listing for both commands, and the inventory stops judging windows by name
With the readings proved equal on one captured catalog, the second copies
go:

- `route_preferences::list_models` no longer assembles its own roster
  (catalog rows, saved default, sort, dedup). It calls
  `provider_lake::listed_models_for_route`, the same function `models`
  prints.
- The routing inventory no longer asks the bundled catalog whether a model
  name has a context window to decide if a number is verified. The
  resolved route's own source decides; a route that does not resolve is
  unverified.

Farm (Linux x86_64), this commit: catalog_parity 1 test run: 1 passed;
config and engine lib tests filtered
`catalog model pricing cost_status route`: 1316 passed, 0 failed; clippy
-D warnings for config and engine (all targets, all features) and for cli
and app-server (all targets): clean.

Landing (push 18): CHANGELOG line for the pair (REF-05, CW-7971) from the
round-2 review, added in this commit.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
2026-11-12 18:01:54 -08:00
CodeWhale BotandClaude Fable 5.2 242938a572 fix(catalog): both model commands, the Runtime API, routing and cost read one captured catalog the same way
The new check loads one captured Models.dev catalog (the bundled seed,
sha256 7c48830c…d256) and an id-only Anthropic roster into a sealed home,
then reads three routes through `model list` / `model resolve`, `models`,
`GET /v1/providers/{id}/models`, the routing inventory and the cost
envelope: the official Anthropic route, a custom endpoint serving a
same-named model, and the Z.ai plan endpoint. It prints every reading and
fails on any disagreement.

On the previous source it found three:

- `model list` skipped a custom endpoint unless its table was literally
  named `custom`. It now lists every configured custom route under the
  route's own name.
- `model resolve` refused a custom endpoint's saved model ("no registered
  model") because the legacy registry was asked about an endpoint that owns
  its own roster. The registry is no longer consulted for those routes; the
  route resolver and a fetched roster still decide.
- The routing inventory gave a custom endpoint the context window and
  output limit of a same-named model and marked the window verified. It
  now takes the window, its source and the output limit from the resolved
  route, so a borrowed number is marked unverified and an unpublished
  output limit stays unknown.

`provider_lake::listed_models_for_route` is the list body of `models`
moved out unchanged so the check can read it.

One spelling difference is left and reported, not asserted: the inventory
normalises a listed `glm-5.1` to `GLM-5.1`.

One disagreement is also named, not hidden: a custom endpoint has no
published window, so routing still reports 200,000 for a saved model whose
name matches a known one (the router's long-context tag reads it), marked
unverified, while the Runtime API reports unknown. The check records the
Runtime reading as unknown and allows exactly that pair as a known
difference; any other window disagreement fails.

The check also pins the charge side: a custom endpoint gets no charge
(`UnknownBillingBasis`); the Z.ai plan route gets no charge
(`NotMoneyMetered`) even for a model that has a list rate, and both
surfaces show that same list rate; an id-only first-party model with no
price row reads unknown on both surfaces and produces no charge.

Farm (Linux x86_64): config, engine and app-server lib tests filtered
`catalog_parity catalog model pricing cost_status route`: 1444 passed,
0 failed; clippy -D warnings for config, engine, cli and app-server (all
targets, all features): clean.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 18:01:39 -08:00
CodeWhale BotandClaude Sonnet 5.5 9e68dcc886 fix(acp): a plugin tool's nested approval reaches the editor as a permission request that names the nested tool (CW-8050)
When a plugin tool asks the core to run a tool that needs approval, the ACP
projection resolves the nested execution id to the admitted plugin call and
shows the request on it. The request is built from the approval.required
payload (nested tool name, input, summary) so title, kind and rawInput show
what is being approved, and it carries no status so the running plugin call is
not reset to pending. The gates test and journey test live in acp_journey and
assert the second request names write and carries nested.txt.

Docs (RUNTIME_API, EXTENSIONS, zh_hans) and web/data/agent-docs.generated.json
updated.

Checks: nextest -p codewhale-app-server -E 'test(acp)' on the build farm,
48 run, 48 passed, 815 skipped; cargo fmt --check clean; preflight --check OK.

Landing (push 18): rebased onto push 17. The zh_hans EXTENSIONS and
RUNTIME_API paragraphs conflicted with push 17's CW-8071 wording of the same
ACP paragraph; kept push 17's text and added this change's sentences where
the English copies have them. agent-docs.generated.json regenerated with
derive-facts. CHANGELOG line from the round-2 review.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 18:01:32 -08:00
CodeWhale BotandClaude Opus 5.5 8aa7707d04 fix(web): list MannXo with the post-0.10.3 contributors and drop the duplicate Qwen Code line (CW-8181)
5d7ca06a5a added @MannXo to RELEASE_CONTRIBUTORS, the website's 0.10.3
credits, but #7293 landed after 0.10.3: docs/CONTRIBUTORS.md lists MannXo in
"Unreleased — landed after v0.10.3", and the 0.10.3 "### Contributors" ledger
in CHANGELOG.md does not name them. public-copy.test.ts ("keeps
current-release website credits in exact changelog parity") then failed, which
failed the Lint & Type Check job of web.yml run 43344178264 and skipped the
Cloudflare deploy. MannXo moves to UNRELEASED_CONTRIBUTORS beside
SamhandsomeLee, matching the contributor doc.

The same commit meant to rewrite the Qwen Code changelog line with (#7293) but
added a second copy; the old "(PR 7293)" line is removed from CHANGELOG.md and
crates/tui/CHANGELOG.md, keeping the (#7293) form the release-note receipt
check looks for.

Evidence (web/, worktree claude-CW-8181b, on 97c1ad923d):
- public-copy.test.ts: passes (part of 5 files, 109/110; the one failure is
  ratatui/catalogue "Test timed out in 5000ms" with load average ~60)
- npm run check: exit 0 (facts OK, docs PASS, runtime routes 243 PASS,
  next build 979/979 static pages)
- npm test: 1303/1306; the 3 failures are 5 s timeouts in
  public-installer-contract and ratatui/catalogue that move between runs
  under that load, in files this change does not touch
- scripts/check-contributor-credit.py: "Every contributor in the window is
  credited on all three surfaces."

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-12 17:37:28 -08:00
45 changed files with 2096 additions and 1176 deletions

No files matched your search

+6 -1
View File
@@ -11,9 +11,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Non-ASCII file names:** `/diff`, the `@git` and `@diff` mentions and the Runtime API workspace diff now show paths with Chinese, emoji and other non-ASCII characters as written instead of as quoted octal escapes, and the turn context no longer lists the workspace root as an empty "Active paths" entry. Thanks [@luochen211](https://github.com/luochen211) (PR 7317).
- **Session names after a crash, and keep-forever snapshot retention:** a session name saved right after a crash that cut off the previous session-index line is no longer lost, because the new entry now starts on its own line; and a very large `[snapshots] max_age_days` (such as `9223372036854775807` for "keep forever") no longer wraps around and deletes every undo snapshot at session start. Thanks [@luochen211](https://github.com/luochen211) (PR 7316).
- **Blank environment variables:** exporting a variable empty (for example `CODEWHALE_MODEL=` from a compose `${VAR:-}` template) no longer hides its legacy alias. `DEEPSEEK_MODEL`, `DEEPSEEK_BASE_URL`, `KIMI_BASE_URL`, `DEEPSEEK_FORCE_HTTP1`, `DEEPSEEK_STREAM_OPEN_TIMEOUT_SECS` and the other provider alias variables now apply as they do when the primary is unset. Thanks [@luochen211](https://github.com/luochen211) (PR 7315).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (PR 7293).
- **MCP and language servers that send requests mid-call:** a tool call no longer fails with "neither a result nor an error" when an MCP server sends a `ping` or other request that reuses the call's id while the call is running, and a language server's `workspace/configuration` or similar request no longer ends a pending LSP request. Codewhale now answers `ping`, answers other server requests with "Method not found" (it never advertised them), keeps waiting for the real reply, and a refused answer over Streamable HTTP no longer drops the connection to legacy SSE. Thanks [@luochen211](https://github.com/luochen211) (PR 7319).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (#7293).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (PR 7293).
- **Telemetry setting default:** the settings schema API reports telemetry's default as on, matching what Codewhale does when you have not chosen.
- **Auto routing abstention:** when the decision model marks a tier, effort or shadow answer `abstained: true`, Auto no longer acts on it, even with `min_confidence = 0`: an abstained tier takes the local fallback, an abstained effort keeps the tier and drops only the effort, and the routing receipt lists the abstained questions.
- **Plugin index (Runtime API):** a new `GET /v1/apps/plugin-index` route lists plugins, skills, connectors, hooks and hand-written commands with kind, source and state, and `/v1/commands` rows say which plugin (`plugin_id`) added a command and whether a command came from the project (`workspace_sourced`) (CW-8092).
- **Model commands and routing:** `codewhale model list` now lists every custom endpoint you have configured, under its own name. `codewhale model resolve` no longer refuses a model on a custom endpoint, router or local server whose model list has not been fetched yet; once a list is fetched, a model missing from it is still refused. Automatic routing marks a context window as verified only when the route itself supplies it, and no longer reports a guessed output limit.
- **ACP:** when a plugin tool asks the core to run a tool that needs approval, the editor now gets a `session/request_permission` on the plugin tool's call that names the nested tool and its input; before, the prompt ended with an internal error (CW-8050).
- **README and contributor credit:** the repository front page now opens the way the website does: the line "The model can be anyone's. The agent should be yours.", one sentence on what Codewhale is, the install command and one picture, with reference detail moved into the linked guides; the 18 translated READMEs follow the same structure and say they are machine-drafted. The contributor wall and the website now count the same 433 people (407 GitHub accounts and the commit authors whose email is not linked to an account), where the website had counted accounts only. `DEPENDENCY_SECURITY.md` moved into `docs/dependency-maintenance.md`, and `.github/SUPPORT.md` says where to ask for help.
- **Website Plugins page:** The website's Plugins page now describes what the released version does, and a labelled section says what 0.10.3 adds: one Plugins place, Standard and Creator mode, and the Permissions control.
- **First-time guide on the website:** a new docs page, "Use Codewhale for the first time" (`/docs/first-time`, English and Simplified Chinese), walks someone who has never used a terminal through opening one, installing, choosing the folder Codewhale works in and when that choice matters, opening a terminal in a folder from Finder or File Explorer, connecting a model, a first task, and stopping, undoing and quitting. The install page gains a short "New to the terminal?" block for macOS and Windows that links to it, and `llms.txt` carries the same steps.
+50 -15
View File
@@ -47,6 +47,18 @@ impl Drop for TurnClaim {
}
}
}
fn admitted_call<'a>(
calls: &'a HashMap<String, PendingToolCall>,
execution: &str,
) -> Option<(&'a str, &'a PendingToolCall)> {
let mut id = execution;
loop {
if let Some((key, call)) = calls.get_key_value(id) {
return Some((key.as_str(), call));
}
id = id.rsplit_once('.')?.0;
}
}
fn update(session: &str, update: acp::SessionUpdate) -> Outgoing {
Outgoing::Update(acp::SessionNotification::new(session.to_string(), update))
}
@@ -419,27 +431,50 @@ impl PromptRun {
&& pending.turn_id == turn
&& pending.tool_call_id.as_deref() == Some(execution)
}) {
let call = calls
.get(execution)
let (shown, call) = admitted_call(calls, execution)
.ok_or_else(|| anyhow!("Approval projection lacks admitted tool start"))?;
let id = uuid::Uuid::new_v4().to_string();
let title = tool_call_title(call);
let text = p
let nested = shown != execution;
let nested_call;
let shown_call = if nested {
nested_call = PendingToolCall {
execution_id: execution.to_string(),
name: p
.get("tool_name")
.and_then(Value::as_str)
.unwrap_or(&call.name)
.to_string(),
input: p.get("input").cloned().unwrap_or(Value::Null),
};
&nested_call
} else {
call
};
let title = tool_call_title(shown_call);
let description = p
.get("description")
.and_then(Value::as_str)
.filter(|text| !text.trim().is_empty())
.map_or_else(|| title.clone(), str::to_string);
.filter(|text| !text.trim().is_empty());
let summary = p
.get("summary")
.and_then(Value::as_str)
.filter(|text| nested && !text.trim().is_empty());
let text = match (summary, description) {
(Some(summary), Some(description)) => format!("{summary}\n{description}"),
(Some(text), None) | (None, Some(text)) => text.to_string(),
(None, None) => title.clone(),
};
let mut fields = acp::ToolCallUpdateFields::new()
.title(title)
.kind(tool_call_kind(shown_call))
.raw_input((!shown_call.input.is_null()).then(|| shown_call.input.clone()))
.content(vec![acp::ToolCallContent::from(text)]);
if !nested {
fields = fields.status(acp::ToolCallStatus::Pending);
}
let request = acp::RequestPermissionRequest::new(
session.to_string(),
acp::ToolCallUpdate::new(
execution.to_string(),
acp::ToolCallUpdateFields::new()
.title(title)
.kind(tool_call_kind(call))
.status(acp::ToolCallStatus::Pending)
.raw_input((!call.input.is_null()).then(|| call.input.clone()))
.content(vec![acp::ToolCallContent::from(text)]),
),
acp::ToolCallUpdate::new(shown.to_string(), fields),
vec![
acp::PermissionOption::new(
"allow-once",
-4
View File
@@ -3115,10 +3115,6 @@ fn write_exec_stdout(text: &str) -> Result<()> {
.and_then(|()| stdout.flush())
{
Err(err) if err.kind() == io::ErrorKind::BrokenPipe => {
let _ = codewhale_telemetry::report_termination(
codewhale_telemetry::ExitClass::Signal,
codewhale_telemetry::SIGNAL_FLUSH_TIMEOUT,
);
// SAFETY: restores the default disposition and re-raises the
// signal the write would have delivered without SIG_IGN.
#[cfg(unix)]
+12
View File
@@ -129,6 +129,7 @@ mod members;
mod memory_lens;
mod mobile;
mod plans;
mod plugin_index;
mod plugins;
pub(crate) mod review;
mod secrets;
@@ -3372,6 +3373,7 @@ pub fn build_router(state: RuntimeApiState) -> Router {
.route("/v1/skills/{name}/audit", get(audit_skill_api))
.route("/v1/apps/mcp/tools", get(list_mcp_tools))
.route("/v1/apps/plugins", get(plugins::list_plugins))
.route("/v1/apps/plugin-index", get(plugin_index::get_plugin_index))
.route("/v1/apps/avatars", get(avatars::list_avatars))
.route(
"/v1/apps/avatars/{handle}/atlases/{page}",
@@ -5875,6 +5877,10 @@ struct CommandCatalogEntry {
#[serde(skip_serializing_if = "Option::is_none")]
origin: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
plugin_id: Option<String>,
#[serde(skip_serializing_if = "std::ops::Not::not")]
workspace_sourced: bool,
#[serde(skip_serializing_if = "Option::is_none")]
registration_id: Option<String>,
}
@@ -5933,6 +5939,8 @@ fn command_catalog(
shadowed_by,
shadowed_aliases,
origin: None,
plugin_id: None,
workspace_sourced: false,
registration_id: None,
});
}
@@ -5956,6 +5964,8 @@ fn command_catalog(
shadowed_by: None,
shadowed_aliases: Vec::new(),
origin: None,
plugin_id: command.plugin_id.clone(),
workspace_sourced: command.workspace_sourced,
registration_id: None,
});
}
@@ -6038,6 +6048,8 @@ async fn list_commands(
shadowed_by: None,
shadowed_aliases: Vec::new(),
origin: Some(reference.origin.clone()),
plugin_id: Some(reference.plugin_id.clone()),
workspace_sourced: false,
registration_id: query
.thread_id
.as_deref()
@@ -0,0 +1,489 @@
use std::collections::{BTreeSet, HashMap};
use std::path::{Path as FsPath, PathBuf};
use axum::Json;
use axum::extract::{Query, State};
use serde::Serialize;
use serde_json::Value;
use super::plugins::{self, PluginSummaryEntry};
use super::{ApiError, CommandsQuery, HooksQuery, RuntimeApiState};
const MAX_ADDED_NAMES: usize = 512;
#[derive(Debug, Serialize)]
pub(super) struct PluginIndexResponse {
workspace: String,
items: Vec<IndexItem>,
unavailable: Vec<IndexNote>,
problems: Vec<IndexNote>,
}
#[derive(Debug, Serialize)]
struct IndexNote {
kind: &'static str,
message: String,
}
#[derive(Debug, Serialize)]
struct IndexItem {
id: String,
kind: &'static str,
name: String,
description: Option<String>,
source: &'static str,
state: &'static str,
labels: Vec<&'static str>,
#[serde(skip_serializing_if = "Option::is_none")]
plugin: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
adds: Option<PluginAdds>,
#[serde(skip_serializing_if = "Option::is_none")]
tier: Option<&'static str>,
owner: &'static str,
row: Value,
}
#[derive(Debug, Default, Serialize)]
struct PluginAdds {
skills: Vec<String>,
connectors: Vec<String>,
hooks: Vec<String>,
commands: Vec<String>,
profiles: Vec<String>,
}
impl PluginAdds {
fn labels(&self, entry: &PluginSummaryEntry) -> Vec<&'static str> {
[
("skill", !self.skills.is_empty()),
("connector", !self.connectors.is_empty()),
("hook", !self.hooks.is_empty()),
("command", !self.commands.is_empty()),
(
"tool",
entry.inventory.native > 0 || entry.inventory.lsp > 0,
),
("profile", !self.profiles.is_empty()),
]
.into_iter()
.filter_map(|(label, present)| present.then_some(label))
.collect()
}
}
fn row_value<T: Serialize>(row: &T) -> Value {
serde_json::to_value(row).unwrap_or(Value::Null)
}
fn note_failure(unavailable: &mut Vec<IndexNote>, kind: &'static str, error: &ApiError) {
unavailable.push(IndexNote {
kind,
message: error.message.clone(),
});
}
fn plugin_source(origin: &str) -> &'static str {
match origin {
"codewhale-builtin" => "built_in",
"workspace-codewhale" => "project",
_ => "user",
}
}
fn plugin_state(entry: &PluginSummaryEntry) -> &'static str {
if matches!(
entry.trust_status,
"content-changed" | "capabilities-changed"
) || entry.state == "enabled-untrusted"
{
return "needs_review";
}
match entry.state {
"active" => "enabled",
"disabled" => "disabled",
_ => "unavailable",
}
}
fn add_stem(names: &mut BTreeSet<String>, path: &FsPath, extension: Option<&str>, lowercase: bool) {
if let Some(wanted) = extension
&& path.extension().and_then(|value| value.to_str()) != Some(wanted)
{
return;
}
if let Some(stem) = path.file_stem().and_then(|value| value.to_str()) {
names.insert(if lowercase {
stem.to_lowercase()
} else {
stem.to_string()
});
}
}
async fn component_names(
paths: &[PathBuf],
extension: Option<&str>,
lowercase: bool,
omitted: &mut bool,
) -> Vec<String> {
let mut names = BTreeSet::new();
for path in paths {
let Ok(meta) = tokio::fs::symlink_metadata(path).await else {
continue;
};
if meta.is_file() {
add_stem(&mut names, path, extension, lowercase);
} else if meta.is_dir()
&& let Ok(mut entries) = tokio::fs::read_dir(path).await
{
let mut seen = 0;
while let Ok(Some(entry)) = entries.next_entry().await {
seen += 1;
if seen > MAX_ADDED_NAMES {
*omitted = true;
break;
}
if entry.file_type().await.is_ok_and(|kind| kind.is_file()) {
add_stem(&mut names, &entry.path(), extension, lowercase);
}
}
}
if names.len() >= MAX_ADDED_NAMES {
*omitted = true;
break;
}
}
if names.iter().any(|name| name.chars().any(char::is_control)) {
*omitted = true;
names.retain(|name| !name.chars().any(char::is_control));
}
names.into_iter().take(MAX_ADDED_NAMES).collect()
}
async fn plugin_adds(
plugin: &codewhale_plugins::types::LoadedPlugin,
omitted: &mut bool,
) -> PluginAdds {
let mut skills: Vec<String> = plugin
.skill_snapshots
.iter()
.map(|skill| skill.name.clone())
.collect();
skills.sort();
let mut connectors: Vec<String> = plugin
.manifest
.mcp_servers
.as_ref()
.map(|servers| servers.keys().cloned().collect())
.unwrap_or_default();
connectors.sort();
PluginAdds {
skills,
connectors,
hooks: component_names(&plugin.components.hooks, None, false, omitted).await,
commands: component_names(&plugin.components.commands, Some("md"), true, omitted).await,
profiles: component_names(&plugin.components.agents, Some("toml"), false, omitted).await,
}
}
async fn plugin_adds_by_id(
state: &RuntimeApiState,
omitted: &mut BTreeSet<String>,
) -> HashMap<String, PluginAdds> {
let registry = state
.plugin_discovery
.registry_for_workspace(&state.workspace);
let mut adds = HashMap::new();
for plugin in registry.list() {
let mut cut = false;
adds.insert(
plugin.id.as_str().to_string(),
plugin_adds(plugin, &mut cut).await,
);
if cut {
omitted.insert(plugin.name().to_string());
}
}
adds
}
fn skill_source(
path: Option<&PathBuf>,
is_bundled: bool,
skills_dir: &FsPath,
workspace: &FsPath,
) -> &'static str {
if is_bundled {
return "built_in";
}
match path {
Some(path) if path.starts_with(skills_dir) => "user",
Some(path) if path.starts_with(workspace) => "project",
Some(_) => "other_app",
None => "plugin",
}
}
pub(super) async fn get_plugin_index(
State(state): State<RuntimeApiState>,
) -> Result<Json<PluginIndexResponse>, ApiError> {
let mut items = Vec::new();
let mut unavailable = Vec::new();
let mut problems = Vec::new();
let commands = super::list_commands(
State(state.clone()),
Query(CommandsQuery { thread_id: None }),
)
.await;
let mut plugin_commands: HashMap<String, BTreeSet<String>> = HashMap::new();
if let Ok(Json(response)) = &commands {
for command in &response.commands {
if let (Some(plugin_id), "user") = (command.plugin_id.as_ref(), command.kind) {
plugin_commands
.entry(plugin_id.clone())
.or_default()
.insert(command.name.clone());
}
}
}
match plugins::list_plugins(State(state.clone())).await {
Ok(Json(response)) => {
let mut omitted = BTreeSet::new();
let mut adds_by_id = plugin_adds_by_id(&state, &mut omitted).await;
for name in &omitted {
problems.push(IndexNote {
kind: "plugin",
message: format!(
"plugin '{name}' adds more files than are listed, or a file name with control characters; some names are left out of `adds`"
),
});
}
for diagnostic in &response.registry_diagnostics {
problems.push(IndexNote {
kind: "plugin",
message: diagnostic.message.clone(),
});
}
for entry in &response.plugins {
let mut adds = adds_by_id.remove(&entry.id);
if entry.state == "active"
&& commands.is_ok()
&& let Some(adds) = adds.as_mut()
{
adds.commands = plugin_commands
.remove(&entry.id)
.map(|names| names.into_iter().collect())
.unwrap_or_default();
}
items.push(IndexItem {
id: format!("plugin:{}", entry.id),
kind: "plugin",
name: entry
.display_name
.clone()
.unwrap_or_else(|| entry.name.clone()),
description: entry.description.clone(),
source: plugin_source(entry.origin),
state: plugin_state(entry),
labels: adds
.as_ref()
.map(|adds| adds.labels(entry))
.unwrap_or_default(),
plugin: None,
adds,
tier: None,
owner: "/v1/apps/plugins",
row: row_value(entry),
});
}
}
Err(error) => note_failure(&mut unavailable, "plugin", &error),
}
match plugins::list_plugin_catalog(State(state.clone())).await {
Ok(Json(catalog)) => {
for (tier, rows) in [
("builtin", &catalog.builtin),
("experimental", &catalog.experimental),
("more", &catalog.more),
] {
for row in rows {
items.push(IndexItem {
id: format!("plugin:catalog:{}", row.id),
kind: "plugin",
name: row.title.clone(),
description: row.description.clone(),
source: "built_in",
state: if !row.available {
"unavailable"
} else if row.enabled {
"enabled"
} else {
"disabled"
},
labels: Vec::new(),
plugin: None,
adds: None,
tier: Some(tier),
owner: "/v1/apps/plugins/catalog",
row: row_value(row),
});
}
}
}
Err(error) => note_failure(&mut unavailable, "plugin", &error),
}
match super::list_skills(State(state.clone())).await {
Ok(Json(response)) => {
for warning in &response.warnings {
problems.push(IndexNote {
kind: "skill",
message: warning.clone(),
});
}
for skill in &response.skills {
if skill.plugin_id.is_some() {
continue;
}
items.push(IndexItem {
id: format!("skill:{}", skill.name),
kind: "skill",
name: skill.name.clone(),
description: Some(skill.description.clone()),
source: skill_source(
skill.path.as_ref(),
skill.is_bundled,
&response.directory,
&state.workspace,
),
state: if skill.enabled { "enabled" } else { "disabled" },
labels: vec!["skill"],
plugin: None,
adds: None,
tier: None,
owner: "/v1/skills",
row: row_value(skill),
});
}
}
Err(error) => note_failure(&mut unavailable, "skill", &error),
}
match super::list_mcp_servers(State(state.clone())).await {
Ok(Json(response)) => {
for server in &response.servers {
let source = match server.origin {
"plugin" => continue,
"project" => "project",
"runtime" => "runtime",
_ => "user",
};
items.push(IndexItem {
id: format!("connector:{}", server.name),
kind: "connector",
name: server.name.clone(),
description: None,
source,
state: if !server.enabled {
"disabled"
} else if server.auth_required {
"needs_sign_in"
} else {
"enabled"
},
labels: vec!["connector"],
plugin: None,
adds: None,
tier: None,
owner: "/v1/apps/mcp/servers",
row: row_value(server),
});
}
}
Err(error) => note_failure(&mut unavailable, "connector", &error),
}
match super::list_hooks(State(state.clone()), Query(HooksQuery { thread_id: None })).await {
Ok(Json(response)) => {
for problem in &response.problems {
problems.push(IndexNote {
kind: "hook",
message: problem.clone(),
});
}
let mut ordinals: HashMap<&'static str, usize> = HashMap::new();
for hook in &response.hooks {
let ordinal = ordinals.entry(hook.event).or_default();
*ordinal += 1;
let source = match hook.source {
"plugin" => continue,
"project" => "project",
_ => "user",
};
let name = hook
.name
.clone()
.unwrap_or_else(|| format!("{} {}", hook.event, ordinal));
items.push(IndexItem {
id: format!("hook:{source}:{}:{name}", hook.event),
kind: "hook",
name,
description: None,
source,
state: if response.enabled {
"enabled"
} else {
"disabled"
},
labels: vec!["hook"],
plugin: None,
adds: None,
tier: None,
owner: "/v1/hooks",
row: row_value(hook),
});
}
}
Err(error) => note_failure(&mut unavailable, "hook", &error),
}
match commands {
Ok(Json(response)) => {
for command in &response.commands {
let (source, plugin) = match (command.kind, command.plugin_id.as_ref()) {
("builtin", _) => continue,
("user", Some(_)) => continue,
("extension", plugin_id) => ("plugin", plugin_id.cloned()),
_ if command.workspace_sourced => ("project", None),
_ => ("user", None),
};
items.push(IndexItem {
id: format!("command:{}", command.name),
kind: "command",
name: command.name.clone(),
description: command.summary.clone(),
source,
state: "enabled",
labels: vec!["command"],
plugin,
adds: None,
tier: None,
owner: "/v1/commands",
row: row_value(command),
});
}
}
Err(error) => note_failure(&mut unavailable, "command", &error),
}
Ok(Json(PluginIndexResponse {
workspace: state.workspace.display().to_string(),
items,
unavailable,
problems,
}))
}
@@ -15,6 +15,7 @@ use tokio::sync::{Mutex, mpsc, oneshot};
use tokio::time::sleep;
use uuid::Uuid;
mod catalog_parity;
mod command_catalog;
#[cfg(unix)]
mod control_socket_selection;
@@ -28,6 +29,7 @@ mod headless_catalog;
mod member_operations;
#[cfg(unix)]
mod owner_bootstrap;
mod plugin_index;
mod plugin_ui;
mod profile_constitution;
#[cfg(any(unix, windows))]
@@ -0,0 +1,600 @@
use super::*;
use codewhale_config::catalog::{
CatalogOffering, CatalogSource, CatalogStatus, ProviderCatalogDelta, base_url_fingerprint,
now_unix,
};
use codewhale_engine::cost_status::EffectiveRouteEnvelope;
use codewhale_engine::model_inventory::ModelInventory;
use codewhale_runtime::config::ProviderIdentity;
use sha2::{Digest, Sha256};
use std::collections::BTreeMap;
const SINGULAR: &str = "model list / model resolve";
const PLURAL: &str = "models";
const RUNTIME: &str = "runtime api";
const ROUTING: &str = "routing inventory";
const COST: &str = "cost";
const UNKNOWN: &str = "unknown";
const ROSTER: &[&str] = &["claude-opus-5", "claude-haiku-4-5", "claude-parity-probe-1"];
type Readings = BTreeMap<(String, String, &'static str), BTreeMap<&'static str, String>>;
struct Route {
label: &'static str,
api_path: &'static str,
identity: ProviderIdentity,
base_url: String,
models: &'static [&'static str],
}
fn cold_process() {
codewhale_engine::provider_catalog_live::reset_cache_for_test();
codewhale_engine::provider_lake::clear_live_snapshot();
}
fn known<T: ToString>(value: Option<T>) -> String {
value.map_or_else(|| UNKNOWN.to_string(), |value| value.to_string())
}
fn rate(value: &Value) -> String {
let number = match value {
Value::Number(number) => number.as_f64(),
Value::String(text) => text.parse::<f64>().ok(),
_ => None,
};
known(number)
}
fn roster(mut models: Vec<String>) -> String {
models.sort();
models.dedup();
models.join(",")
}
fn read(
readings: &mut Readings,
route: &Route,
model: &str,
field: &'static str,
surface: &'static str,
value: String,
) {
readings
.entry((route.label.to_string(), model.to_string(), field))
.or_default()
.insert(surface, value);
}
fn persist_id_only_roster(identity: &ProviderIdentity, base_url: &str) {
let fingerprint = base_url_fingerprint(base_url);
let fetched_at = now_unix();
let ticket = codewhale_engine::provider_catalog_live::begin_refresh_for_identity(
identity.provider,
identity.key.as_str(),
base_url,
);
assert_eq!(
codewhale_engine::provider_catalog_live::record_success_if_current(
&ticket,
ProviderCatalogDelta {
provider: identity.key.to_string(),
base_url_fingerprint: fingerprint.clone(),
fetched_at,
offerings: ROSTER
.iter()
.map(|id| CatalogOffering {
provider: identity.key.to_string(),
wire_model_id: (*id).to_string(),
endpoint_key: "chat".into(),
source: CatalogSource::Live {
base_url_fingerprint: fingerprint.clone(),
fetched_at,
},
..Default::default()
})
.collect(),
},
),
Some(CatalogStatus::Fresh)
);
}
fn read_singular(readings: &mut Readings, config_path: &Path, route: &Route) -> Result<()> {
cold_process();
let store = codewhale_config::ConfigStore::load(Some(config_path.to_path_buf()))?;
let listed =
codewhale_engine::route_preferences::list_models(&store, Some(route.identity.provider))?
.into_iter()
.find(|(listed, _)| listed == route.identity.key.as_str())
.map(|(_, models)| roster(models));
read(
readings,
route,
"*",
"roster",
SINGULAR,
listed.unwrap_or_else(|| "route not listed".into()),
);
for model in route.models {
let limits = codewhale_engine::route_preferences::resolve_model(
config_path,
route.identity.key.as_str(),
&route.base_url,
Some(model),
)
.map(|candidate| candidate.limits());
let (context, output) = match limits {
Ok(limits) => (known(limits.context_tokens), known(limits.output_tokens)),
Err(error) => (format!("refused: {error}"), format!("refused: {error}")),
};
read(readings, route, model, "limit.context", SINGULAR, context);
read(readings, route, model, "limit.output", SINGULAR, output);
}
Ok(())
}
fn read_plural(readings: &mut Readings, config: &Config, route: &Route) -> Result<()> {
cold_process();
codewhale_engine::models_dev_live::maybe_load_persisted_cache();
let mut route_config = config.clone();
route_config
.scope_to_provider_identity(&route.identity)
.map_err(anyhow::Error::msg)?;
read(
readings,
route,
"*",
"roster",
PLURAL,
roster(codewhale_engine::provider_lake::listed_models_for_route(
&route_config,
&route.identity,
)),
);
Ok(())
}
async fn read_runtime(
readings: &mut Readings,
client: &reqwest::Client,
addr: &SocketAddr,
route: &Route,
) -> Result<()> {
let body: Value = client
.get(format!("http://{addr}/v1/providers/{}", route.api_path))
.send()
.await?
.error_for_status()?
.json()
.await?;
let models = body["models"].as_array().context("models array")?;
read(
readings,
route,
"*",
"roster",
RUNTIME,
roster(
models
.iter()
.filter_map(|model| model["id"].as_str().map(str::to_string))
.collect(),
),
);
for model in route.models {
let facts = models
.iter()
.find(|entry| entry["id"] == *model)
.map_or(&Value::Null, |entry| &entry["facts"]);
read(
readings,
route,
model,
"limit.context",
RUNTIME,
known(facts["effective_limits"]["context_tokens"].as_u64()),
);
read(
readings,
route,
model,
"limit.output",
RUNTIME,
known(facts["effective_limits"]["output_tokens"].as_u64()),
);
read(
readings,
route,
model,
"window",
RUNTIME,
known(facts["effective_context"]["tokens"].as_u64()),
);
read(
readings,
route,
model,
"window.source",
RUNTIME,
known(facts["effective_context"]["source"].as_str()),
);
read(
readings,
route,
model,
"price.input",
RUNTIME,
rate(&facts["pricing"]["input_per_million"]),
);
read(
readings,
route,
model,
"price.output",
RUNTIME,
rate(&facts["pricing"]["output_per_million"]),
);
read(
readings,
route,
model,
"price.basis",
RUNTIME,
format!(
"provenance={} status={} source={}",
facts["pricing"]["provenance"], facts["pricing_status"], facts["source"]
),
);
}
Ok(())
}
fn read_routing(readings: &mut Readings, config: &Config, route: &Route) -> Result<()> {
let inventory = ModelInventory::from_config(config).map_err(anyhow::Error::msg)?;
let candidates: Vec<_> = inventory
.candidates
.iter()
.filter(|candidate| candidate.identity == route.identity)
.collect();
read(
readings,
route,
"*",
"roster",
ROUTING,
roster(
candidates
.iter()
.map(|candidate| candidate.model.clone())
.collect(),
),
);
for model in route.models {
let candidate = candidates
.iter()
.find(|candidate| candidate.model == *model);
read(
readings,
route,
model,
"window",
ROUTING,
known(candidate.map(|candidate| candidate.context_window)),
);
read(
readings,
route,
model,
"window.source",
ROUTING,
known(candidate.map(|candidate| {
if candidate.context_window_unverified {
"unverified"
} else {
"verified"
}
})),
);
read(
readings,
route,
model,
"limit.output",
ROUTING,
known(candidate.and_then(|candidate| candidate.max_output)),
);
}
Ok(())
}
fn read_cost(readings: &mut Readings, config: &Config, route: &Route) {
let usage = Usage {
input_tokens: 100_000,
output_tokens: 10_000,
..Default::default()
};
for model in route.models {
let envelope = EffectiveRouteEnvelope::from_admitted(
Some(config),
&route.identity,
*model,
Some(&route.base_url),
chrono::Utc::now(),
);
let quote = envelope.provider_live_pricing.as_ref();
let audit = envelope.audit(&usage);
read(
readings,
route,
model,
"price.input",
COST,
quote
.and_then(|quote| quote.input_per_million.as_deref())
.map_or_else(|| UNKNOWN.to_string(), |text| rate(&json!(text))),
);
read(
readings,
route,
model,
"price.output",
COST,
quote
.and_then(|quote| quote.output_per_million.as_deref())
.map_or_else(|| UNKNOWN.to_string(), |text| rate(&json!(text))),
);
read(
readings,
route,
model,
"price.basis",
COST,
format!(
"provenance={:?} billing={:?} charge_usd={} unpriced={:?}",
quote.map(|quote| quote.provenance.clone()),
envelope.billing_mode,
known(audit.exact_usd.as_deref()),
audit.unpriced_reason,
),
);
}
}
#[tokio::test]
async fn one_captured_catalog_reads_the_same_on_every_model_surface() -> Result<()> {
let _env = lock_test_env();
let _live = codewhale_engine::provider_lake::lock_live_snapshot();
let root = tempfile::tempdir()?;
let captured = codewhale_config::catalog::BUNDLED_MODELS_DEV_JSON;
let catalog_path = root.path().join("captured-models-dev.json");
fs::write(&catalog_path, captured)?;
let _catalog = EnvVarGuard::set("CODEWHALE_MODELS_DEV_PATH", &catalog_path);
let _home = EnvVarGuard::set("CODEWHALE_HOME", root.path());
let _config_path = EnvVarGuard::remove("CODEWHALE_CONFIG_PATH");
let _legacy_config_path = EnvVarGuard::remove("DEEPSEEK_CONFIG_PATH");
let _cli = EnvVarGuard::remove(codewhale_config::CLI_API_KEY_ENV);
cold_process();
let config_path = root.path().join("config.toml");
fs::write(
&config_path,
r#"provider = "anthropic"
[providers.anthropic]
api_key = "parity-fixture-key"
model = "claude-opus-5"
[providers.zai]
api_key = "parity-fixture-key"
model = "GLM-5.3"
[providers.lab]
kind = "openai-compatible"
base_url = "http://127.0.0.1:9/v1"
api_key = "parity-fixture-key"
model = "claude-haiku-4-5"
"#,
)?;
let config = Config::load(Some(config_path.clone()), None)?;
let route = |label: &'static str,
api_path: &'static str,
key: &str,
models: &'static [&'static str]|
-> Result<Route> {
let identity = config
.resolve_provider_pin_identity(key)
.map_err(anyhow::Error::msg)?;
Ok(Route {
label,
api_path,
base_url: config.base_url_for_route(&identity),
identity,
models,
})
};
let routes = [
route(
"anthropic (official, id-only roster)",
"anthropic/models",
"anthropic",
ROSTER,
)?,
route(
"lab (custom endpoint, same spelling)",
"custom/models?model_provider_id=lab",
"lab",
&["claude-haiku-4-5"],
)?,
route(
"zai (plan endpoint)",
"zai/models",
"zai",
&["GLM-5.3", "GLM-5.2"],
)?,
];
assert!(
codewhale_engine::models_dev_live::refresh(false)
.await
.map_err(|error| anyhow::anyhow!("{error}"))?
> 0
);
persist_id_only_roster(&routes[0].identity, &routes[0].base_url);
let mut readings = Readings::new();
for route in &routes {
read_singular(&mut readings, &config_path, route)?;
read_plural(&mut readings, &config, route)?;
}
cold_process();
let (addr, _, handle) = spawn_test_server_with_config_path(config_path.clone())
.await?
.expect("loopback server required");
let client = codewhale_release::tls::reqwest_client();
for route in &routes {
read_runtime(&mut readings, &client, &addr, route).await?;
read_routing(&mut readings, &config, route)?;
read_cost(&mut readings, &config, route);
}
handle.abort();
cold_process();
println!(
"PARITY captured_catalog sha256={} bytes={}",
Sha256::digest(captured.as_bytes())
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>(),
captured.len()
);
let mut differing = Vec::new();
let mut known_differences: Vec<String> = Vec::new();
for ((route, model, field), surfaces) in &readings {
let mut values: Vec<String> = surfaces
.values()
.map(|value| {
if *field == "roster" {
roster(
value
.to_ascii_lowercase()
.split(',')
.map(str::to_string)
.collect(),
)
} else {
value.clone()
}
})
.collect();
values.sort();
values.dedup();
let compared = !matches!(*field, "price.basis" | "window.source");
let borrowed_window = *field == "window"
&& surfaces.get(RUNTIME).map(String::as_str) == Some(UNKNOWN)
&& readings
.get(&(route.clone(), model.clone(), "window.source"))
.and_then(|sources| sources.get(ROUTING))
.map(String::as_str)
== Some("unverified");
let verdict = if !compared {
"info"
} else if borrowed_window {
known_differences.push(format!("{route} | {model} | {surfaces:?}"));
"KNOWN-DIFFERENCE"
} else if values.len() == 1 {
"agree"
} else {
differing.push(format!("{route} | {model} | {field} | {surfaces:?}"));
"DIFFER"
};
println!("PARITY {verdict} | {route} | {model} | {field} | {surfaces:?}");
}
let value = |route: &Route, model: &str, field: &'static str, surface: &'static str| {
readings
.get(&(route.label.to_string(), model.to_string(), field))
.and_then(|surfaces| surfaces.get(surface))
.cloned()
.unwrap_or_default()
};
assert_eq!(
value(&routes[0], "*", "roster", SINGULAR),
"claude-haiku-4-5,claude-opus-5,claude-parity-probe-1",
"the singular command lists the route's own roster, not every bundled row"
);
assert_eq!(
value(&routes[0], "claude-opus-5", "limit.context", RUNTIME),
"1000000"
);
assert_eq!(
value(&routes[0], "claude-haiku-4-5", "limit.context", RUNTIME),
"200000"
);
assert_eq!(value(&routes[0], "claude-opus-5", "price.input", COST), "5");
assert_eq!(
value(&routes[0], "claude-haiku-4-5", "price.output", COST),
"5"
);
for field in ["limit.context", "limit.output", "price.input"] {
assert_eq!(
value(&routes[1], "claude-haiku-4-5", field, RUNTIME),
UNKNOWN,
"a custom endpoint borrows no fact from a same-named model: {field}"
);
}
assert_eq!(
value(&routes[1], "claude-haiku-4-5", "window.source", ROUTING),
"unverified",
"routing never presents a same-named model's window as this endpoint's"
);
assert_eq!(
value(&routes[1], "claude-haiku-4-5", "limit.output", ROUTING),
UNKNOWN
);
for model in ["GLM-5.3", "GLM-5.2"] {
let basis = value(&routes[2], model, "price.basis", COST);
assert!(
basis.contains("NotMoneyMetered") && basis.contains("charge_usd=unknown"),
"a plan route is never billed per token: {model}: {basis}"
);
}
for (model, rate) in [("GLM-5.3", UNKNOWN), ("GLM-5.2", "1.4")] {
for surface in [COST, RUNTIME] {
assert_eq!(
value(&routes[2], model, "price.input", surface),
rate,
"the list rate is shown the same everywhere and never turned into a charge: {model} {surface}"
);
}
}
let basis = value(&routes[1], "claude-haiku-4-5", "price.basis", COST);
assert!(
basis.contains("UnknownBillingBasis") && basis.contains("charge_usd=unknown"),
"a custom endpoint is never given a charge: {basis}"
);
for surface in [COST, RUNTIME] {
assert_eq!(
value(&routes[0], "claude-parity-probe-1", "price.input", surface),
UNKNOWN,
"a metered first-party model with no price row reads unknown: {surface}"
);
}
let basis = value(&routes[0], "claude-parity-probe-1", "price.basis", COST);
assert!(
basis.contains("charge_usd=unknown") && !basis.contains("unpriced=None"),
"an unpriced model produces no charge and says why: {basis}"
);
assert_eq!(
known_differences,
vec![
"lab (custom endpoint, same spelling) | claude-haiku-4-5 | {\"routing inventory\": \"200000\", \"runtime api\": \"unknown\"}"
],
"the only named difference is the custom endpoint's borrowed routing window"
);
assert!(
differing.is_empty(),
"surfaces disagree on the same captured catalog:\n{}",
differing.join("\n")
);
Ok(())
}
@@ -28,6 +28,8 @@ fn user(name: &str) -> UserCommand {
takes_arguments: false,
hidden: false,
extension: false,
plugin_id: None,
workspace_sourced: false,
builtin_bindings: vec![name.to_string()],
}
}
@@ -0,0 +1,139 @@
use super::*;
use codewhale_engine::command_catalog::{BuiltinCommand, CommandCatalog, UserCommand};
struct Catalog(Arc<std::sync::Mutex<Option<String>>>);
impl CommandCatalog for Catalog {
fn answers_to(&self, _name: &str) -> bool {
false
}
fn builtins(&self) -> Vec<BuiltinCommand> {
Vec::new()
}
fn user_commands(&self, _workspace: &Path) -> Vec<UserCommand> {
let plugin_id = self.0.lock().expect("plugin id").clone();
let command = |name: &str, plugin: Option<String>| UserCommand {
name: name.to_string(),
aliases: Vec::new(),
description: Some("A command.".to_string()),
usage: None,
takes_arguments: false,
hidden: false,
extension: false,
plugin_id: plugin,
workspace_sourced: false,
builtin_bindings: Vec::new(),
};
vec![command("mine", None), command("ship", plugin_id)]
}
}
#[tokio::test]
async fn the_index_lists_every_kind_once_and_a_plugin_carries_what_it_adds() -> Result<()> {
let _env = lock_test_env();
let temp = tempfile::tempdir()?;
let root = temp.path().join("index-route");
let workspace = root.join("workspace");
fs::create_dir_all(&workspace)?;
let plugin_id = Arc::new(std::sync::Mutex::new(None));
let _catalog = codewhale_engine::command_catalog::CatalogGuard::install(Arc::new(Catalog(
plugin_id.clone(),
)));
let sources = root.join("sources");
fs::create_dir_all(&sources)?;
let bundle = write_plugin_source_bundle(&sources, "indexed")?;
fs::create_dir_all(bundle.join("commands"))?;
fs::write(bundle.join("commands/ship.md"), "Ship it.\n")?;
fs::create_dir_all(bundle.join("agents"))?;
fs::write(bundle.join("agents/reviewer.toml"), "name = \"reviewer\"\n")?;
fs::write(
bundle.join("plugin.toml"),
"schema_version = 1\n[plugin]\nname = \"indexed\"\nversion = \"1.0.0\"\n\
description = \"test bundle\"\n\n[skills]\npath = \"skills\"\n\
[commands]\npath = \"commands\"\n[agents]\npath = \"agents\"\n",
)?;
let Some((addr, handle)) = spawn_plugin_api_server(root, workspace).await? else {
return Ok(());
};
let client = codewhale_release::tls::reqwest_client();
plugin_ui::install_active(
&client,
&format!("http://{addr}/v1/apps/plugins"),
bundle.display().to_string(),
)
.await?;
let listed: Value = client
.get(format!("http://{addr}/v1/apps/plugins"))
.send()
.await?
.error_for_status()?
.json()
.await?;
let id = listed["plugins"]
.as_array()
.and_then(|plugins| plugins.iter().find(|plugin| plugin["name"] == "indexed"))
.and_then(|plugin| plugin["id"].as_str())
.expect("installed plugin id")
.to_string();
*plugin_id.lock().expect("plugin id") = Some(id.clone());
let commands: Value = client
.get(format!("http://{addr}/v1/commands"))
.send()
.await?
.error_for_status()?
.json()
.await?;
let ship = commands["commands"]
.as_array()
.and_then(|rows| rows.iter().find(|row| row["name"] == "ship"))
.expect("ship row");
assert_eq!(ship["plugin_id"], id.as_str());
let index: Value = client
.get(format!("http://{addr}/v1/apps/plugin-index"))
.send()
.await?
.error_for_status()?
.json()
.await?;
let items = index["items"].as_array().expect("items");
let find = |id: &str| items.iter().find(|item| item["id"] == id);
let plugin = items
.iter()
.find(|item| item["kind"] == "plugin" && item["row"]["name"] == "indexed")
.unwrap_or_else(|| panic!("plugin row in {index}"));
assert_eq!(plugin["kind"], "plugin");
assert_eq!(plugin["state"], "enabled");
assert_eq!(plugin["adds"]["skills"], serde_json::json!(["greet"]));
assert_eq!(plugin["adds"]["commands"], serde_json::json!(["ship"]));
assert_eq!(plugin["adds"]["profiles"], serde_json::json!(["reviewer"]));
assert_eq!(
plugin["labels"],
serde_json::json!(["skill", "command", "profile"])
);
assert_eq!(plugin["row"]["name"], "indexed");
assert!(find("skill:greet").is_none());
assert!(find("command:ship").is_none());
let mine = find("command:mine").expect("a hand-written command is a row");
assert_eq!(mine["source"], "user");
assert_eq!(mine["labels"], serde_json::json!(["command"]));
assert!(
items
.iter()
.any(|item| item["kind"] == "plugin" && item["tier"] == "builtin"),
"built-in switches are rows"
);
assert_eq!(
index["unavailable"].as_array().map(Vec::len),
Some(0),
"{index}"
);
handle.abort();
Ok(())
}
@@ -79,7 +79,7 @@ fn write_native_ui_bundle(dir: &Path, name: &str) -> Result<PathBuf> {
Ok(bundle)
}
async fn install_active(
pub(super) async fn install_active(
client: &reqwest::Client,
base: &str,
source: String,
+345 -2
View File
@@ -3,13 +3,13 @@ mod acp_support;
use acp_support::{Client, fixture_config, text_of, updates};
use anyhow::{Result, anyhow};
use codewhale_app_server::acp_server::capture_frontend;
use codewhale_engine::llm_client::mock::{MockLlmClient, canned};
use codewhale_engine::llm_client::mock::{CannedTurn, MockLlmClient, canned};
use codewhale_engine::runtime_threads::RuntimeTurnStatus;
use codewhale_engine::runtime_threads::manager::{
RuntimeThreadManager, RuntimeThreadManagerConfig,
};
use serde_json::{Value, json};
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
use tokio::io::{AsyncWriteExt, BufReader};
@@ -497,3 +497,346 @@ async fn extension_journey() -> Result<()> {
host.shutdown().await;
Ok(())
}
const GATES_MCP_FIXTURE: &str = r#"import readline from 'node:readline';
readline.createInterface({input:process.stdin}).on('line', line => {
const r = JSON.parse(line); if (r.id === undefined) return;
let result = {};
if (r.method === 'initialize') result = {protocolVersion:'2024-11-05',capabilities:{tools:{}},serverInfo:{name:'fixture',version:'1'}};
if (r.method === 'tools/list') result = {tools:['echo','hidden'].map(name => ({name,description:'Echo text',inputSchema:{type:'object',properties:{text:{type:'string'}},required:['text']}}))};
if (r.method === 'tools/call') result = {content:[{type:'text',text:'mcp-ran:' + r.params.name + ':' + r.params.arguments.text}]};
process.stdout.write(JSON.stringify({jsonrpc:'2.0',id:r.id,result}) + '\n');
});
"#;
struct Outcome {
response: Value,
frames: Vec<Value>,
asked: Vec<Value>,
}
impl Outcome {
fn rendered(&self) -> String {
serde_json::to_string(&self.frames).unwrap_or_default()
}
fn completed(&self) -> usize {
updates(&self.frames, "tool_call_update")
.iter()
.filter(|frame| frame.pointer("/params/update/status") == Some(&json!("completed")))
.count()
}
fn titles(&self) -> Vec<String> {
self.asked
.iter()
.map(|frame| {
frame["params"]["toolCall"]["title"]
.as_str()
.unwrap_or_default()
.to_string()
})
.collect()
}
fn announced_ids(&self) -> Vec<String> {
updates(&self.frames, "tool_call")
.iter()
.filter_map(|frame| frame.pointer("/params/update/toolCallId")?.as_str())
.map(str::to_string)
.collect()
}
}
async fn run_prompt(
client: &mut Client,
session: &str,
allow: &dyn Fn(usize) -> bool,
) -> Result<Outcome> {
client.next_id += 1;
let prompt_id = client.next_id;
client
.send(
json!({"jsonrpc":"2.0","id":prompt_id,"method":"session/prompt","params":{
"sessionId":session,"prompt":[{"type":"text","text":"go"}]}}),
)
.await?;
let mut frames = Vec::new();
let mut asked = Vec::new();
let response = loop {
let frame = client.recv().await?;
if frame["method"] == "session/request_permission" {
let option = if allow(asked.len()) {
"allow-once"
} else {
"reject-once"
};
client
.send(json!({"jsonrpc":"2.0","id":frame["id"],
"result":{"outcome":{"outcome":"selected","optionId":option}}}))
.await?;
asked.push(frame);
} else if frame["id"] == json!(prompt_id) && frame.get("method").is_none() {
break frame;
} else {
frames.push(frame);
}
};
Ok(Outcome {
response,
frames,
asked,
})
}
async fn new_session(
client: &mut Client,
manager: &RuntimeThreadManager,
workspace: &Path,
turns: Vec<CannedTurn>,
plan: bool,
) -> Result<String> {
manager.set_test_model_client(Arc::new(MockLlmClient::new(turns)));
let (created, _) = client
.request("session/new", json!({"cwd":workspace}))
.await?;
let session = created["result"]["sessionId"]
.as_str()
.ok_or_else(|| anyhow!("session/new returned no sessionId: {created}"))?
.to_string();
if plan {
let (set, _) = client
.request(
"session/set_mode",
json!({"sessionId":session,"modeId":"plan"}),
)
.await?;
assert!(set.get("error").is_none(), "{set}");
}
Ok(session)
}
#[test]
fn extension_tools_keep_the_permission_boundary_in_an_acp_session() -> Result<()> {
std::thread::Builder::new()
.stack_size(64 << 20)
.spawn(|| {
tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()?
.block_on(async {
tokio::time::timeout(Duration::from_secs(180), gates())
.await
.map_err(|_| anyhow!("extension gates exceeded 180s"))?
})
})?
.join()
.map_err(|_| anyhow!("extension gates thread panicked"))?
}
async fn gates() -> Result<()> {
use codewhale_engine::extension_host::test_support::{FixturePlugins, node_for_tests};
let Some(node) = node_for_tests("extension_tools_keep_the_permission_boundary") else {
return Ok(());
};
let _policy = codewhale_plugins::activation::TestPolicyGuard::extension_host(true);
let fixture = FixturePlugins::new(&["plugin-context", "core-call"]).await;
let host = fixture.manager(node.clone());
let _host = codewhale_engine::extension_host::TestManagerGuard::install(Arc::clone(&host));
let dir = tempfile::tempdir()?;
let _home = codewhale_engine::test_support::SealedHome::at(dir.path());
let workspace = fixture.workspace().to_path_buf();
std::fs::write(workspace.join("seed.txt"), "seed-content-xyz")?;
let server = dir.path().join("fixture-mcp.mjs");
std::fs::write(&server, GATES_MCP_FIXTURE)?;
let mcp_config = dir.path().join("mcp.json");
std::fs::write(
&mcp_config,
serde_json::to_vec(&json!({"servers":{
"fixture":{"command":node,"args":[server],"disabled_tools":["hidden"]},
"off":{"command":node,"args":[server],"disabled":true}
}}))?,
)?;
let runtime_dir = dir.path().join("runtime");
let mut config = fixture_config();
config.set_feature("mcp", true).unwrap();
config.set_feature("extension_host", true).unwrap();
config.mcp_config_path = Some(mcp_config.to_string_lossy().into_owned());
let manager = Arc::new(RuntimeThreadManager::open_host(
config.clone(),
workspace.clone(),
RuntimeThreadManagerConfig {
mcp_auth_context: Default::default(),
data_dir: runtime_dir.clone(),
task_data_dir: runtime_dir,
sessions_dir: None,
max_active_threads: 16,
},
fixture.registry(),
codewhale_engine::core::engine::EngineHostProfile::Acp,
None,
None,
)?);
let frontend = capture_frontend(
config,
"fixture-model".into(),
workspace.clone(),
Arc::clone(&manager),
codewhale_engine::session_manager::default_sessions_dir()?,
None,
None,
)?;
let (mut client, agent_read, agent_write) = Client::connect(false);
let connection = frontend.serve(Box::new(BufReader::new(agent_read)), Box::new(agent_write));
let manager_for_script = Arc::clone(&manager);
let script =
async {
let manager = manager_for_script;
let result = async {
client
.request(
"initialize",
json!({"protocolVersion":1,"clientCapabilities":{}}),
)
.await?;
let session = new_session(
&mut client,
&manager,
&workspace,
vec![
canned::tool_call_turn(
"n1",
"cc_call",
r#"{"name":"write","input":{"path":"nested.txt","content":"via-plugin"}}"#,
),
canned::simple_text_turn("nested allowed"),
],
false,
)
.await?;
let allowed = run_prompt(&mut client, &session, &|_| true).await?;
assert!(allowed.response.get("error").is_none(), "{}", allowed.response);
assert_eq!(allowed.response["result"]["stopReason"], "end_turn");
assert_eq!(allowed.asked.len(), 2, "{:?}", allowed.titles());
assert!(allowed.titles()[0].starts_with("cc_call"));
let announced = allowed.announced_ids();
for request in &allowed.asked {
let id = request["params"]["toolCall"]["toolCallId"]
.as_str()
.unwrap_or_default();
assert!(
announced.iter().any(|known| known == id),
"a nested approval is shown on a tool call the editor knows: {id} {announced:?}"
);
}
let nested = &allowed.asked[1]["params"]["toolCall"];
assert_eq!(nested["title"], "write: nested.txt", "{nested}");
assert_eq!(nested["kind"], "edit", "{nested}");
assert_eq!(nested["rawInput"]["path"], "nested.txt", "{nested}");
assert!(nested.get("status").is_none(), "{nested}");
assert_eq!(
std::fs::read_to_string(workspace.join("nested.txt"))?,
"via-plugin"
);
std::fs::remove_file(workspace.join("nested.txt"))?;
let session = new_session(
&mut client,
&manager,
&workspace,
vec![
canned::tool_call_turn(
"n2",
"cc_call",
r#"{"name":"write","input":{"path":"nested.txt","content":"via-plugin"}}"#,
),
canned::simple_text_turn("nested denied"),
],
false,
)
.await?;
let denied = run_prompt(&mut client, &session, &|index| index == 0).await?;
assert!(denied.response.get("error").is_none(), "{}", denied.response);
assert_eq!(denied.response["result"]["stopReason"], "end_turn");
assert_eq!(denied.asked.len(), 2, "{:?}", denied.titles());
assert!(
!workspace.join("nested.txt").exists(),
"a rejected nested write must not run"
);
let session = new_session(
&mut client,
&manager,
&workspace,
vec![
canned::tool_call_turn("r1", "ctx_probe", "{}"),
canned::tool_call_turn("r2", "mcp_fixture_echo", r#"{"text":"control"}"#),
canned::tool_call_turn("r3", "ctx_note", r#"{"text":"must-not-be-written"}"#),
canned::tool_call_turn("r4", "mcp_fixture_echo", r#"{"text":"rejected"}"#),
canned::simple_text_turn("rejections done"),
],
false,
)
.await?;
let rejected = run_prompt(&mut client, &session, &|index| index < 2).await?;
assert_eq!(rejected.asked.len(), 4, "{:?}", rejected.titles());
let rendered = rejected.rendered();
assert!(rendered.contains("mcp-ran:echo:control"), "{rendered}");
assert!(!rendered.contains("mcp-ran:echo:rejected"), "{rendered}");
assert!(!rendered.contains("note.txt"), "{rendered}");
let session = new_session(
&mut client,
&manager,
&workspace,
vec![
canned::tool_call_turn("p1", "ctx_probe", "{}"),
canned::tool_call_turn("p2", "mcp_fixture_echo", r#"{"text":"plan"}"#),
canned::simple_text_turn("plan done"),
],
true,
)
.await?;
let plan = run_prompt(&mut client, &session, &|_| true).await?;
assert!(plan.asked.is_empty(), "{:?}", plan.titles());
assert_eq!(plan.completed(), 0, "{}", plan.rendered());
assert!(!plan.rendered().contains("mcp-ran:"));
let session = new_session(
&mut client,
&manager,
&workspace,
vec![
canned::tool_call_turn("f1", "mcp_fixture_hidden", r#"{"text":"hid"}"#),
canned::tool_call_turn("f2", "mcp_off_echo", r#"{"text":"off"}"#),
canned::tool_call_turn("f3", "mcp_evil_echo", r#"{"text":"evil"}"#),
canned::tool_call_turn(
"f4",
"start_mcp_server",
r#"{"name":"evil","command":"node","args":["-e","0"]}"#,
),
canned::tool_call_turn("f5", "agent", r#"{"action":"start","prompt":"p"}"#),
canned::simple_text_turn("fabricated done"),
],
false,
)
.await?;
let fabricated = run_prompt(&mut client, &session, &|_| true).await?;
assert!(fabricated.asked.is_empty(), "{:?}", fabricated.titles());
assert_eq!(fabricated.completed(), 0, "{}", fabricated.rendered());
assert!(!fabricated.rendered().contains("mcp-ran:"));
client.request("shutdown", json!({})).await?;
Ok::<_, anyhow::Error>(())
}
.await;
if result.is_err() {
let _ = client.write.shutdown().await;
}
result
};
let (served, scripted) = tokio::join!(connection, script);
served?;
scripted?;
manager.shutdown_and_wait().await?;
host.shutdown().await;
Ok(())
}
@@ -293,17 +293,11 @@ fn dispatch_and_read_telemetry_with_endpoint(
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => panic!("read telemetry dry-run sink: {error}"),
};
let pending = fs::read_dir(telemetry_dir.join("spool"))
.ok()
.and_then(|entries| {
let bodies: Vec<String> = entries
.filter_map(Result::ok)
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "jsonl"))
.filter_map(|path| fs::read_to_string(path).ok())
.collect();
(!bodies.is_empty()).then(|| bodies.join("\n"))
});
let pending = match fs::read_to_string(telemetry_dir.join("buffer.jsonl")) {
Ok(contents) => Some(contents),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => panic!("read telemetry pending buffer: {error}"),
};
DispatchEvidence {
telemetry_dir_exists: telemetry_dir.exists(),
dry_run,
+1 -1
View File
@@ -1313,7 +1313,7 @@ pub const SETTINGS_SCHEMA: &[SettingDef] = &[
def(
"telemetry",
SettingKind::Bool(TELEMETRY),
"false",
"true",
ui(
TAB_TRUST,
"network",
+12
View File
@@ -138,6 +138,16 @@ pub struct SystemOneResponse {
pub answers_validated: Option<bool>,
}
impl SystemOneResponse {
pub fn abstained_questions(&self, names: &[&str]) -> Vec<String> {
names
.iter()
.filter(|name| self.answers.get(**name).is_some_and(|a| a.abstained))
.map(|name| (*name).to_string())
.collect()
}
}
impl<'de> Deserialize<'de> for SystemOneResponse {
fn deserialize<D: serde::Deserializer<'de>>(
deserializer: D,
@@ -201,6 +211,8 @@ pub struct SystemOneAnswer {
pub probabilities: BTreeMap<String, Option<f64>>,
#[serde(default)]
pub confidence: Option<f64>,
#[serde(default)]
pub abstained: bool,
}
/// Provider usage. OpenRouter adds `cost`; the token-count casing differs
+2
View File
@@ -29,6 +29,8 @@ pub struct UserCommand {
pub takes_arguments: bool,
pub hidden: bool,
pub extension: bool,
pub plugin_id: Option<String>,
pub workspace_sourced: bool,
pub builtin_bindings: Vec<String>,
}
+32 -14
View File
@@ -456,7 +456,7 @@ async fn captured_identity_cancel_and_task_bounds_refuse_before_any_provider() {
#[tokio::test]
async fn dropping_an_actual_dispatched_core_call_keeps_unknown_usage_without_retaining_authority() {
struct Pending;
struct Pending(Arc<tokio::sync::Notify>);
impl Replies for Pending {
fn effective_route_envelope(
&self,
@@ -478,24 +478,42 @@ async fn dropping_an_actual_dispatched_core_call_keeps_unknown_usage_without_ret
&self,
_: MessageRequest,
) -> Pin<Box<dyn Future<Output = anyhow::Result<MessageResponse>> + Send + '_>> {
self.0.notify_one();
Box::pin(std::future::pending())
}
}
let caller = Arc::new(caller_for(Arc::new(Pending), "captured-model"));
let entered = Arc::new(tokio::sync::Notify::new());
let caller = Arc::new(caller_for(
Arc::new(Pending(entered.clone())),
"captured-model",
));
let weak_caller = Arc::downgrade(&caller);
let usage = RlmUsageAccumulator::new();
let result = caller
.dispatch(RlmInvocation {
prompt: "pending".into(),
mode: RlmMode::Completion,
max_tokens: None,
task_instructions: None,
deadline: Some(tokio::time::Instant::now() + Duration::from_millis(100)),
gate: None,
events: None,
usage: usage.clone(),
})
.await;
let mut dispatch = caller.dispatch(RlmInvocation {
prompt: "pending".into(),
mode: RlmMode::Completion,
max_tokens: None,
task_instructions: None,
deadline: Some(tokio::time::Instant::now() + Duration::from_secs(600)),
gate: None,
events: None,
usage: usage.clone(),
});
tokio::time::timeout(Duration::from_secs(10), async {
tokio::select! {
biased;
() = entered.notified() => {}
result = &mut dispatch => panic!(
"dropped dispatch never reached the client: {:?}",
result.error
),
}
})
.await
.expect("the dispatched call must reach the client");
tokio::time::pause();
tokio::time::advance(Duration::from_secs(601)).await;
let result = dispatch.await;
assert!(
result
.error
+1
View File
@@ -2643,6 +2643,7 @@ pub fn decision_receipt_fixture(source_id: &str) -> RuntimeDecisionReceipt {
probabilities_bp: Default::default(),
confidence_bp: 0,
min_confidence_bp: 5_000,
abstained_questions: Vec::new(),
cost_saving_kept_fast: false,
thinking: None,
provider_reported_cost_usd: Some("0.000012054".to_string()),
+5 -23
View File
@@ -282,31 +282,20 @@ impl ModelInventory {
// vendor-family heuristic) carries the number *and* the fact
// that nobody verified it — the auto-router must not read a
// guessed window as a route capability.
let mut context_window_unverified =
codewhale_config::catalog::reviewed::intrinsic_model(&model)
.and_then(|row| row.context_window)
.is_none();
let mut context_window_unverified = true;
if let Ok(route) = crate::route_runtime::resolve_runtime_route_for_identity(
config,
&identity,
Some(&model),
) {
if let Some(context_window) = route.candidate.limits().context_tokens {
capability.context_window = context_window.min(u64::from(u32::MAX)) as u32;
context_window_unverified = !route.context_window.source.is_verified();
}
// A concrete offering maximum is a stronger fact than the
// static compatibility matrix — and is the only way a
// membership route (no static cap) gets a known ceiling.
if let Some(max_output) = route
capability.context_window = route.context_window.tokens;
context_window_unverified = !route.context_window.source.is_verified();
capability.max_output = route
.candidate
.limits()
.output_tokens
.and_then(|tokens| u32::try_from(tokens).ok())
.filter(|tokens| *tokens > 0)
{
capability.max_output = Some(max_output);
}
.filter(|tokens| *tokens > 0);
user_declared = route
.candidate
.applied_limit_overrides()
@@ -316,13 +305,6 @@ impl ModelInventory {
== codewhale_config::route::OverrideSource::UserModelMetadata
});
if user_declared {
context_window_unverified = !route.context_window.source.is_verified();
capability.context_window = route.context_window.tokens;
capability.max_output = route
.candidate
.limits()
.output_tokens
.and_then(|value| u32::try_from(value).ok());
capability.thinking_supported = route.candidate.capabilities().reasoning
== codewhale_config::route::CapabilityState::Supported;
}
+60 -2
View File
@@ -1852,6 +1852,7 @@ pub struct ValidChoice {
pub choice: String,
pub probabilities_bp: BTreeMap<String, u16>,
pub confidence_bp: u16,
pub abstained: bool,
}
/// Validate one `choice` answer against the offered options. Any violation
@@ -1894,6 +1895,7 @@ pub fn validated_choice(answer: Option<&SystemOneAnswer>, options: &[&str]) -> O
choice: choice.to_string(),
probabilities_bp,
confidence_bp: probability_bp(confidence),
abstained: answer.abstained,
})
}
@@ -1982,6 +1984,7 @@ fn decision_attempt_from_response(
probabilities_bp: Default::default(),
confidence_bp: 0,
min_confidence_bp: inventory.router_min_confidence_bp,
abstained_questions: response.abstained_questions(&["tier", "thinking"]),
cost_saving_kept_fast: false,
thinking: None,
provider_reported_cost_usd: response
@@ -2005,17 +2008,18 @@ fn decision_attempt_from_response(
let min_confidence_bp = inventory.router_min_confidence_bp;
// An invalid or unsure `thinking` answer drops only the effort.
let thinking = validated_choice(response.answers.get("thinking"), &DECISION_THINKING_OPTIONS)
.filter(|thinking| thinking.confidence_bp >= min_confidence_bp)
.filter(|thinking| !thinking.abstained && thinking.confidence_bp >= min_confidence_bp)
.map(|thinking| thinking.choice);
let strong_bp = tier.probabilities_bp.get("strong").copied().unwrap_or(0);
let cost_saving_kept_fast =
cost_saving && tier.choice == "strong" && strong_bp < COST_SAVING_STRONG_MIN_BP;
let acted = tier.confidence_bp >= min_confidence_bp;
let acted = !tier.abstained && tier.confidence_bp >= min_confidence_bp;
attempt.decision = Some(AutoRouteDecisionEvidence {
choice: tier.choice.clone(),
probabilities_bp: tier.probabilities_bp,
confidence_bp: tier.confidence_bp,
min_confidence_bp,
abstained_questions: response.abstained_questions(&["tier", "thinking"]),
cost_saving_kept_fast: acted && cost_saving_kept_fast,
thinking: thinking.clone().filter(|_| acted),
provider_reported_cost_usd: response
@@ -4187,6 +4191,60 @@ mod decision_router_tests {
);
}
#[test]
fn explicit_abstention_survives_decode_and_never_authorizes_auto_at_floor_zero() {
let _env = hermetic_env();
let mut config = decision_config("https://openrouter.invalid/api/v1", false, 2);
config
.auto
.as_mut()
.unwrap()
.router
.as_mut()
.unwrap()
.min_confidence = Some(0.0);
let inventory = ModelInventory::from_config(&config).unwrap();
let attempt = |body: serde_json::Value| {
decision_attempt_from_response(false, &inventory, &pair(), None, &parsed(body), 120)
};
let mut body = answer_body(0.1, 0.9);
body["answers"]["thinking"]["abstained"] = true.into();
let effort_abstained = attempt(body.clone());
let recommendation = effort_abstained
.recommendation
.expect("valid tier survives effort abstention");
assert_eq!(recommendation.model, "deepseek-v4-flash");
assert_eq!(recommendation.reasoning_effort, None);
assert_eq!(
effort_abstained.decision.unwrap().abstained_questions,
vec!["thinking"]
);
body["answers"]["tier"]["abstained"] = true.into();
let tier_abstained = attempt(body);
assert!(tier_abstained.recommendation.is_none());
assert_eq!(
tier_abstained.fallback_reason,
Some(AutoRouteHeuristicReason::LowConfidence)
);
assert_eq!(tier_abstained.failure, None);
let decision = tier_abstained.decision.unwrap();
assert_eq!(decision.abstained_questions, vec!["tier", "thinking"]);
assert_eq!(decision.choice, "fast");
assert_eq!(
decision.provider_reported_cost_usd.as_deref(),
Some("0.000019992")
);
}
#[test]
fn non_boolean_abstention_is_refused_not_ignored() {
let mut body = answer_body(0.1, 0.9);
body["answers"]["tier"]["abstained"] = "yes".into();
let attempt = policy(false, &parsed(body));
assert!(attempt.recommendation.is_none());
assert_eq!(attempt.failure, Some(AutoRouterFailure::InvalidAnswer));
}
#[test]
fn invalid_answers_are_rejected_not_repaired() {
let mut outside = answer_body(0.8, 0.6);
+19 -16
View File
@@ -1455,6 +1455,24 @@ pub fn configured_catalog_models_for_route(
ids
}
pub fn listed_models_for_route(route_config: &Config, identity: &ProviderIdentity) -> Vec<String> {
let mut models = configured_catalog_models_for_route(
route_config,
identity.provider,
identity.key.as_str(),
&route_config.active_route_base_url(),
);
if identity.provider != ProviderKind::OpenaiCodex {
let default_model = route_config.default_model();
if !default_model.is_empty() && !default_model.eq_ignore_ascii_case("auto") {
push_unique_model(&mut models, &default_model);
}
models.sort();
models.dedup();
}
models
}
/// Look up the **bundled-snapshot** offering for `(provider, wire_model_id)`,
/// ignoring any live rows merged over it.
///
@@ -2219,23 +2237,8 @@ pub async fn run_models(
route_config
.scope_to_provider_identity(identity)
.map_err(anyhow::Error::msg)?;
let mut models = configured_catalog_models_for_route(
&route_config,
identity.provider,
identity.key.as_str(),
&route_config.active_route_base_url(),
);
let models = listed_models_for_route(&route_config, identity);
let default_model = route_config.default_model();
if identity.provider != ProviderKind::OpenaiCodex
&& !default_model.is_empty()
&& !default_model.eq_ignore_ascii_case("auto")
{
push_unique_model(&mut models, &default_model);
}
if identity.provider != ProviderKind::OpenaiCodex {
models.sort();
models.dedup();
}
if json {
// Preserve the existing array + AvailableModel field shape.
let chatgpt_roster = (identity.provider == ProviderKind::OpenaiCodex)
+28 -35
View File
@@ -325,7 +325,7 @@ pub fn selected_route(path: &Path) -> Result<(String, String, codewhale_config::
pub fn list_models(
store: &codewhale_config::ConfigStore,
filter: Option<ProviderKind>,
) -> Result<Vec<(ProviderKind, Vec<String>)>> {
) -> Result<Vec<(String, Vec<String>)>> {
crate::models_dev_live::maybe_load_persisted_cache();
let mut config = saved_config(store)?;
config.base_url_env_receipt = config
@@ -333,42 +333,33 @@ pub fn list_models(
.map_or(BaseUrlEnvReceipt::NoOwner, |identity| {
BaseUrlEnvReceipt::Route(identity.provider, identity.key.to_string())
});
let mut custom_routes: Vec<String> = config
.providers
.as_ref()
.map(|providers| providers.custom.keys().cloned().collect())
.unwrap_or_default();
custom_routes.sort();
let providers = filter
.as_ref()
.map_or(ProviderKind::all(), std::slice::from_ref);
let mut catalogs = Vec::with_capacity(providers.len());
for provider in providers {
if *provider == ProviderKind::Custom
&& config
.providers
.as_ref()
.and_then(|providers| providers.custom_provider_config(provider.as_str()))
.is_none()
{
continue;
let routes = if *provider == ProviderKind::Custom {
custom_routes.clone()
} else {
vec![provider.as_str().to_string()]
};
for route in routes {
let identity = config
.legacy_selection_identity(&route)
.map_err(anyhow::Error::msg)?;
let mut route_config = config.clone();
route_config
.scope_to_provider_identity(&identity)
.map_err(anyhow::Error::msg)?;
let models = crate::provider_lake::listed_models_for_route(&route_config, &identity);
catalogs.push((route, models));
}
let identity = config
.legacy_selection_identity(provider.as_str())
.map_err(anyhow::Error::msg)?;
let mut route_config = config.clone();
route_config
.scope_to_provider_identity(&identity)
.map_err(anyhow::Error::msg)?;
let mut models = crate::provider_lake::configured_catalog_models_for_route(
&route_config,
identity.provider,
identity.key.as_str(),
&route_config.active_route_base_url(),
);
if identity.provider != ProviderKind::OpenaiCodex {
let default_model = route_config.default_model();
if !default_model.is_empty() && !default_model.eq_ignore_ascii_case("auto") {
models.push(default_model);
}
models.sort();
models.dedup();
}
catalogs.push((*provider, models));
}
Ok(catalogs)
}
@@ -418,10 +409,12 @@ pub fn resolve_model(
.into(),
);
}
codewhale_agent::ModelRegistry::default().resolve(
model.filter(|model| *model != "auto"),
Some(identity.provider),
)?;
if !crate::provider_catalog_live::provider_owns_live_catalog(identity.provider, base_url) {
codewhale_agent::ModelRegistry::default().resolve(
model.filter(|model| *model != "auto"),
Some(identity.provider),
)?;
}
}
Ok(route?.candidate)
}
+26 -2
View File
@@ -451,6 +451,11 @@ async fn evaluate(
.map_or_else(Default::default, |v| v.probabilities_bp.clone()),
confidence_bp: intent.as_ref().map_or(0, |v| v.confidence_bp),
min_confidence_bp: 5_000,
abstained_questions: response.abstained_questions(&[
"intent",
"needs_tool",
"answerable_from_context",
]),
cost_saving_kept_fast: false,
thinking: None,
provider_reported_cost_usd: response
@@ -521,7 +526,7 @@ fn unit_interval(value: Option<f64>) -> Option<f64> {
/// fast route.
fn read_noul(response: &SystemOneResponse, key: &str) -> Option<f64> {
let answer = response.answers.get(key)?;
(answer.kind == "noul")
(answer.kind == "noul" && !answer.abstained)
.then_some(answer.noul)
.and_then(unit_interval)
}
@@ -548,7 +553,9 @@ fn derive_route(response: &SystemOneResponse) -> Route {
response.answers.get("intent"),
&["code_change", "code_question", "command", "chat", "other"],
)
.is_some_and(|intent| intent.choice == "chat" && intent.confidence_bp >= 5_000);
.is_some_and(|intent| {
!intent.abstained && intent.choice == "chat" && intent.confidence_bp >= 5_000
});
if intent_is_calibrated_chat && needs_tool.is_some_and(|nt| nt <= 0.2) {
return Route::PlainChat;
}
@@ -564,6 +571,23 @@ mod tests {
const TYPESAFE_TEST_KEY: &str = "sf-typesafe-test-key-0123456789";
#[test]
fn explicit_abstention_never_authorizes_a_shadow_route() {
let mut value = json!({"answers":{
"intent":{"type":"choice","choice":"chat","confidence":0.9,
"probabilities":{"chat":0.9,"code_change":0.1,"code_question":0.0,"command":0.0,"other":0.0}},
"needs_tool":{"type":"noul","noul":0.1},
"answerable_from_context":{"type":"noul","noul":0.1}}});
let decoded =
|value: Value| serde_json::from_value::<SystemOneResponse>(value).expect("response");
assert_eq!(derive_route(&decoded(value.clone())), Route::PlainChat);
value["answers"]["intent"]["abstained"] = true.into();
assert_eq!(derive_route(&decoded(value.clone())), Route::Unknown);
value["answers"]["needs_tool"]["noul"] = 0.95.into();
value["answers"]["needs_tool"]["abstained"] = true.into();
assert_eq!(derive_route(&decoded(value)), Route::Unknown);
}
/// Fields drop in declaration order: restore the environment before the
/// lock is released, or another test observes our overrides.
struct Env {
+2
View File
@@ -11,6 +11,8 @@ pub struct AutoRouteDecisionEvidence {
pub probabilities_bp: BTreeMap<String, u16>,
pub confidence_bp: u16,
pub min_confidence_bp: u16,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub abstained_questions: Vec<String>,
/// `[auto] cost_saving` kept the fast tier because the strong tier's
/// probability was below the cost-saving floor.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
+14 -131
View File
@@ -23,7 +23,6 @@ use crate::client::{self, SendOutcome};
use crate::decision::{self, TelemetryDecision};
use crate::envelope;
use crate::event::{Batch, Event, SCHEMA_VERSION, Surface};
use crate::spool::{self, SpoolContext};
/// Events per batch.
pub const BATCH_MAX_EVENTS: usize = 200;
@@ -65,19 +64,6 @@ pub(crate) struct Context {
pub app_version: String,
pub git_sha: Option<String>,
pub tty: bool,
pub spool: PathBuf,
pub nonce: String,
}
impl Context {
fn header(&self) -> SpoolContext {
SpoolContext::new(
self.surface,
self.app_version.clone(),
self.git_sha.clone(),
self.tty,
)
}
}
/// A handle to the writer thread.
@@ -223,7 +209,8 @@ fn append(context: &Context, event: &Event) {
let Ok(line) = serde_json::to_string(event) else {
return;
};
let _ = buffer::append(&context.root, &context.spool, &line);
let path = buffer::buffer_path(&context.root);
let _ = buffer::append(&context.root, &path, &line);
}
/// Seal every event queued before this message without reaching the network.
@@ -232,7 +219,7 @@ fn append(context: &Context, event: &Event) {
/// runs. Re-deciding here preserves the same mid-session opt-out boundary as a
/// full flush. An explicitly empty endpoint is the local dry-run sink and can
/// therefore be finalized immediately; a configured endpoint leaves the
/// events in this process's spool for a later start to send.
/// events in `buffer.jsonl` for the next interactive flush.
fn persist_local(context: &Context) -> FlushOutcome {
match decision::re_decide(context.config_path.as_deref(), context.surface) {
TelemetryDecision::Enabled(_) => {}
@@ -246,10 +233,7 @@ fn persist_local(context: &Context) -> FlushOutcome {
if context.endpoint.is_none() {
return flush(context);
}
if buffer::read_lines(&context.spool)
.iter()
.all(|line| spool::is_header_line(line))
{
if buffer::read_lines(&buffer::buffer_path(&context.root)).is_empty() {
FlushOutcome::Empty
} else {
FlushOutcome::Buffered
@@ -273,7 +257,7 @@ fn flush(context: &Context) -> FlushOutcome {
return FlushOutcome::Suppressed;
}
let lines = spool::take_own(&context.root, &context.spool);
let lines = buffer::drain(&context.root);
if lines.is_empty() {
return FlushOutcome::Empty;
}
@@ -293,94 +277,28 @@ fn flush(context: &Context) -> FlushOutcome {
// attempts at most once per interval rather than on every launch.
let _ = envelope::write_state(&context.root, &state);
deliver(
&context.root,
context.endpoint.as_deref(),
&context.header(),
install.install_id,
events,
)
}
fn deliver(
root: &std::path::Path,
endpoint: Option<&str>,
header: &SpoolContext,
install_id: String,
events: Vec<Event>,
) -> FlushOutcome {
let batch = Batch {
schema_version: header.schema_version,
notice_version: header.notice_version,
schema_version: SCHEMA_VERSION,
notice_version: crate::event::NOTICE_VERSION,
sent_at: envelope::now_rfc3339(),
install_id,
app_version: header.app_version.clone(),
git_sha: header.git_sha.clone(),
surface: header.surface,
install_id: install.install_id,
app_version: context.app_version.clone(),
git_sha: context.git_sha.clone(),
surface: context.surface,
os: envelope::current_os(),
arch: envelope::current_arch(),
libc: envelope::current_libc(),
tty: header.tty,
tty: context.tty,
events,
};
match client::send(root, endpoint, &batch) {
match client::send(&context.root, context.endpoint.as_deref(), &batch) {
SendOutcome::DryRun => FlushOutcome::DryRun,
SendOutcome::Accepted => FlushOutcome::Sent,
SendOutcome::Dropped => FlushOutcome::Dropped,
}
}
/// Send what dead processes left behind, once each, on their own thread so a
/// slow request never holds up this session's writer.
///
/// Consent and the tombstone are decided before anything is read, and again by
/// the sender for every batch. Each spool is deleted before its request.
pub(crate) fn spawn_recovery(context: Context) {
if !matches!(
context.surface,
Surface::Tui | Surface::Exec | Surface::Serve
) {
return;
}
let _ = std::thread::Builder::new()
.name("codewhale-telemetry-recover".to_string())
.spawn(move || {
let _ = catch_unwind(AssertUnwindSafe(|| recover(&context)));
});
}
pub(crate) fn recover(context: &Context) -> Vec<FlushOutcome> {
match decision::re_decide(context.config_path.as_deref(), context.surface) {
TelemetryDecision::Enabled(_) => {}
TelemetryDecision::OptedOut | TelemetryDecision::ForcedOff => return Vec::new(),
}
send_dead(context)
}
pub(crate) fn send_dead(context: &Context) -> Vec<FlushOutcome> {
if buffer::tombstone_present(&context.root) {
return Vec::new();
}
let Ok(install) = envelope::read_or_create_install_id(&context.root) else {
return Vec::new();
};
spool::take_dead(&context.root, &context.nonce, std::time::SystemTime::now())
.into_iter()
.filter_map(|recovered| {
let events = parse_events(&recovered.lines);
(!events.is_empty()).then(|| {
deliver(
&context.root,
context.endpoint.as_deref(),
&recovered.context,
install.install_id.clone(),
events,
)
})
})
.collect()
}
/// Parse drained lines into events, capped at [`BATCH_MAX_EVENTS`] and
/// [`BATCH_MAX_BYTES`], skipping anything that does not parse **or does not
/// satisfy its declared string bounds**.
@@ -455,39 +373,4 @@ mod tests {
};
assert_eq!(handle.persist_local(), FlushOutcome::TimedOut);
}
#[test]
fn a_hung_writer_cannot_hold_a_termination_send_past_its_deadline() {
let (tx, rx) = sync_channel::<Message>(buffer::MAX_EVENTS + 1);
let handle = Handle {
tx,
pending: Arc::new(AtomicUsize::new(0)),
closing: AtomicBool::new(false),
};
let started = Instant::now();
let outcome = handle.shutdown(Duration::from_millis(50));
assert_eq!(outcome, FlushOutcome::TimedOut);
assert!(started.elapsed() < Duration::from_secs(5));
assert!(matches!(rx.try_recv(), Ok(Message::Shutdown(_))));
}
#[test]
fn a_second_termination_send_does_not_wait() {
let (tx, _rx) = sync_channel::<Message>(buffer::MAX_EVENTS + 1);
let handle = Handle {
tx,
pending: Arc::new(AtomicUsize::new(0)),
closing: AtomicBool::new(false),
};
assert_eq!(
handle.shutdown(Duration::from_millis(20)),
FlushOutcome::TimedOut
);
let started = Instant::now();
assert_eq!(
handle.shutdown(Duration::from_secs(30)),
FlushOutcome::TimedOut
);
assert!(started.elapsed() < Duration::from_secs(5));
}
}
+33 -47
View File
@@ -5,10 +5,8 @@
//!
//! | file | role |
//! |---|---|
//! | `spool/<nonce>.jsonl` | one process's header line, then one JSON event per line, awaiting its flush |
//! | `spool/<nonce>.lock` | held exclusively by that process while it lives |
//! | `buffer.jsonl` | the pre-spool sink, only ever truncated now |
//! | `buffer.jsonl.lock` | the **privacy** lock, a sibling file; never a data file |
//! | `buffer.jsonl` | one JSON event per line, awaiting a flush |
//! | `buffer.jsonl.lock` | a **sibling** lock file; never the data file |
//! | `dryrun.jsonl` | the sink when the endpoint resolves empty, same ring policy |
//! | `state.json` | last version seen and last flush attempt |
//! | `install_id.json` | the random install id |
@@ -220,39 +218,6 @@ fn append_under_lock(root: &Path, path: &Path, buf: &[u8]) -> Result<bool> {
Ok(true)
}
/// Write a spool header line, while the caller holds the blocking privacy lock.
pub(crate) fn append_header(root: &Path, path: &Path, line: &str) -> Result<bool> {
let mut buf = Vec::with_capacity(line.len() + 1);
buf.extend_from_slice(line.as_bytes());
buf.push(b'\n');
append_under_lock(root, path, &buf)
}
/// Re-attach the spool header that a ring cut dropped, keeping the newest events.
fn keep_spool_header(path: &Path, mut lines: Vec<String>) -> Vec<String> {
use std::io::BufRead as _;
let Ok(file) = File::open(path) else {
return lines;
};
let mut first = String::new();
if std::io::BufReader::new(file.take(MAX_LINE_BYTES as u64 + 2))
.read_line(&mut first)
.is_err()
{
return lines;
}
let first = first.trim_end().to_string();
if first.is_empty() || !crate::spool::is_header_line(&first) || lines.first() == Some(&first) {
return lines;
}
lines.insert(0, first);
let size = |lines: &[String]| lines.iter().map(|l| l.len() as u64 + 1).sum::<u64>();
while lines.len() > 1 && (lines.len() > MAX_EVENTS || size(&lines) > MAX_BYTES) {
lines.remove(1);
}
lines
}
/// Keep the newest [`MAX_EVENTS`] lines and at most [`MAX_BYTES`], under the
/// compaction lock. On lock contention this cycle is skipped: the next append
/// tries again, and the cap is a ceiling on disk footprint, not an invariant
@@ -278,7 +243,7 @@ fn enforce_ring(root: &Path, path: &Path) {
if len < PROBE_BYTES {
return Ok(());
}
let lines = keep_spool_header(path, read_ring_tail(path)?);
let lines = read_ring_tail(path)?;
let mut body = lines.join("\n");
if !body.is_empty() {
body.push('\n');
@@ -406,6 +371,33 @@ pub fn read_lines(path: &Path) -> Vec<String> {
read_ring_tail(path).unwrap_or_default()
}
/// Take every buffered line and truncate the buffer, under the compaction lock.
///
/// Returns an empty vector when the tombstone is present or the lock is held
/// elsewhere. Truncates rather than unlinks — `crates/engine/src/fleet/ledger.rs`
/// documents the rule: replacing the file leaves appenders holding the old
/// inode.
#[must_use]
pub fn drain(root: &Path) -> Vec<String> {
if tombstone_present(root) {
return Vec::new();
}
let path = buffer_path(root);
let drained = try_with_lock(root, || {
// Re-check under the lock: a wipe may have landed between the check
// above and the acquisition.
if tombstone_present(root) {
return Ok(Vec::new());
}
let lines = read_lines(&path);
if !lines.is_empty() {
truncate(&path)?;
}
Ok(lines)
});
drained.ok().flatten().unwrap_or_default()
}
/// Truncate a file to zero length, leaving the inode in place. A missing file
/// is not an error.
pub fn truncate(path: &Path) -> Result<()> {
@@ -428,7 +420,7 @@ pub fn truncate(path: &Path) -> Result<()> {
/// 1. take the blocking lock — this is not an exit path;
/// 2. write the tombstone **first**, and never remove it here;
/// 3. truncate `buffer.jsonl` and `dryrun.jsonl` — do **not** unlink them, and
/// never unlink the privacy lock file — and delete every spool;
/// never unlink the lock file;
/// 4. remove `install_id.json` and `state.json`.
///
/// If any step after the tombstone fails, the error is returned and the caller
@@ -474,11 +466,6 @@ pub fn wipe(root: &Path) -> Result<()> {
);
}
}
if let Err(error) = fs::remove_dir_all(crate::spool::dir(root))
&& error.kind() != std::io::ErrorKind::NotFound
{
failure.get_or_insert(anyhow::Error::new(error).context("failed to remove the spools"));
}
match failure {
Some(error) => Err(error),
None => Ok(()),
@@ -486,8 +473,8 @@ pub fn wipe(root: &Path) -> Result<()> {
})
}
/// Clear the tombstone, deleting every spool when there was one, and truncate
/// the pre-spool buffer.
/// Clear the tombstone and drop anything buffered before this process was
/// permitted.
///
/// Called by `init` on every arming. Both the exact tombstone generation and a
/// fresh durable permission check must still match while the wipe lock is held.
@@ -509,7 +496,6 @@ pub(crate) fn arm(
}
let tombstone = tombstone_path(root);
if tombstone.exists() {
let _ = fs::remove_dir_all(crate::spool::dir(root));
fs::remove_file(&tombstone)
.with_context(|| format!("failed to remove {}", tombstone.display()))?;
}
+6 -136
View File
@@ -21,8 +21,8 @@
//! construction empty until resolution completes. A disabled user's panic
//! therefore writes nothing and creates no directory.
//!
//! Arming also gives the process its own spool file and, when the machine was
//! opted out until now, deletes every spool left from before.
//! Arming also **truncates** any stale buffer before a newly permitted process
//! begins recording.
//!
//! # Failure posture
//!
@@ -41,14 +41,13 @@ pub mod decision;
pub mod envelope;
pub mod event;
pub mod notice;
pub mod spool;
#[cfg(test)]
mod tests;
use std::sync::OnceLock;
use std::sync::atomic::{AtomicBool, AtomicU8, AtomicU64, Ordering};
use std::time::{Duration, Instant};
use std::sync::atomic::{AtomicU8, Ordering};
use std::time::Duration;
pub use actor::{BATCH_MAX_BYTES, BATCH_MAX_EVENTS, FlushOutcome};
pub use counters::{Counter, ErrorCounter, SessionCounters, TurnOutcome, TurnTokens};
@@ -70,27 +69,11 @@ pub use event::{
/// handshake would hold a user's terminal past exit.
pub const SHUTDOWN_FLUSH_TIMEOUT: Duration = Duration::from_secs(3);
/// How long the send attempt on a terminating signal or a broken pipe may hold
/// the process.
///
/// Shorter than [`SHUTDOWN_FLUSH_TIMEOUT`] so that, run alongside the runtime's
/// two second signal drain, it fits inside the two second grace a supervisor
/// gives SIGTERM before it escalates.
pub const SIGNAL_FLUSH_TIMEOUT: Duration = Duration::from_millis(1500);
const COLD_START_UNSET: u64 = u64::MAX;
static COLD_START_MS: AtomicU64 = AtomicU64::new(COLD_START_UNSET);
/// Everything a write path needs once the process is armed.
struct Armed {
handle: actor::Handle,
root: std::path::PathBuf,
spool: std::path::PathBuf,
surface: Surface,
started: Instant,
exit_class: AtomicU8,
session_end_recorded: AtomicBool,
}
/// The one gate. Unset means every write path is a hard no-op.
@@ -124,14 +107,6 @@ pub fn init(consent: TelemetryConsent) {
notice::show_startup_disclosure(consent.surface());
let header = spool::SpoolContext::new(
consent.surface(),
env!("CARGO_PKG_VERSION").to_string(),
envelope::release_build_sha(),
envelope::current_tty(),
);
let (spool_path, nonce) = spool::open(&root, &header);
let context = actor::Context {
root: root.clone(),
endpoint: consent.endpoint().map(str::to_string),
@@ -140,19 +115,12 @@ pub fn init(consent: TelemetryConsent) {
app_version: env!("CARGO_PKG_VERSION").to_string(),
git_sha: envelope::release_build_sha(),
tty: envelope::current_tty(),
spool: spool_path.clone(),
nonce,
};
actor::spawn_recovery(context.clone());
let _ = ARMED.set(Armed {
handle: actor::Handle::spawn(context),
root: root.clone(),
spool: spool_path,
surface: consent.surface(),
started: Instant::now(),
exit_class: AtomicU8::new(ExitClass::Clean.as_u8()),
session_end_recorded: AtomicBool::new(false),
});
record_install_or_upgrade(&root);
@@ -273,7 +241,8 @@ pub fn record_blocking(event: Event) {
let Ok(line) = serde_json::to_string(&event) else {
return;
};
let _ = buffer::append(&armed.root, &armed.spool, &line);
let path = buffer::buffer_path(&armed.root);
let _ = buffer::append(&armed.root, &path, &line);
}
/// Record how this process is ending.
@@ -297,105 +266,6 @@ pub fn exit_class() -> ExitClass {
})
}
/// Report how long the interactive event loop took to start. The terminal UI is
/// the only caller; every other surface leaves the cold-start bucket absent.
pub fn set_cold_start_ms(millis: u64) {
let _ = COLD_START_MS.compare_exchange(
COLD_START_UNSET,
millis,
Ordering::Relaxed,
Ordering::Relaxed,
);
}
/// Build `session_end` from what this process actually accumulated.
///
/// The exit class is read from the process-wide atomic and never derived from
/// an exit code. Building the event writes nothing; an unarmed process reports a
/// zero duration and nothing reaches a sink.
#[must_use]
pub fn session_end_event() -> Event {
let counters = session_counters();
let cold_start = COLD_START_MS.load(Ordering::Relaxed);
Event::SessionEnd {
duration_bucket: DurationBucket::from_secs(
ARMED
.get()
.map_or(0, |armed| armed.started.elapsed().as_secs()),
),
exit_class: exit_class(),
cold_start_bucket: (cold_start != COLD_START_UNSET)
.then(|| ColdStartBucket::from_millis(cold_start)),
providers: counters.providers(),
counters: counters.counters(),
errors: counters.errors(),
turn_wall: counters.turn_wall(),
}
}
/// Queue this session's usage rows and `session_end` once.
///
/// Whichever of the clean path, the signal path or a broken pipe gets here
/// first reports; later callers do nothing, so one session never ends twice.
/// A no-op when unarmed.
pub fn record_session_end() {
let Some(armed) = ARMED.get() else {
return;
};
if armed.session_end_recorded.swap(true, Ordering::AcqRel) {
return;
}
for event in session_counters().usage_events() {
record(event);
}
record(session_end_event());
}
/// Close the session synchronously, for a path that cannot wait for the writer.
///
/// States `class`, then appends the usage rows and `session_end` to the local
/// buffer with the same lock-free-for-the-caller write as [`record_blocking`].
/// A no-op when unarmed, and a no-op for the events when the session already
/// ended.
pub fn record_session_end_blocking(class: ExitClass) {
let Some(armed) = ARMED.get() else {
return;
};
set_exit_class(class);
if armed.session_end_recorded.swap(true, Ordering::AcqRel) {
return;
}
for event in session_counters().usage_events() {
record_blocking(event);
}
record_blocking(session_end_event());
}
/// One bounded network attempt for a process that is being terminated.
///
/// Runs the ordinary flush, so consent, the opt-out tombstone and the run kill
/// switch are re-checked immediately beforehand and a person who has opted out
/// sends nothing. Short CLI commands stay disk-only. Never retried, and the
/// caller is released after `deadline` whether or not the request finished.
#[must_use]
pub fn flush_for_termination(deadline: Duration) -> FlushOutcome {
let Some(armed) = ARMED.get() else {
return FlushOutcome::Empty;
};
if armed.surface == Surface::Cli {
return FlushOutcome::Buffered;
}
armed.handle.shutdown(deadline)
}
/// Record the session end and make the one bounded send attempt, for a process
/// that is about to be ended by a signal. Returns [`FlushOutcome::Empty`] when
/// unarmed.
pub fn report_termination(class: ExitClass, deadline: Duration) -> FlushOutcome {
record_session_end_blocking(class);
flush_for_termination(deadline)
}
/// Final flush, then stop the writer thread.
///
/// Returns [`FlushOutcome::Empty`] when unarmed.
-277
View File
@@ -1,277 +0,0 @@
//! Per-process spools and the recovery of the ones a dead process left behind.
//!
//! Each armed process owns `telemetry/spool/<nonce>.jsonl`, whose first line is
//! a context header (the envelope facts that belong to that process), and holds
//! an exclusive lock on `<nonce>.lock` for as long as it lives. Recording and
//! the clean flush touch only the process's own spool, so one process can
//! neither truncate nor send another's events under the wrong surface or
//! version.
//!
//! A spool whose lock can be taken belongs to a process that is gone. A later
//! TUI, exec or serve start sends at most [`MAX_RECOVERED`] of them, each once:
//! the file is deleted **before** the request, so a crash or a failed send never
//! causes a second attempt.
//!
//! The nonce is local only. It names a file and is never sent.
//!
//! Known limitation: a spool is only as trustworthy as the user account, like
//! every other file here. Recovery therefore re-validates the header and every
//! event as untrusted input instead of inheriting any bound.
use std::collections::BTreeSet;
use std::fs::{self, File, OpenOptions};
use std::path::{Path, PathBuf};
use std::time::{Duration, SystemTime};
use serde::{Deserialize, Serialize};
use crate::buffer;
use crate::event::{NOTICE_VERSION, SCHEMA_VERSION, Surface, is_release_version_string};
/// Dead spools sent at one start. Older or surplus ones are deleted unsent.
pub const MAX_RECOVERED: usize = 4;
/// A dead spool older than this is deleted unsent.
pub const MAX_AGE: Duration = Duration::from_secs(7 * 24 * 60 * 60);
const HEADER_VERSION: u32 = 1;
const MAX_VERSION_BYTES: usize = 64;
/// The envelope facts one process stamps on everything it spools.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub(crate) struct SpoolContext {
spool: u32,
pub schema_version: u32,
pub notice_version: u32,
pub app_version: String,
pub git_sha: Option<String>,
pub surface: Surface,
pub tty: bool,
}
impl SpoolContext {
pub(crate) fn new(
surface: Surface,
app_version: String,
git_sha: Option<String>,
tty: bool,
) -> Self {
Self {
spool: HEADER_VERSION,
schema_version: SCHEMA_VERSION,
notice_version: NOTICE_VERSION,
app_version,
git_sha,
surface,
tty,
}
}
/// Whether a header read from disk may be sent as an envelope.
pub(crate) fn is_sendable(&self) -> bool {
self.spool == HEADER_VERSION
&& self.schema_version == SCHEMA_VERSION
&& self.notice_version == NOTICE_VERSION
&& self.app_version.len() <= MAX_VERSION_BYTES
&& is_release_version_string(&self.app_version)
&& self.git_sha.as_deref().is_none_or(|sha| {
sha.len() == 12 && sha.bytes().all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f'))
})
}
}
/// Whether `line` is a spool header rather than an event.
pub(crate) fn is_header_line(line: &str) -> bool {
serde_json::from_str::<SpoolContext>(line).is_ok()
}
/// `$CODEWHALE_HOME/telemetry/spool`.
#[must_use]
pub fn dir(root: &Path) -> PathBuf {
root.join("spool")
}
/// Every spool data file under `root`, for tests and diagnostics.
#[must_use]
pub fn data_files(root: &Path) -> Vec<PathBuf> {
let Ok(entries) = fs::read_dir(dir(root)) else {
return Vec::new();
};
let mut files: Vec<PathBuf> = entries
.filter_map(Result::ok)
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "jsonl"))
.collect();
files.sort();
files
}
fn data_path(root: &Path, nonce: &str) -> PathBuf {
dir(root).join(format!("{nonce}.jsonl"))
}
fn lock_file_path(root: &Path, nonce: &str) -> PathBuf {
dir(root).join(format!("{nonce}.lock"))
}
fn is_nonce(value: &str) -> bool {
value.len() == 32
&& value
.bytes()
.all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f'))
}
fn open_lock_file(path: &Path) -> Option<File> {
OpenOptions::new()
.create(true)
.read(true)
.write(true)
.truncate(false)
.open(path)
.ok()
}
/// Create this process's spool and hold its lock until the process ends.
///
/// Runs at arming, where blocking on the privacy lock is allowed, so the header
/// is on disk before any other writer, including the signal path, can append.
/// Returns the data path and the nonce.
pub(crate) fn open(root: &Path, context: &SpoolContext) -> (PathBuf, String) {
let nonce = uuid::Uuid::new_v4().simple().to_string();
let path = data_path(root, &nonce);
if buffer::ensure_dir(&dir(root)).is_ok() {
if let Some(file) = open_lock_file(&lock_file_path(root, &nonce)) {
let lock = Box::leak(Box::new(fd_lock::RwLock::new(file)));
if let Ok(guard) = lock.try_write() {
std::mem::forget(guard);
}
}
if let Ok(header) = serde_json::to_string(context) {
let _ = buffer::with_lock(root, || buffer::append_header(root, &path, &header));
}
}
(path, nonce)
}
/// Take this process's own spool for delivery: read it and delete it, under the
/// privacy lock.
///
/// Empty when the tombstone is present, the lock is held elsewhere, or the file
/// could not be removed. A spool that cannot be removed is never sent, because
/// a later start would send it again.
#[must_use]
pub(crate) fn take_own(root: &Path, path: &Path) -> Vec<String> {
if buffer::tombstone_present(root) {
return Vec::new();
}
buffer::try_with_lock(root, || {
if buffer::tombstone_present(root) {
return Ok(Vec::new());
}
let lines = buffer::read_lines(path);
if lines.is_empty() {
return Ok(lines);
}
if remove_or_truncate(path) {
Ok(lines)
} else {
Ok(Vec::new())
}
})
.ok()
.flatten()
.unwrap_or_default()
}
fn remove_or_truncate(path: &Path) -> bool {
fs::remove_file(path).is_ok() || buffer::truncate(path).is_ok_and(|()| !path.exists())
}
/// A dead process's spool, already deleted from disk.
#[derive(Debug)]
pub(crate) struct Recovered {
pub context: SpoolContext,
pub lines: Vec<String>,
}
/// Claim the dead spools that may be sent, newest first, deleting every dead
/// spool it looks at.
///
/// Live spools are skipped by their lock, and `own_nonce` is never touched.
/// Unknown file names, symbolic links and anything without a valid header are
/// deleted or ignored, never read as events.
pub(crate) fn take_dead(root: &Path, own_nonce: &str, now: SystemTime) -> Vec<Recovered> {
let Ok(entries) = fs::read_dir(dir(root)) else {
return Vec::new();
};
let mut stems = BTreeSet::new();
for entry in entries.filter_map(Result::ok) {
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
if let Some((stem, ext)) = name.rsplit_once('.')
&& matches!(ext, "jsonl" | "lock")
&& is_nonce(stem)
&& stem != own_nonce
{
stems.insert(stem.to_string());
}
}
let mut ranked: Vec<(SystemTime, String)> = stems
.into_iter()
.map(|stem| {
let modified = [data_path(root, &stem), lock_file_path(root, &stem)]
.iter()
.find_map(|path| fs::symlink_metadata(path).and_then(|m| m.modified()).ok())
.unwrap_or(SystemTime::UNIX_EPOCH);
(modified, stem)
})
.collect();
ranked.sort_by(|a, b| b.cmp(a));
let mut recovered = Vec::new();
for (modified, stem) in ranked {
if buffer::tombstone_present(root) {
break;
}
let data = data_path(root, &stem);
let lock_path = lock_file_path(root, &stem);
let eligible = recovered.len() < MAX_RECOVERED
&& now.duration_since(modified).is_ok_and(|age| age <= MAX_AGE);
let claimed = with_dead_lock(&lock_path, || {
let claimed = if eligible { claim(&data) } else { None };
if claimed.is_none() {
let _ = fs::remove_file(&data);
}
let _ = fs::remove_file(&lock_path);
claimed
});
if let Some(Some(claimed)) = claimed {
recovered.push(claimed);
}
}
recovered
}
fn with_dead_lock<T>(lock_path: &Path, operation: impl FnOnce() -> T) -> Option<T> {
let file = open_lock_file(lock_path)?;
let mut lock = fd_lock::RwLock::new(file);
let _guard = lock.try_write().ok()?;
Some(operation())
}
fn claim(data: &Path) -> Option<Recovered> {
let meta = fs::symlink_metadata(data).ok()?;
if !meta.is_file() {
return None;
}
let lines = buffer::read_lines(data);
let (header, events) = lines.split_first()?;
let context = serde_json::from_str::<SpoolContext>(header).ok()?;
if !context.is_sendable() || events.is_empty() || !remove_or_truncate(data) {
return None;
}
Some(Recovered {
context,
lines: events.to_vec(),
})
}
+5 -330
View File
@@ -1074,7 +1074,7 @@ fn failed_wipe_fails_closed() {
assert!(result.is_err(), "a failed truncate must be reported");
}
assert!(
crate::spool::take_own(&root, &buffer_path).is_empty(),
buffer::drain(&root).is_empty(),
"a tombstoned buffer must never drain, wipe failure or not"
);
assert!(
@@ -1262,14 +1262,14 @@ fn drain_skips_a_torn_trailing_line() {
.expect("open");
file.write_all(b"{\"event\":\"pan").expect("tear");
}
let drained = crate::spool::take_own(&root, &path);
let drained = buffer::drain(&root);
assert_eq!(drained.len(), 3, "the drain returns raw lines");
let parsed: Vec<Event> = drained
.iter()
.filter_map(|l| serde_json::from_str(l).ok())
.collect();
assert_eq!(parsed.len(), 2, "the torn line must not reach a batch");
assert!(!path.exists(), "taking a spool deletes it");
assert!(buffer::read_lines(&path).is_empty(), "drain truncates");
}
#[test]
@@ -1320,25 +1320,21 @@ fn a_tombstoned_buffer_never_appends_or_drains() {
assert!(buffer::append(&root, &buffer::buffer_path(&root), &line(2)).is_none());
assert!(buffer::append_locked(&root, &buffer::dryrun_path(&root), "{}").is_none());
assert!(crate::spool::take_own(&root, &buffer::buffer_path(&root)).is_empty());
assert!(buffer::drain(&root).is_empty());
}
#[test]
fn arming_after_a_tombstone_deletes_spools_and_truncates_the_legacy_buffer() {
fn arming_truncates_a_pre_consent_buffer() {
let home = temp_home();
let root = root_of(&home);
buffer::ensure_dir(&root).expect("create root");
buffer::append(&root, &buffer::buffer_path(&root), &line(1)).expect("append");
buffer::wipe(&root).expect("wipe");
std::fs::create_dir_all(crate::spool::dir(&root)).expect("spool dir");
let planted = crate::spool::dir(&root).join(format!("{}.jsonl", "a".repeat(32)));
std::fs::write(&planted, "{}\n").expect("plant");
let generation = buffer::tombstone_generation(&root).expect("read wipe generation");
buffer::arm(&root, generation.as_ref(), || true).expect("arm");
assert!(!buffer::tombstone_present(&root));
assert!(buffer::read_lines(&buffer::buffer_path(&root)).is_empty());
assert!(!planted.exists(), "re-enabling must not inherit old spools");
}
#[test]
@@ -1438,21 +1434,6 @@ fn completed_wipe_blocks_identity_and_state_recreation() {
// ------------------------------------------------------------ unarmed gate --
#[test]
fn termination_reporting_is_a_noop_when_unarmed() {
assert!(!crate::is_armed());
crate::record_session_end();
crate::record_session_end_blocking(ExitClass::Signal);
assert_eq!(
crate::report_termination(ExitClass::Signal, Duration::from_millis(10)),
crate::FlushOutcome::Empty
);
assert_eq!(
crate::flush_for_termination(Duration::from_millis(10)),
crate::FlushOutcome::Empty
);
}
#[test]
fn record_blocking_is_a_noop_when_unarmed() {
// The process panic hook is installed before the command line is parsed, so
@@ -2013,309 +1994,3 @@ fn the_notice_summarizes_what_the_schema_collects_and_states_every_red_line() {
assert!(!body.contains("CODEWHALE_TELEMETRY=0"));
assert!(body.contains("docs/TELEMETRY.md"));
}
// ------------------------------------------------------------- the spools --
mod spools {
use super::*;
use crate::actor::{Context, send_dead};
use crate::spool::{self, SpoolContext};
fn header(surface: Surface, version: &str) -> SpoolContext {
SpoolContext::new(
surface,
version.to_string(),
Some("abcdef123456".to_string()),
true,
)
}
fn event_line(source: SessionSource) -> String {
serde_json::to_string(&Event::SessionStart { source }).expect("serialize")
}
fn plant(root: &Path, nonce: char, header_line: &str, events: &[String]) -> PathBuf {
std::fs::create_dir_all(spool::dir(root)).expect("spool dir");
let path = spool::dir(root).join(format!("{}.jsonl", nonce.to_string().repeat(32)));
let mut body = format!("{header_line}\n");
for event in events {
body.push_str(event);
body.push('\n');
}
std::fs::write(&path, body).expect("plant spool");
path
}
fn header_line(surface: Surface, version: &str) -> String {
serde_json::to_string(&header(surface, version)).expect("serialize header")
}
fn current(root: &Path) -> Context {
let header = header(Surface::Tui, "0.10.3");
let (spool, nonce) = spool::open(root, &header);
Context {
root: root.to_path_buf(),
endpoint: None,
surface: Surface::Tui,
config_path: None,
app_version: "0.10.3".to_string(),
git_sha: None,
tty: false,
spool,
nonce,
}
}
fn dryrun_batches(root: &Path) -> Vec<Batch> {
buffer::read_lines(&buffer::dryrun_path(root))
.iter()
.map(|line| serde_json::from_str(line).expect("a dry-run batch"))
.collect()
}
#[test]
fn a_dead_spool_is_recovered_with_its_own_surface_and_version() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
let path = plant(
&root,
'b',
&header_line(Surface::Serve, "0.9.9"),
&[event_line(SessionSource::Interactive)],
);
let outcomes = send_dead(&context);
assert_eq!(outcomes, vec![crate::FlushOutcome::DryRun]);
let batches = dryrun_batches(&root);
assert_eq!(batches.len(), 1);
assert_eq!(batches[0].surface, Surface::Serve);
assert_eq!(batches[0].app_version, "0.9.9");
assert_eq!(batches[0].git_sha.as_deref(), Some("abcdef123456"));
assert_eq!(batches[0].events.len(), 1);
assert!(!path.exists());
}
#[test]
fn a_dead_spool_is_never_sent_twice() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
plant(
&root,
'b',
&header_line(Surface::Exec, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
assert_eq!(send_dead(&context).len(), 1);
assert!(send_dead(&context).is_empty());
assert_eq!(dryrun_batches(&root).len(), 1);
}
#[test]
fn a_live_locked_spool_is_untouched() {
let home = temp_home();
let root = root_of(&home);
let live = header(Surface::Serve, "0.10.3");
let (live_path, live_nonce) = spool::open(&root, &live);
buffer::append(&root, &live_path, &event_line(SessionSource::Interactive)).expect("append");
let taken = spool::take_dead(&root, &"c".repeat(32), std::time::SystemTime::now());
assert!(taken.is_empty());
assert_eq!(buffer::read_lines(&live_path).len(), 2);
assert!(
spool::dir(&root)
.join(format!("{live_nonce}.lock"))
.exists()
);
}
#[test]
fn the_running_process_never_recovers_its_own_spool() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
buffer::append(
&root,
&context.spool,
&event_line(SessionSource::Interactive),
)
.expect("append");
assert!(send_dead(&context).is_empty());
assert_eq!(buffer::read_lines(&context.spool).len(), 2);
}
#[test]
fn a_tombstone_blocks_recovery_and_append() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
buffer::wipe(&root).expect("wipe");
let planted = plant(
&root,
'b',
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
assert!(send_dead(&context).is_empty());
assert!(dryrun_batches(&root).is_empty());
assert!(
planted.exists(),
"nothing is read or sent under a tombstone"
);
assert!(buffer::append(&root, &context.spool, "{}").is_none());
}
#[test]
fn wipe_removes_every_spool() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
plant(
&root,
'b',
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
buffer::wipe(&root).expect("wipe");
assert!(!spool::dir(&root).exists());
assert!(!context.spool.exists());
}
#[test]
fn a_hostile_context_drops_the_whole_spool_unsent() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
let good = event_line(SessionSource::Interactive);
let hostile_headers = [
header_line(Surface::Serve, "/Users/victim/secret"),
header_line(Surface::Serve, "0.10.3").replace("abcdef123456", "ZZZZZZZZZZZZ"),
header_line(Surface::Serve, "0.10.3").replace("\"spool\":1", "\"spool\":2"),
header_line(Surface::Serve, "0.10.3")
.replace("\"schema_version\":", "\"x\":9,\"schema_version\":"),
header_line(Surface::Serve, "0.10.3").replace("\"serve\"", "\"not-a-surface\""),
format!(
"{{\"schema_version\":{},\"notice_version\":{},\"app_version\":\"0.10.3\"}}",
SCHEMA_VERSION + 1,
NOTICE_VERSION
),
good.clone(),
];
for (index, hostile) in hostile_headers.iter().enumerate() {
let nonce = char::from(b'1' + index as u8);
let path = plant(&root, nonce, hostile, std::slice::from_ref(&good));
assert!(send_dead(&context).is_empty(), "{hostile}");
assert!(!path.exists(), "{hostile}");
}
let mut stale = header(Surface::Serve, "0.10.3");
stale.schema_version = SCHEMA_VERSION + 1;
plant(
&root,
'f',
&serde_json::to_string(&stale).expect("serialize"),
std::slice::from_ref(&good),
);
assert!(send_dead(&context).is_empty());
assert!(dryrun_batches(&root).is_empty());
}
#[test]
fn a_hostile_event_in_a_dead_spool_is_dropped_and_the_rest_sent() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
plant(
&root,
'b',
&header_line(Surface::Serve, "0.10.3"),
&[
r#"{"event":"panic","site":"/Users/victim/src/secret-repo/main.rs"}"#.to_string(),
event_line(SessionSource::Interactive),
],
);
assert_eq!(send_dead(&context).len(), 1);
let batches = dryrun_batches(&root);
assert_eq!(batches[0].events.len(), 1);
assert!(!serde_json::to_string(&batches).unwrap().contains("victim"));
}
#[test]
fn the_count_and_age_caps_hold() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
let now = std::time::SystemTime::now();
let mut paths = Vec::new();
for (index, nonce) in ['1', '2', '3', '4', '5', '6'].into_iter().enumerate() {
let path = plant(
&root,
nonce,
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
let age = Duration::from_secs(60 * (index as u64 + 1));
std::fs::File::options()
.write(true)
.open(&path)
.expect("open")
.set_modified(now - age)
.expect("set mtime");
paths.push(path);
}
let stale = plant(
&root,
'7',
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
std::fs::File::options()
.write(true)
.open(&stale)
.expect("open")
.set_modified(now - Duration::from_secs(8 * 24 * 60 * 60))
.expect("set mtime");
assert_eq!(send_dead(&context).len(), spool::MAX_RECOVERED);
assert_eq!(dryrun_batches(&root).len(), spool::MAX_RECOVERED);
for path in paths.iter().chain([&stale]) {
assert!(!path.exists(), "{} must be gone", path.display());
}
}
#[test]
fn two_processes_never_mix_lines() {
let home = temp_home();
let root = root_of(&home);
let (tui_path, _) = spool::open(&root, &header(Surface::Tui, "0.10.3"));
let (exec_path, _) = spool::open(&root, &header(Surface::Exec, "0.10.3"));
assert_ne!(tui_path, exec_path);
buffer::append(&root, &tui_path, &event_line(SessionSource::Interactive)).expect("append");
buffer::append(&root, &exec_path, &event_line(SessionSource::Ci)).expect("append");
let tui = spool::take_own(&root, &tui_path);
assert!(tui.iter().any(|l| l.contains("interactive")));
assert!(
tui.iter()
.all(|l| !l.contains("\"ci\"") && !l.contains("exec"))
);
assert!(exec_path.exists(), "taking one spool leaves the other");
}
#[test]
fn the_ring_keeps_the_header() {
let home = temp_home();
let root = root_of(&home);
let (path, _) = spool::open(&root, &header(Surface::Serve, "0.10.3"));
for n in 0..700 {
buffer::append(&root, &path, &line(n)).expect("append");
}
let lines = buffer::read_lines(&path);
assert!(lines.len() <= buffer::MAX_EVENTS);
assert!(spool::is_header_line(&lines[0]));
assert!(lines.last().unwrap().contains(&line(699)));
}
}
+6 -1
View File
@@ -11,9 +11,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Non-ASCII file names:** `/diff`, the `@git` and `@diff` mentions and the Runtime API workspace diff now show paths with Chinese, emoji and other non-ASCII characters as written instead of as quoted octal escapes, and the turn context no longer lists the workspace root as an empty "Active paths" entry. Thanks [@luochen211](https://github.com/luochen211) (PR 7317).
- **Session names after a crash, and keep-forever snapshot retention:** a session name saved right after a crash that cut off the previous session-index line is no longer lost, because the new entry now starts on its own line; and a very large `[snapshots] max_age_days` (such as `9223372036854775807` for "keep forever") no longer wraps around and deletes every undo snapshot at session start. Thanks [@luochen211](https://github.com/luochen211) (PR 7316).
- **Blank environment variables:** exporting a variable empty (for example `CODEWHALE_MODEL=` from a compose `${VAR:-}` template) no longer hides its legacy alias. `DEEPSEEK_MODEL`, `DEEPSEEK_BASE_URL`, `KIMI_BASE_URL`, `DEEPSEEK_FORCE_HTTP1`, `DEEPSEEK_STREAM_OPEN_TIMEOUT_SECS` and the other provider alias variables now apply as they do when the primary is unset. Thanks [@luochen211](https://github.com/luochen211) (PR 7315).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (PR 7293).
- **MCP and language servers that send requests mid-call:** a tool call no longer fails with "neither a result nor an error" when an MCP server sends a `ping` or other request that reuses the call's id while the call is running, and a language server's `workspace/configuration` or similar request no longer ends a pending LSP request. Codewhale now answers `ping`, answers other server requests with "Method not found" (it never advertised them), keeps waiting for the real reply, and a refused answer over Streamable HTTP no longer drops the connection to legacy SSE. Thanks [@luochen211](https://github.com/luochen211) (PR 7319).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (#7293).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (PR 7293).
- **Telemetry setting default:** the settings schema API reports telemetry's default as on, matching what Codewhale does when you have not chosen.
- **Auto routing abstention:** when the decision model marks a tier, effort or shadow answer `abstained: true`, Auto no longer acts on it, even with `min_confidence = 0`: an abstained tier takes the local fallback, an abstained effort keeps the tier and drops only the effort, and the routing receipt lists the abstained questions.
- **Plugin index (Runtime API):** a new `GET /v1/apps/plugin-index` route lists plugins, skills, connectors, hooks and hand-written commands with kind, source and state, and `/v1/commands` rows say which plugin (`plugin_id`) added a command and whether a command came from the project (`workspace_sourced`) (CW-8092).
- **Model commands and routing:** `codewhale model list` now lists every custom endpoint you have configured, under its own name. `codewhale model resolve` no longer refuses a model on a custom endpoint, router or local server whose model list has not been fetched yet; once a list is fetched, a model missing from it is still refused. Automatic routing marks a context window as verified only when the route itself supplies it, and no longer reports a guessed output limit.
- **ACP:** when a plugin tool asks the core to run a tool that needs approval, the editor now gets a `session/request_permission` on the plugin tool's call that names the nested tool and its input; before, the prompt ended with an internal error (CW-8050).
- **README and contributor credit:** the repository front page now opens the way the website does: the line "The model can be anyone's. The agent should be yours.", one sentence on what Codewhale is, the install command and one picture, with reference detail moved into the linked guides; the 18 translated READMEs follow the same structure and say they are machine-drafted. The contributor wall and the website now count the same 433 people (407 GitHub accounts and the commit authors whose email is not linked to an account), where the website had counted accounts only. `DEPENDENCY_SECURITY.md` moved into `docs/dependency-maintenance.md`, and `.github/SUPPORT.md` says where to ask for help.
- **Website Plugins page:** The website's Plugins page now describes what the released version does, and a labelled section says what 0.10.3 adds: one Plugins place, Standard and Creator mode, and the Permissions control.
- **First-time guide on the website:** a new docs page, "Use Codewhale for the first time" (`/docs/first-time`, English and Simplified Chinese), walks someone who has never used a terminal through opening one, installing, choosing the folder Codewhale works in and when that choice matters, opening a terminal in a folder from Finder or File Explorer, connecting a model, a first task, and stopping, undoing and quitting. The install page gains a short "New to the terminal?" block for macOS and Windows that links to it, and `llms.txt` carries the same steps.
+1
View File
@@ -85,6 +85,7 @@ mod tests {
aliases: aliases.iter().map(|s| s.to_string()).collect(),
hidden,
plugin_authority: None,
workspace_sourced: false,
extension: None,
}
}
+5
View File
@@ -296,6 +296,11 @@ impl codewhale_engine::command_catalog::CommandCatalog for CommandTable {
takes_arguments: metadata.takes_arguments(),
hidden: metadata.hidden,
extension: metadata.extension.is_some(),
plugin_id: metadata
.plugin_authority
.as_ref()
.map(|authority| authority.plugin_id.as_str().to_string()),
workspace_sourced: metadata.workspace_sourced,
builtin_bindings: bindings.remove(&metadata.name).unwrap_or_default(),
})
.collect()
+4
View File
@@ -80,6 +80,7 @@ pub struct UserCommandMetadata {
pub aliases: Vec<String>,
pub hidden: bool,
pub plugin_authority: Option<codewhale_config::PluginAuthority>,
pub workspace_sourced: bool,
/// Set for a command contributed by an extension-host plugin: it runs in
/// the host (`command/run`) instead of expanding `body`.
pub extension: Option<codewhale_engine::extension_host::command::ExtensionCommandRef>,
@@ -246,6 +247,7 @@ impl UserCommandRegistry {
let (mut metadata, errors) =
parse_metadata(entry.name, &entry.content, &entry.path);
metadata.plugin_authority = entry.plugin_authority;
metadata.workspace_sourced = entry.workspace_sourced;
(metadata, errors, entry.path, entry.workspace_sourced)
})
.collect::<Vec<_>>();
@@ -395,6 +397,7 @@ impl UserCommandRegistry {
aliases: Vec::new(),
hidden: false,
plugin_authority: Some(entry.authority),
workspace_sourced: false,
extension: Some(reference),
},
);
@@ -482,6 +485,7 @@ fn parse_metadata(
aliases: Vec::new(),
hidden: false,
plugin_authority: None,
workspace_sourced: false,
extension: None,
};
let mut configured_name = None;
+60 -8
View File
@@ -862,10 +862,6 @@ fn spawn_signal_cleanup_task() {
let mut signals = TerminatingSignals::register();
tokio::spawn(async move {
let exit_code = signals.wait().await;
codewhale_telemetry::record_session_end_blocking(codewhale_telemetry::ExitClass::Signal);
let telemetry_send = tokio::task::spawn_blocking(|| {
codewhale_telemetry::flush_for_termination(codewhale_telemetry::SIGNAL_FLUSH_TIMEOUT)
});
// Persistence and a serving Runtime API share a bounded window to
// flush queued saves and end open streams with a typed `stream.end`.
// A second signal skips both waits.
@@ -874,9 +870,6 @@ fn spawn_signal_cleanup_task() {
tokio::join!(
codewhale_app_server::runtime_api::drain_for_signal_exit(),
crate::tui::persistence_actor::flush_for_signal_exit(),
async {
let _ = telemetry_send.await;
},
)
} => {}
_ = signals.wait() => {}
@@ -892,11 +885,66 @@ fn spawn_signal_cleanup_task() {
#[cfg(unix)]
codewhale_runtime::process_tree::kill_contained_trees_for_exit();
crate::tui::ui::emergency_restore_terminal();
// Nothing async survives the `exit` below, so this is the last
// chance to say how the session ended. `record_blocking` is one
// `O_APPEND` write with no lock: taking the compaction lock here
// would let a second Codewhale process sharing CODEWHALE_HOME hang
// Ctrl-C, and the second-signal short-circuit below has to stay
// reachable. A no-op unless this process was armed.
//
// The class is stated, not derived: `RunTerminationReason::Canceled`
// also exits 130, so `exit_code` cannot tell a signal from an
// Esc-cancelled turn.
record_signal_session_end();
}
std::process::exit(exit_code);
});
}
/// When this process's armed telemetry session began. Set once, at arming, and
/// read from both the ordinary teardown and the signal path.
static TELEMETRY_SESSION_START: std::sync::OnceLock<std::time::Instant> =
std::sync::OnceLock::new();
/// Build `session_end` from what this process actually accumulated.
///
/// The exit class is read from the process-wide atomic and never derived from
/// an exit code: `RunTerminationReason::Canceled` maps to 130, the same value
/// the SIGINT path uses, so a code-based derivation would report every
/// Esc-cancelled turn as a signal.
///
/// The cold-start bucket is `None` unless the interactive event loop actually
/// began, which is what keeps it absent rather than invented on the surfaces
/// that have no event loop.
fn telemetry_session_end() -> codewhale_telemetry::Event {
let counters = codewhale_telemetry::session_counters();
codewhale_telemetry::Event::SessionEnd {
duration_bucket: codewhale_telemetry::DurationBucket::from_secs(
TELEMETRY_SESSION_START
.get()
.map_or(0, |start| start.elapsed().as_secs()),
),
exit_class: codewhale_telemetry::exit_class(),
cold_start_bucket: codewhale_runtime::startup_trace::cold_start_ms()
.map(codewhale_telemetry::ColdStartBucket::from_millis),
providers: counters.providers(),
counters: counters.counters(),
errors: counters.errors(),
turn_wall: counters.turn_wall(),
}
}
/// Close the session synchronously, from the signal handler.
///
/// A no-op unless this process was armed.
fn record_signal_session_end() {
codewhale_telemetry::set_exit_class(codewhale_telemetry::ExitClass::Signal);
for event in codewhale_telemetry::session_counters().usage_events() {
codewhale_telemetry::record_blocking(event);
}
codewhale_telemetry::record_blocking(telemetry_session_end());
}
/// Terminating-signal streams, registered up front and awaited later.
///
/// Splitting registration from the await is the point: the OS disposition
@@ -2466,6 +2514,7 @@ fn arm_telemetry_with_setup(
return;
};
codewhale_telemetry::init(consent.with_config_path(Some(store.path().to_path_buf())));
let _ = TELEMETRY_SESSION_START.set(std::time::Instant::now());
codewhale_telemetry::record(codewhale_telemetry::Event::SessionStart { source });
}
@@ -2531,7 +2580,10 @@ async fn finish_telemetry(outcome: &Result<()>, surface: codewhale_telemetry::Su
{
codewhale_telemetry::set_exit_class(codewhale_telemetry::ExitClass::Error);
}
codewhale_telemetry::record_session_end();
for event in codewhale_telemetry::session_counters().usage_events() {
codewhale_telemetry::record(event);
}
codewhale_telemetry::record(telemetry_session_end());
if surface == codewhale_telemetry::Surface::Cli {
let persistence = codewhale_telemetry::persist_local_blocking();
logging::info(format!(
+1 -1
View File
@@ -227,7 +227,7 @@ fn the_session_source_distinguishes_resume_and_fork_from_a_fresh_launch() {
#[test]
fn a_session_end_built_without_arming_writes_nothing() {
let event = codewhale_telemetry::session_end_event();
let event = telemetry_session_end();
assert!(matches!(
event,
codewhale_telemetry::Event::SessionEnd { .. }
+1
View File
@@ -1771,6 +1771,7 @@ mod tests {
.collect(),
confidence_bp: 6400,
min_confidence_bp: 5000,
abstained_questions: Vec::new(),
cost_saving_kept_fast: false,
thinking: Some("max".to_string()),
provider_reported_cost_usd: Some("0.000019992".to_string()),
-3
View File
@@ -1418,9 +1418,6 @@ pub async fn run_tui(
// than through it. Only this path calls it, which is what keeps the
// cold-start bucket absent on surfaces with no event loop.
codewhale_runtime::startup_trace::mark_cold_start();
if let Some(millis) = codewhale_runtime::startup_trace::cold_start_ms() {
codewhale_telemetry::set_cold_start_ms(millis);
}
let result = run_event_loop(
&mut terminal,
&mut app,
@@ -51,7 +51,7 @@ const SENTINEL_FILENAME: &str = "tc-workspace-sentinel-file.txt";
const SENTINEL_PROVIDER_TABLE: &str = "tc_custom_provider_sentinel";
const SENTINEL_MCP_SERVER: &str = "tc-mcp-server-sentinel";
const SENTINEL_API_KEY: &str = "tc-api-key-sentinel-not-a-real-key";
/// Planted by writing to a spool file directly, which is what any other
/// Planted by writing to `buffer.jsonl` directly, which is what any other
/// process running as this user can do.
const SENTINEL_INJECTED: &str = "tc-injected-sentinel-/Users/victim/secret-repo";
/// The key lives only in the child's environment, never in a file, so the
@@ -279,43 +279,22 @@ async fn assert_no_batches(server: &MockServer, why: &str) {
);
}
fn spool_files(fixture: &Fixture) -> Vec<PathBuf> {
let Ok(entries) = std::fs::read_dir(fixture.telemetry_root().join("spool")) else {
return Vec::new();
};
let mut files: Vec<PathBuf> = entries
.filter_map(Result::ok)
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "jsonl"))
.collect();
files.sort();
files
}
fn only_spool_file(fixture: &Fixture) -> Option<PathBuf> {
let mut files = spool_files(fixture);
(files.len() == 1).then(|| files.remove(0))
}
fn buffered_events(fixture: &Fixture) -> Vec<Value> {
let mut events = Vec::new();
for path in spool_files(fixture) {
let body = std::fs::read_to_string(&path).unwrap_or_else(|error| {
panic!(
"read locally buffered telemetry at {}: {error}",
path.display()
)
});
for line in body.lines().filter(|line| !line.trim().is_empty()) {
let value = serde_json::from_str::<Value>(line).unwrap_or_else(|error| {
let path = fixture.telemetry_root().join("buffer.jsonl");
let body = std::fs::read_to_string(&path).unwrap_or_else(|error| {
panic!(
"read locally buffered telemetry at {}: {error}",
path.display()
)
});
body.lines()
.filter(|line| !line.trim().is_empty())
.map(|line| {
serde_json::from_str::<Value>(line).unwrap_or_else(|error| {
panic!("buffered telemetry must be JSON: {error}\nline: {line}")
});
if value.get("event").is_some() {
events.push(value);
}
}
}
events
})
})
.collect()
}
async fn assert_short_cli_persistence_without_network(
@@ -401,12 +380,8 @@ async fn incomplete_short_cli_sessions_require_an_explicit_timeout() {
let server = start_recorder().await;
let fixture = Fixture::new().with_endpoint(&server.uri());
std::fs::create_dir_all(fixture.telemetry_root()).expect("create local buffer root");
std::fs::create_dir_all(fixture.telemetry_root().join("spool")).expect("create spool dir");
std::fs::write(
fixture
.telemetry_root()
.join("spool")
.join(format!("{}.jsonl", "a".repeat(32))),
fixture.telemetry_root().join("buffer.jsonl"),
"{\"event\":\"session_start\",\"source\":\"unknown\"}\n",
)
.expect("write incomplete session");
@@ -870,10 +845,8 @@ async fn a_hostile_buffer_line_never_reaches_a_batch() {
let stdout = read_in_background(child.stdout.take().expect("stdout pipe"));
let stderr = read_in_background(child.stderr.take().expect("stderr pipe"));
wait_until(Duration::from_secs(30), || {
only_spool_file(&fixture).is_some()
});
let buffer = only_spool_file(&fixture).expect("this session's spool");
let buffer = fixture.telemetry_root().join("buffer.jsonl");
wait_until(Duration::from_secs(30), || buffer.exists());
append_lines(
&buffer,
&[
@@ -956,10 +929,11 @@ async fn mid_session_opt_out_stops_the_shutdown_flush() {
// Wait until this session's event is actually buffered, then leave enough
// time for any accidental background flush to reach the recorder. Nothing
// may be sent before shutdown.
let buffer = fixture.telemetry_root().join("buffer.jsonl");
wait_until(Duration::from_secs(30), || {
only_spool_file(&fixture)
.and_then(|buffer| std::fs::read_to_string(buffer).ok())
.is_some_and(|body| body.contains("\"event\":\"session_start\""))
std::fs::read_to_string(&buffer)
.map(|body| body.contains("\"event\":\"session_start\""))
.unwrap_or(false)
});
tokio::time::sleep(Duration::from_millis(250)).await;
assert_no_batches(&server, "before the shutdown flush").await;
@@ -1007,7 +981,7 @@ async fn ctrl_c_exits_while_a_second_process_holds_the_lock() {
// Arming itself takes the lock, so the holder below must wait until the
// session is armed — otherwise this test would pin a deadlock it created.
wait_until(Duration::from_secs(30), || {
only_spool_file(&fixture).is_some()
root.join("buffer.jsonl").exists()
});
let _holder = LockHolder::take(&lock_path);
+1 -1
View File
@@ -545,7 +545,7 @@ Return `{kind:'deny', reason}`, `{kind:'ask', reason?}`, `{kind:'revise', input}
Native hooks run first, then mod listeners in core registration order. Denial always wins; the last accepted input revision wins. Rust re-prepares revised arguments and reruns the existing authority, policy, and approval checks. Input revisions use the existing 32 KiB hook limit; context and reasons use the existing sanitizers. Unloading a fiber removes its listener, and disabling or revoking an owner retires its admitted handles before host teardown. Other workspaces do not receive the call.
This bridge runs on Engine sessions, including model tools, gated code-mode calls, and `exec.core` calls. Sessions an editor drives over ACP (`codewhale serve --acp`) get the same extension host, under the same `[features]` flags (`extension_host`, `mcp`): reviewed plugin tools (TypeScript and DSH) and the MCP servers configured in Codewhale appear as ACP tool calls and ask through the session's permission setting; Plan stays read-only. The ACP server's capabilities say so (`agentCapabilities._meta.codewhale`). Servers an ACP client passes in `mcpServers` and configured shell hooks are not attached. Prepend/global listener ordering, DSH runtime/agent handles, around-execution wrappers, and post-result rewriting are not provided. Native `tool_call_after` remains an observer contract.
This bridge runs on Engine sessions, including model tools, gated code-mode calls, and `exec.core` calls. Sessions an editor drives over ACP (`codewhale serve --acp`) get the same extension host, under the same `[features]` flags (`extension_host`, `mcp`): reviewed plugin tools (TypeScript and DSH) and the MCP servers configured in Codewhale appear as ACP tool calls and ask through the session's permission setting; Plan stays read-only. When a plugin tool asks the core to run a tool that needs approval (`exec.core`), the editor's permission request is shown on the plugin tool's call and names the nested tool and its input. The ACP server's capabilities say so (`agentCapabilities._meta.codewhale`). Servers an ACP client passes in `mcpServers` and configured shell hooks are not attached. Prepend/global listener ordering, DSH runtime/agent handles, around-execution wrappers, and post-result rewriting are not provided. Native `tool_call_after` remains an observer contract.
## Asking the core to run a tool
+41 -1
View File
@@ -886,6 +886,13 @@ Plugin tools and configured MCP servers on this transport are on main after
0.10.2; 0.10.2 itself does not have them.
Client-supplied MCP servers, dynamic tools, task/PTY/background shell,
interpreter, subagent and RLM lifecycles are unavailable on this transport.
ACP has no tool search, so every connected MCP tool and its schema is sent with
every request, and a server marked `lazy = true` never starts. A large MCP set
therefore costs tokens on each turn and can exceed a provider's tool limit.
MCP servers a model starts at run time stay out of an ACP-profile host. When
`serve --acp` instead attaches to a running Runtime owner (see below), its turns
use that owner's MCP pool, which includes servers started at run time in other
sessions of the same workspace.
Built-in overrides remove the
whole compatibility alias family. Final dispatch rechecks the profile, so a
fabricated alias, hook rewrite or omitted catalog entry cannot bypass it. Plan
@@ -906,7 +913,11 @@ produced.
Tool updates begin `pending`. `in_progress` means Core reached final dispatch;
`completed`/`failed` and typed image blocks come from the actual Core result.
`session/request_permission` correlates a private JSON-RPC ID to the same
Runtime-minted pending approval and Core execution ID. Only an exact live
Runtime-minted pending approval and Core execution ID. When a plugin tool asks
the core to run a tool that needs approval (`exec.core`), the request is shown
on the plugin tool's call, names the nested tool and carries its input, and the
answer releases the nested call; the nested call is decided under the session's
permission setting. Only an exact live
`allow-once` response can release that waiter; wrong IDs are ignored, invalid
options deny, and cancellation withdraws it. ACP grants neither remembered
permissions nor Native capabilities. Core typed rules, strict hooks, repo law,
@@ -2672,6 +2683,35 @@ thread deletion.
pure over the snapshot — so a client must not look for one. Anything that
mutates the session stays a terminal-side command.
**Plugin index**
- `GET /v1/apps/plugin-index` — `{workspace, items: [...], unavailable: [...],
problems: [...]}`: one list across every plugin, skill, connector, hook and
hand-written command. It is a projection: each item's `row` is the row its
owner's own route returns (`/v1/apps/plugins`, `/v1/apps/plugins/catalog`,
`/v1/skills`, `/v1/apps/mcp/servers`, `/v1/hooks`, `/v1/commands`), and
`owner` names that route. Per item: `id`, `kind` (`plugin`, `skill`,
`connector`, `hook`, `command`), `name`, `description`, `source`
(`built_in`, `user`, `project`, `other_app`, `runtime`, `plugin`), `state`
(`enabled`, `disabled`, `needs_review`, `needs_sign_in`, `unavailable`) and
`labels` (what the item adds: `skill`, `connector`, `hook`, `command`,
`tool`, `profile`). A plugin item also has `adds` (`skills`, `connectors`,
`hooks`, `commands`, `profiles`: names, not rows), and what a plugin adds is
not listed a second time as an item of its own. A built-in switch is a
`plugin` item with a `tier` and no labels. `/v1/commands` rows carry
`plugin_id` (the same id as `/v1/apps/plugins`, and the `plugin:<id>` item)
for a command a plugin adds, and `workspace_sourced` for one read from the
project; a command item's `plugin` is that id. An owner that fails is named in
`unavailable` and the other kinds still list; `problems` carries the owners'
load warnings. Names under `adds` are file names, so a profile's `id` inside its
file can differ; a command another plugin registers through the extension host
is an item with `source` `plugin`. For an active plugin `adds.commands` comes
from the `/v1/commands` rows; for any other plugin, and for `hooks` and
`profiles`, it is the file names found in the bundle, at most 512 per kind,
with names holding control characters left out; a cut list is reported
under `problems`. A hook's row names no plugin, so `adds.hooks` stays file
names. `tool` means the plugin declares native or language-server components;
a screen a plugin draws is not labelled yet.
**Hooks**
- `GET /v1/hooks[?thread_id=...]` — `{workspace, enabled, hooks: [...],
problems: [...]}`: the hook set Runtime API threads run for that workspace
+2 -1
View File
@@ -54,6 +54,7 @@ registration index.
| POST | `/v1/apps/mcp/servers/{name}/enable` |
| POST | `/v1/apps/mcp/servers/{name}/reconnect` |
| GET | `/v1/apps/mcp/tools` |
| GET | `/v1/apps/plugin-index` |
| GET | `/v1/apps/plugins` |
| DELETE / GET | `/v1/apps/plugins/{selector}` |
| POST | `/v1/apps/plugins/{selector}/disable` |
@@ -263,4 +264,4 @@ registration index.
| GET / POST | `/v1/worktrees` |
| POST | `/v1/worktrees/remove` |
The source contains 243 distinct literal paths in this table.
The source contains 244 distinct literal paths in this table.
+20 -49
View File
@@ -65,11 +65,11 @@ codewhale --telemetry false # the same kill switch, for one command
**`telemetry = false` in the config file is the opt-out.** It is a floor:
`--telemetry true` and `CODEWHALE_TELEMETRY=1` both lose to it, because a
setting you can undo by accident from a wrapper script is not a setting. It
deletes the random install id, deletes every spool, truncates every dry-run
deletes the random install id, truncates every buffered event and every dry-run
record, and writes a tombstone. Appends, identity/state writes, and delivery all
share the wipe's ordering lock, so once opt-out returns no pre-opt-out write or
POST remains in flight. If any part of that wipe fails, the tombstone is still
there and no spool can be sent — a failed wipe fails closed. Every later
there and the buffer is undrainable — a failed wipe fails closed. Every later
run re-asserts the same tombstone for as long as the setting stands, so it
survives. Turning telemetry back on in `/settings` records that explicit preference
in both existing privacy registers; a fresh launch can then clear the tombstone. Nothing buffered before that point is
@@ -101,21 +101,17 @@ Everything is under `$CODEWHALE_HOME/telemetry/` (`0700`), every file `0600`:
| file | role |
|---|---|
| `spool/<nonce>.jsonl` | one running process's pending events: a header line (schema, notice and app version, release sha, surface, whether a terminal), then one JSON object per line |
| `spool/<nonce>.lock` | held by that process while it runs; a spool whose lock is free belongs to a process that has ended |
| `buffer.jsonl` | pending events, one JSON object per line |
| `buffer.jsonl.lock` | the sibling ordering lock shared by writes, delivery, arming, and wipe |
| `buffer.jsonl` | left over from older versions; only ever emptied, never written or sent |
| `dryrun.jsonl` | where batches go when the endpoint is configured empty |
| `state.json` | the last app version seen and the last flush attempt |
| `install_id.json` | the random install id and when it was minted |
| `disabled` | the tombstone; present means nothing is appended or sent |
The nonce is random, names a file on your disk and is never sent. Each spool and
`dryrun.jsonl` is a ring capped at 512 records (the header counts as one) or
256 KiB, whichever comes first, with the oldest events dropped. The documented
footprint ceiling is therefore **256 KiB for each running process, 256 KiB for
`dryrun.jsonl`, and at most four leftover spools of ended processes kept after
each TUI, `exec` or `serve` start**, plus a few hundred bytes of metadata.
Both `buffer.jsonl` and `dryrun.jsonl` are rings capped at 512 records or
256 KiB, whichever comes first, with the oldest dropped. The documented
footprint ceiling for the whole directory is therefore **512 KiB plus a few
hundred bytes of metadata**.
The install id is a random v4 UUID. It is never derived from your hostname,
MAC address, `machine-id`, home directory, username, or executable path — a
@@ -130,43 +126,18 @@ claim one.
## When anything is sent, and where
Nothing is collected or sent when a persistent opt-out or run-scoped kill
switch is in force, or when privacy state is unreadable. TUI and `exec` sessions have one network flush point at a normal end: an attempt
during shutdown, bounded at three seconds. That flush sends only the process's
own spool, under that process's own surface and version, and deletes it before
the request is made. Short CLI commands such as `config`, `doctor`, and `auth`
do not wait for that network request. They record `session_end`, seal the event
to their own spool with a much shorter bound, and return; the next TUI, `exec`
or `serve` start sends what they left (below). An explicitly empty endpoint
finalizes its local dry-run batch immediately.
switch is in force, or when privacy state is unreadable. TUI and `exec` sessions have exactly one network flush point: an attempt
during shutdown, bounded at three seconds. Short CLI commands such as `config`,
`doctor`, and `auth` do not wait for that network request. They record
`session_end`, seal the event to the local buffer with a much shorter bound, and
return; a configured endpoint sends those buffered events with the next
interactive shutdown flush. An explicitly empty endpoint finalizes its local
dry-run batch immediately.
A session that is ended early makes one more bounded attempt, so its usage is
not lost with it. When a terminating signal (interrupt, terminate, hang-up, or
console close) reaches a TUI, `exec` or `serve` session, and when `exec` ends
because the program reading its output has closed the pipe, the session records
`session_end` with exit class `signal`, then makes one send attempt bounded at
one and a half seconds, alongside the process's own shutdown work. A second
signal ends the process at once without waiting. This is the same single `POST`
of the same fields, never retried. Short CLI commands stay local-only on this
path too.
A session that ends without any report (killed outright, a power loss, a crash
before the handler runs) leaves its spool behind. When a later TUI, `exec` or
`serve` session starts, it sends the leftovers once: at most four, newest first,
one `POST` per leftover session, each carrying that session's own surface and
app version and this installation's current install id and send time. Each
leftover is deleted before its request is made and is never retried. A leftover
older than seven days, one beyond the newest four, one written by a build with a
different schema or notice version, and any line that fails the schema bounds is
deleted without being sent. Consent and the opt-out tombstone are checked
before anything is read and again before each request, and the request runs on
its own thread so it never delays your session. Opting out deletes every spool.
Opting back in starts without any spool from before.
There is no mid-session flush, per-turn flush, or per-tool-call
flush. Every network flush re-resolves your setting from disk
There is no startup flush, mid-session flush, per-turn flush, or per-tool-call
flush. Every network shutdown flush re-resolves your setting from disk
immediately beforehand, so `codewhale config set telemetry false` written from
another terminal stops the flush of a session that is already running, including
one that is being terminated.
another terminal stops the flush of a session that is already running.
A flush is **one `POST`** to the resolved endpoint — by default
`https://telemetry.codewhale.net/v1/telemetry`. The request carries a
@@ -235,7 +206,7 @@ new fields, surfaces, and `product_usage` events.
| `arch` | enum | `std::env::consts::ARCH` | `x86_64 \| aarch64 \| other`. |
| `libc` | enum | `cfg!(target_env)` — **compile time** | `gnu \| musl \| none`. Runtime detection reads distro vendor strings; compile-time is free and leaks nothing. |
| `tty` | bool | `std::io::IsTerminal`, as at `crates/telemetry/src/envelope.rs:196` | `stdin().is_terminal() && stdout().is_terminal()`. |
| `events` | array | the process's spool | Every element is one of the six events below and nothing else. Capped at 200 events or 64 KiB per batch; events beyond either cap are dropped with the spool, never carried to a later flush. |
| `events` | array | the drained buffer | Every element is one of the six events below and nothing else. Capped at 200 events or 64 KiB per batch; a batch that would exceed either cap leaves the remainder buffered for the next flush. |
**`os_major` is not collected.** Reading it costs unsafe FFI on two platforms plus a file parser on a third, in the one crate whose entire value is being small enough to audit — and `os`, `arch`, and `libc` are free and answer the platform question. It may be reconsidered if the stored data ever shows that the OS-version cut is what triage is missing; a hunch is not that evidence.
@@ -346,7 +317,7 @@ The workhorse. Everything a session accumulated ships here, once.
**`duration_bucket`** — `chrono` delta from `app.session_started_at` (`crates/tui/src/tui/app.rs:1889`). Half-open, seconds: `lt_1m` (`d < 60`), `1m_10m` (`60 ≤ d < 600`), `10m_60m` (`600 ≤ d < 3600`), `gt_60m` (`d ≥ 3600`).
**`exit_class`** — `clean | signal | panic | error`. **Derived from an explicit `AtomicU8`, never from an exit code.** `RunTerminationReason::Canceled` maps to exit 130 (`crates/engine/src/core/runtime_contract/termination.rs:53`), the same value the signal task uses (`crates/tui/src/lib.rs:682`, 128+SIGINT), so a code-based derivation would report every Esc-cancelled turn as a signal. The atomic is set by the panic hook (`crates/tui/src/lib.rs:1582`), by the signal task when the first signal arrives, by `exec` when its output pipe closes, and on the clean path from `RunTerminationReason::is_success()` (`crates/engine/src/core/runtime_contract/termination.rs:44-46`) — `error` otherwise. Exit codes remain insufficient: an approval-required exit (3) and a terminal exec failure (1) need their actual termination classification.
**`exit_class`** — `clean | signal | panic | error`. **Derived from an explicit `AtomicU8`, never from an exit code.** `RunTerminationReason::Canceled` maps to exit 130 (`crates/engine/src/core/runtime_contract/termination.rs:53`), the same value the signal task uses (`crates/tui/src/lib.rs:682`, 128+SIGINT), so a code-based derivation would report every Esc-cancelled turn as a signal. The atomic is set by the panic hook (`crates/tui/src/lib.rs:1582`), by the signal task (`:678-696`) before `std::process::exit`, and on the clean path from `RunTerminationReason::is_success()` (`crates/engine/src/core/runtime_contract/termination.rs:44-46`) — `error` otherwise. Exit codes remain insufficient: an approval-required exit (3) and a terminal exec failure (1) need their actual termination classification.
**`cold_start_bucket`** — from `startup_trace::elapsed_ms()`, which reads `PROCESS_START` directly and is independent of the startup summary's buffer clear (`crates/runtime/src/startup_trace.rs:39-45`). Boundaries: `lt_250`, `250_1000`, `1000_3000`, `gte_3000`. Absent on non-TUI surfaces.
@@ -570,7 +541,7 @@ is established by this source change.
`install_id` rotates client-side every 90 days (`rotated_at` in `install_id.json`), so no single identifier spans a long history. This costs longitudinal accuracy and the docs say so: **no count derived from `install_id` is a user count.** It is a lower bound on distinct machine-installs in a window, and it undercounts a returning user across a rotation.
**Turning it off deletes what was kept locally, not what was already sent.** `codewhale config set telemetry false` erases the install id, every spool, and the dry-run records on your machine, and stops anything further. Rows already accepted by the endpoint are keyed only by a rotating random id that is now gone; first-party rows age out with the three-month window. If PostHog processing is activated, already-delivered processor records follow that project’s configured retention. The client has no remote deletion API and does not claim that local opt-out erases delivered records.
**Turning it off deletes what was kept locally, not what was already sent.** `codewhale config set telemetry false` erases the install id, the buffer, and the dry-run records on your machine, and stops anything further. Rows already accepted by the endpoint are keyed only by a rotating random id that is now gone; first-party rows age out with the three-month window. If PostHog processing is activated, already-delivered processor records follow that project’s configured retention. The client has no remote deletion API and does not claim that local opt-out erases delivered records.
### What the owner reads back — observed active installs
+2
View File
@@ -252,6 +252,8 @@ DSH 的 `allow` 被视为不干预,每个所有者提示一次;不能预先
调用。编辑器通过 ACP(`codewhale serve --acp`)驱动的会话获得同一个扩展宿主,并受相同的
`[features]` 开关(`extension_host`、`mcp`)约束:已审查的插件工具(TypeScript 与 DSH)和
Codewhale 中配置的 MCP 服务器会作为 ACP 工具调用出现,并通过会话的权限设置询问;Plan 保持只读。
插件工具通过 `exec.core` 请求核心执行需要审批的工具时,审批请求显示在该插件
工具的调用上,并写明嵌套工具及其输入。
ACP 服务器的能力声明会说明这一点(`agentCapabilities._meta.codewhale`)。ACP 客户端在
`mcpServers` 中传入的服务器和已配置的 shell hooks 不会挂载。`prepend`/`global` 顺序
覆盖、包裹实际执行的中间件、DSH 运行时句柄和执行后结果改写均不支持。Native
+21 -2
View File
@@ -660,7 +660,11 @@ shell 工具,外加已审查的插件工具(`extension_host` 功能)和 Co
MCP 服务器的工具(`mcp` 功能),均受同一权限设置约束;这一点见于 0.10.2 之后的 main,0.10.2 本身不含。Shell 同时要求编辑器声明
terminal 支持、操作者允许 `allow_shell`;指定的外部沙箱不可用时不提供 shell。此接口
不提供客户端传入的 MCP 服务器、动态工具、任务、PTY、后台 shell、解释器、子智能体
或 RLM 生命周期。内置工具覆盖会移除整个兼容别名族。
或 RLM 生命周期。ACP 没有工具搜索,因此每个已连接的 MCP 工具及其 schema 都会随每个请求
发送,标记为 `lazy = true` 的服务器永不启动;MCP 集合很大时每回合都会消耗令牌,并可能超出
提供商的工具数量上限。模型在运行时启动的 MCP 服务器不会进入 ACP profile 宿主;但当
`serve --acp` 附着到正在运行的 Runtime owner 时(见下文),其回合使用该 owner 的 MCP 池,
其中包含同一工作区其他会话在运行时启动的服务器。内置工具覆盖会移除整个兼容别名族。
最终派发再次校验 profile;伪造别名、hook 改写或目录中缺失的工具不能绕过限制。
Full Access 下 Plan 仍只读。Full Access 与普通审批姿态是服务端拥有的只读选项,
编辑器不能放宽。`approval_policy = "never"` 不是 ACP 审批姿态,服务端会拒绝
@@ -669,7 +673,9 @@ Full Access 下 Plan 仍只读。Full Access 与普通审批姿态是服务端
工具首先显示 `pending`;只有 Core 到达最终派发才显示 `in_progress`。
`completed`/`failed` 和类型化图片块来自实际 Core 结果。审批请求的私有 JSON-RPC ID
绑定同一个 Runtime 铸造的待决审批与 Core 执行 ID。只有精确匹配、仍有效的
绑定同一个 Runtime 铸造的待决审批与 Core 执行 ID。插件工具通过 `exec.core` 请求核心运行
需要审批的工具时,请求显示在该插件工具的调用上,写明嵌套工具及其输入,答复释放嵌套调用,嵌套调用在会话的
权限设置下决定。只有精确匹配、仍有效的
`allow-once` 响应能释放等待器;错误 ID 被忽略,无效选项拒绝,取消会撤销等待器。
ACP 不授予记忆权限或 Native 能力。
@@ -1987,6 +1993,19 @@ POST 前保留预期指纹,并在响应及按键 GET 恢复时核对,之后
保留命令回执——每个提供的处理器对快照都是纯函数——因此客户端不应寻找回执。
任何会修改会话的操作仍是终端侧命令。
**插件索引**
- `GET /v1/apps/plugin-index` — `{workspace, items: [...], unavailable: [...],
problems: [...]}`:跨所有插件、技能、连接器、钩子和手写命令的统一列表。它只是投影:每个条目的
`row` 就是其所属路由(`/v1/apps/plugins`、`/v1/apps/plugins/catalog`、`/v1/skills`、
`/v1/apps/mcp/servers`、`/v1/hooks`、`/v1/commands`)返回的那一行,`owner` 给出该路由。
每个条目有 `id`、`kind`(`plugin`、`skill`、`connector`、`hook`、`command`)、`name`、
`description`、`source`(`built_in`、`user`、`project`、`other_app`、`runtime`、`plugin`)、
`state`(`enabled`、`disabled`、`needs_review`、`needs_sign_in`、`unavailable`)和
`labels`(它添加的内容:`skill`、`connector`、`hook`、`command`、`tool`、`profile`)。插件条目另有 `adds`(`skills`、`connectors`、`hooks`、`commands`、
`profiles`:只含名称),插件所添加的内容不会再作为独立条目重复列出。内置开关是带 `tier`、无 `labels` 的
`plugin` 条目。`/v1/commands` 的行对插件添加的命令带有 `plugin_id`(与 `/v1/apps/plugins` 的 id 相同,即 `plugin:<id>` 条目),对从项目读取的命令带有 `workspace_sourced`;命令条目的 `plugin` 即该 id。某个所属路由失败时记入 `unavailable`,其余种类照常列出;`problems`
汇总各所属路由的加载警告。`adds` 中的名称是文件名,因此档案文件内部的 `id` 可能不同;通过扩展宿主注册的命令是 `source` 为 `plugin` 的条目。对已启用的插件,`adds.commands` 取自 `/v1/commands` 的行;其他插件以及 `hooks`、`profiles` 取自包内的文件名,每类最多 512 个,含控制字符的名称被略去,列表被截断时在 `problems` 中说明。钩子的行不带插件名,因此 `adds.hooks` 仍是文件名。`tool` 表示插件声明了原生或语言服务器组件;插件绘制的界面暂无标签。
**钩子**
- `GET /v1/hooks[?thread_id=...]` — `{workspace, enabled, hooks: [...],
problems: [...]}`:Runtime API 线程为该工作区运行的钩子集合
+10 -10
View File
@@ -259,8 +259,8 @@
},
{
"path": "docs/TELEMETRY.md",
"sha256": "6ecccb66215a6aae5076cd00735028c10a50ce6e5902ce4151a9606ee86696a9",
"content": "# Codewhale product telemetry\n\n> 阅读简体中文版:[zh_hans/TELEMETRY.md](zh_hans/TELEMETRY.md)\n\n**Usage analytics are on by default in the current source.** The first\ninteractive launch gives a localized, nonblocking disclosure naming **Codewhale\nand PostHog**, with a direct route to turn counting off. Notice version `6`\ndescribes this policy; displaying it is not recorded as human acceptance.\nMissing an older acceptance record does not turn a fresh installation off.\nExisting explicit opt-outs remain off across upgrades, and unreadable privacy\nstate fails closed. Environment and CLI kill switches still take precedence.\n\nRead the disclosure with `codewhale config telemetry`. Turn usage off in\n`/settings` or run `codewhale config set telemetry false`. Turning it back on\nin Settings or with `codewhale config set telemetry true` updates the existing\ndurable preference and privacy records for new sessions. The compatibility\ncommand `codewhale config telemetry --accept-notice 5` remains available, but\nis not required for a fresh installation. Collection never includes work content or credentials.\n\nThe PostHog sink is optional and inert until the ingest operator explicitly\nconfigures a project token and approved regional host, and records staging\nproof that the actual egress path does not forward the original client IP.\nSource readiness does not establish a deployment, an activated processor,\nan egress receipt, or a retention setting.\n\nCodewhale does not collect conversations, code, prompts, files, file/repo/branch\nnames, model content, or credentials. It sends no per-turn or per-tool timeline.\nIt does send the closed, aggregate schema below: version and platform classes,\nsession duration/outcome, feature/error counters, and a random install id that\nrotates every 90 days.\n\n**There is now a real endpoint.** An enabled session sends its batches to the\nfirst-party ingest service at `https://telemetry.codewhale.net/v1/telemetry`,\nwhich is the shipped default for `telemetry_endpoint`. What that service is,\nwhat it stores, and what it structurally cannot store is spelled out in \"What\nthe endpoint does\" below.\n\n**To send nothing anywhere, turn telemetry off** (see \"Turning it off\"). To stay\nenabled but contact nobody, set `telemetry_endpoint = \"\"`: batches are then\nappended to `$CODEWHALE_HOME/telemetry/dryrun.jsonl` on your own machine, byte\nfor byte what the server would have received, and no HTTP client is ever\nconstructed. That file is how you audit this document against reality.\n\nThis document is the schema. It is not a summary of the schema: a test in\n`crates/telemetry` parses the field names out of this file and asserts set\nequality against the structs the serializer actually uses, so a field that is\nhere and not in the code — or in the code and not here — fails the build.\n\nLocal saved sessions and `codewhale metrics` remain available with telemetry\ndisabled and require no account or hosted reporting service. Disabling telemetry\ndoes not disable local usage receipts.\n\n## Turning it off\n\nThere are two off switches and they do different things. Both stop collection\ncompletely; only one of them erases anything.\n\n```sh\ncodewhale config set telemetry false # opt out: stops collection and erases state\nCODEWHALE_TELEMETRY=0 codewhale # kill switch: stops collection, erases nothing\nDO_NOT_TRACK=1 codewhale # the same kill switch, by the common convention\ncodewhale --telemetry false # the same kill switch, for one command\n```\n\n**`telemetry = false` in the config file is the opt-out.** It is a floor:\n`--telemetry true` and `CODEWHALE_TELEMETRY=1` both lose to it, because a\nsetting you can undo by accident from a wrapper script is not a setting. It\ndeletes the random install id, deletes every spool, truncates every dry-run\nrecord, and writes a tombstone. Appends, identity/state writes, and delivery all\nshare the wipe's ordering lock, so once opt-out returns no pre-opt-out write or\nPOST remains in flight. If any part of that wipe fails, the tombstone is still\nthere and no spool can be sent — a failed wipe fails closed. Every later\nrun re-asserts the same tombstone for as long as the setting stands, so it\nsurvives. Turning telemetry back on in `/settings` records that explicit preference\nin both existing privacy registers; a fresh launch can then clear the tombstone. Nothing buffered before that point is\never sent.\n\n**The environment variable and the flag are kill switches, not opt-outs.**\nTelemetry is off for the run, nothing is written, nothing is sent — and\nnothing on disk is touched or deleted. That is deliberate: a harness or agent\nthat sets `CODEWHALE_TELEMETRY=0` for one command must not silently discard the\ninstall id and the dry-run records of the person who owns the machine. If you\nwant the erasing kind, use the config file.\n\n`CODEWHALE_TELEMETRY` (andLine truncated
"sha256": "0ac65dbd0e1d9d196d1c4421116852c3ab6a855062c375c25b09db993084c563",
"content": "# Codewhale product telemetry\n\n> 阅读简体中文版:[zh_hans/TELEMETRY.md](zh_hans/TELEMETRY.md)\n\n**Usage analytics are on by default in the current source.** The first\ninteractive launch gives a localized, nonblocking disclosure naming **Codewhale\nand PostHog**, with a direct route to turn counting off. Notice version `6`\ndescribes this policy; displaying it is not recorded as human acceptance.\nMissing an older acceptance record does not turn a fresh installation off.\nExisting explicit opt-outs remain off across upgrades, and unreadable privacy\nstate fails closed. Environment and CLI kill switches still take precedence.\n\nRead the disclosure with `codewhale config telemetry`. Turn usage off in\n`/settings` or run `codewhale config set telemetry false`. Turning it back on\nin Settings or with `codewhale config set telemetry true` updates the existing\ndurable preference and privacy records for new sessions. The compatibility\ncommand `codewhale config telemetry --accept-notice 5` remains available, but\nis not required for a fresh installation. Collection never includes work content or credentials.\n\nThe PostHog sink is optional and inert until the ingest operator explicitly\nconfigures a project token and approved regional host, and records staging\nproof that the actual egress path does not forward the original client IP.\nSource readiness does not establish a deployment, an activated processor,\nan egress receipt, or a retention setting.\n\nCodewhale does not collect conversations, code, prompts, files, file/repo/branch\nnames, model content, or credentials. It sends no per-turn or per-tool timeline.\nIt does send the closed, aggregate schema below: version and platform classes,\nsession duration/outcome, feature/error counters, and a random install id that\nrotates every 90 days.\n\n**There is now a real endpoint.** An enabled session sends its batches to the\nfirst-party ingest service at `https://telemetry.codewhale.net/v1/telemetry`,\nwhich is the shipped default for `telemetry_endpoint`. What that service is,\nwhat it stores, and what it structurally cannot store is spelled out in \"What\nthe endpoint does\" below.\n\n**To send nothing anywhere, turn telemetry off** (see \"Turning it off\"). To stay\nenabled but contact nobody, set `telemetry_endpoint = \"\"`: batches are then\nappended to `$CODEWHALE_HOME/telemetry/dryrun.jsonl` on your own machine, byte\nfor byte what the server would have received, and no HTTP client is ever\nconstructed. That file is how you audit this document against reality.\n\nThis document is the schema. It is not a summary of the schema: a test in\n`crates/telemetry` parses the field names out of this file and asserts set\nequality against the structs the serializer actually uses, so a field that is\nhere and not in the code — or in the code and not here — fails the build.\n\nLocal saved sessions and `codewhale metrics` remain available with telemetry\ndisabled and require no account or hosted reporting service. Disabling telemetry\ndoes not disable local usage receipts.\n\n## Turning it off\n\nThere are two off switches and they do different things. Both stop collection\ncompletely; only one of them erases anything.\n\n```sh\ncodewhale config set telemetry false # opt out: stops collection and erases state\nCODEWHALE_TELEMETRY=0 codewhale # kill switch: stops collection, erases nothing\nDO_NOT_TRACK=1 codewhale # the same kill switch, by the common convention\ncodewhale --telemetry false # the same kill switch, for one command\n```\n\n**`telemetry = false` in the config file is the opt-out.** It is a floor:\n`--telemetry true` and `CODEWHALE_TELEMETRY=1` both lose to it, because a\nsetting you can undo by accident from a wrapper script is not a setting. It\ndeletes the random install id, truncates every buffered event and every dry-run\nrecord, and writes a tombstone. Appends, identity/state writes, and delivery all\nshare the wipe's ordering lock, so once opt-out returns no pre-opt-out write or\nPOST remains in flight. If any part of that wipe fails, the tombstone is still\nthere and the buffer is undrainable — a failed wipe fails closed. Every later\nrun re-asserts the same tombstone for as long as the setting stands, so it\nsurvives. Turning telemetry back on in `/settings` records that explicit preference\nin both existing privacy registers; a fresh launch can then clear the tombstone. Nothing buffered before that point is\never sent.\n\n**The environment variable and the flag are kill switches, not opt-outs.**\nTelemetry is off for the run, nothing is written, nothing is sent — and\nnothing on disk is touched or deleted. That is deliberate: a harness or agent\nthat sets `CODEWHALE_TELEMETRY=0` for one command must not silently discard the\ninstall id and the dry-run records of the person who owns the machine. If you\nwant the erasing kind, use the config file.\n\n`CODEWHALE_TELEMELine truncated
},
{
"path": "docs/public-surface-facts.json",
@@ -354,8 +354,8 @@
},
{
"path": "docs/EXTENSIONS.md",
"sha256": "45dc87bc8293156cc221a0d1181ddcdfe6970caef3874d2127a5b86ba2b32d9d",
"content": "# Writing an extension tool or command\n\nThe experimental TypeScript extension host runs reviewed plugin code in a\nshared Node process (or, as an opt-in, Bun). Rust still owns sessions, tool admission, approval and\nexecution. Extensions currently contribute tools and slash commands; they\ncannot provide an approval service, run a second agent loop or replace\nbuilt-in tools or commands.\n\n## Quick start\n\nStart with [hello-extension](examples/plugins/hello-extension/hello.mts) and its\n[plugin.json](examples/plugins/hello-extension/plugin.json). The example is\nexecuted unchanged by the host tests. It returns a greeting and does not read\nfiles or use the network.\n\n1. Explicitly enable `[features] extension_host = true` in your configuration,\n or start Codewhale with `codewhale --enable extension_host` (the flag goes\n before any subcommand).\n2. Install the example's directory using `/plugin install <local-directory>`.\n3. Run `/plugin validate hello-extension`, then `/plugin show hello-extension`.\n4. Run `/plugin enable hello-extension` to open its content/capability review.\n Read the review and personally run its exact `/plugin trust <token>` command,\n then enable the plugin. Trust alone does not execute the code.\n5. Ask Codewhale to use `hello_greet`. The tool follows the normal approval gate.\n6. Run `/hello-greet Codewhale`. The command runs when you type it, and shows\n its answer in the transcript with its origin, `extension:hello-extension`.\n\nAn authoring agent should stop after installation and validation and present\nthe review to the person, unless it runs in Full Access and the plugin is one\nit wrote and installed from a folder in its project: then it may trust and\nenable it for that project with `codewhale plugins trust` and `enable`. Those\nshell commands go through the session's approval like any other: Ask asks the\nperson, Auto-Review's reviewer judges, and Full Access runs them.\n\n## Locations, installation and trust\n\nUser bundles live under `~/.codewhale/plugins/`; workspace bundles live under\n`<workspace>/.codewhale/plugins/`. New bundles use Agent Plugins v1.0.0\n`plugin.json`. Declare `extensions[\"net.codewhale\"].native.path` as one regular\n`.mjs`, `.js` or `.mts` file inside the bundle. Directories and other extensions\nfail validation when the host is enabled. With the flag off, native entries\nremain inventory-only.\n\nA plugin may split its code across several entries with `native.paths` (up to\n64, alone or beside `path`). They are activated in the order declared, as the\nfibers of one plugin: they share one owner, so one disable, review change or\ncrash tears all of them down together. Each entry is a separate module with its\nown `apply`, and tool and command names must be unique across them. If an entry\nfails to activate (throws, requires a service the host does not provide, or has\na registration refused), only that entry is withdrawn: entries that already\nactivated stay registered, later entries still activate, and the plugin fails\nonly when no entry activates. Listing the same file twice is one entry.\n\n## Runtime\n\n`[extension_host] runtime` selects what runs the host. Node is the default.\nBun is an opt-in: it is qualified on macOS only, and becomes the default only\nafter an explicit, recorded cutover.\n\n```toml\n[extension_host]\nruntime = \"node\" # default: Node only\n# runtime = \"bun\" # Bun only; fails rather than falling back to Node\n# runtime = \"auto\" # Bun >= 1.4.0 if one is found and starts, otherwise Node\n# bun = \"~/.bun/bin/bun\" # the only Bun tried when set\n# node = \"/opt/homebrew/bin/node\" # the only Node tried when set\n# mcp_backend = \"rust\" # temporary opt-out; default is \"host\"\n```\n\nThe default `mcp_backend = \"host\"` uses the pinned MCP SDK independently of\n`[features] extension_host`, which controls optional Native extensions. Existing\nstdio and HTTP servers use modern discovery when supported; explicitly configured\nlegacy SSE servers use their legacy handshake. Rust retains process and network\nauthority, credential resolution, catalog admission, tool approval and cancellation.\nThe SDK receives opaque session selectors and exact operation grants. Modern\nrequests carry Rust-approved protocol metadata and headers without a session id.\n\nA typed discovery refusal can admit a legacy handshake. Only an explicit HTTP\ntransport refusal during that handshake can admit legacy SSE with a fresh exact\nRust grant. Legacy HTTP retains its GET preflight and bounded OAuth refresh;\nmodern HTTP does not inherit those session or resend behaviors. Incomplete result\nflows are reported without automatically repeating the operation. Credentials,\nconfigured URLs, OAuth browser/login operations and Computer Use decision keys\nstay in Rust.\n\nBoth backends share recorded semantic MCP goldens and retain separate transport\ngoldens, including discovery, authentication and sessioLine truncated
"sha256": "8582de93c01083314119639678c1ea1ab4ba46317c5a400fb9dd139ffdcb232a",
"content": "# Writing an extension tool or command\n\nThe experimental TypeScript extension host runs reviewed plugin code in a\nshared Node process (or, as an opt-in, Bun). Rust still owns sessions, tool admission, approval and\nexecution. Extensions currently contribute tools and slash commands; they\ncannot provide an approval service, run a second agent loop or replace\nbuilt-in tools or commands.\n\n## Quick start\n\nStart with [hello-extension](examples/plugins/hello-extension/hello.mts) and its\n[plugin.json](examples/plugins/hello-extension/plugin.json). The example is\nexecuted unchanged by the host tests. It returns a greeting and does not read\nfiles or use the network.\n\n1. Explicitly enable `[features] extension_host = true` in your configuration,\n or start Codewhale with `codewhale --enable extension_host` (the flag goes\n before any subcommand).\n2. Install the example's directory using `/plugin install <local-directory>`.\n3. Run `/plugin validate hello-extension`, then `/plugin show hello-extension`.\n4. Run `/plugin enable hello-extension` to open its content/capability review.\n Read the review and personally run its exact `/plugin trust <token>` command,\n then enable the plugin. Trust alone does not execute the code.\n5. Ask Codewhale to use `hello_greet`. The tool follows the normal approval gate.\n6. Run `/hello-greet Codewhale`. The command runs when you type it, and shows\n its answer in the transcript with its origin, `extension:hello-extension`.\n\nAn authoring agent should stop after installation and validation and present\nthe review to the person, unless it runs in Full Access and the plugin is one\nit wrote and installed from a folder in its project: then it may trust and\nenable it for that project with `codewhale plugins trust` and `enable`. Those\nshell commands go through the session's approval like any other: Ask asks the\nperson, Auto-Review's reviewer judges, and Full Access runs them.\n\n## Locations, installation and trust\n\nUser bundles live under `~/.codewhale/plugins/`; workspace bundles live under\n`<workspace>/.codewhale/plugins/`. New bundles use Agent Plugins v1.0.0\n`plugin.json`. Declare `extensions[\"net.codewhale\"].native.path` as one regular\n`.mjs`, `.js` or `.mts` file inside the bundle. Directories and other extensions\nfail validation when the host is enabled. With the flag off, native entries\nremain inventory-only.\n\nA plugin may split its code across several entries with `native.paths` (up to\n64, alone or beside `path`). They are activated in the order declared, as the\nfibers of one plugin: they share one owner, so one disable, review change or\ncrash tears all of them down together. Each entry is a separate module with its\nown `apply`, and tool and command names must be unique across them. If an entry\nfails to activate (throws, requires a service the host does not provide, or has\na registration refused), only that entry is withdrawn: entries that already\nactivated stay registered, later entries still activate, and the plugin fails\nonly when no entry activates. Listing the same file twice is one entry.\n\n## Runtime\n\n`[extension_host] runtime` selects what runs the host. Node is the default.\nBun is an opt-in: it is qualified on macOS only, and becomes the default only\nafter an explicit, recorded cutover.\n\n```toml\n[extension_host]\nruntime = \"node\" # default: Node only\n# runtime = \"bun\" # Bun only; fails rather than falling back to Node\n# runtime = \"auto\" # Bun >= 1.4.0 if one is found and starts, otherwise Node\n# bun = \"~/.bun/bin/bun\" # the only Bun tried when set\n# node = \"/opt/homebrew/bin/node\" # the only Node tried when set\n# mcp_backend = \"rust\" # temporary opt-out; default is \"host\"\n```\n\nThe default `mcp_backend = \"host\"` uses the pinned MCP SDK independently of\n`[features] extension_host`, which controls optional Native extensions. Existing\nstdio and HTTP servers use modern discovery when supported; explicitly configured\nlegacy SSE servers use their legacy handshake. Rust retains process and network\nauthority, credential resolution, catalog admission, tool approval and cancellation.\nThe SDK receives opaque session selectors and exact operation grants. Modern\nrequests carry Rust-approved protocol metadata and headers without a session id.\n\nA typed discovery refusal can admit a legacy handshake. Only an explicit HTTP\ntransport refusal during that handshake can admit legacy SSE with a fresh exact\nRust grant. Legacy HTTP retains its GET preflight and bounded OAuth refresh;\nmodern HTTP does not inherit those session or resend behaviors. Incomplete result\nflows are reported without automatically repeating the operation. Credentials,\nconfigured URLs, OAuth browser/login operations and Computer Use decision keys\nstay in Rust.\n\nBoth backends share recorded semantic MCP goldens and retain separate transport\ngoldens, including discovery, authentication and sessioLine truncated
}
]
},
@@ -365,8 +365,8 @@
"sources": [
{
"path": "docs/RUNTIME_API.md",
"sha256": "6f3b8885978ef2e1b33d0fdf66ad6ccf5c25464c1951b59658aec00745390db2",
"content": "# Codewhale Engine Runtime API & Integration Contract\n\n> 阅读简体中文版:[zh_hans/RUNTIME_API.md](zh_hans/RUNTIME_API.md)。\n\n`codewhale app-server --http` exposes the [Codewhale Engine](ARCHITECTURE.md) through\nits canonical local Runtime API and control plane.\nLocal SDKs, mobile/remote-control clients, and editor integrations talk to it\ninstead of screen-scraping terminal output. It serves the full HTTP/SSE runtime\nAPI (`/v1/*`), a JSON-RPC control transport over stdio, and the phone-friendly\nmobile page. `codewhale doctor --json` provides machine-readable health, and\n`codewhale serve --acp` speaks the Agent Client Protocol over stdio for editors\nsuch as Zed.\n\n`codewhale serve --http` / `serve --mobile` remain as **compatibility aliases**\nfor `codewhale app-server --http` / `--mobile`; both launch the identical\nserver. Desktop, web, SDK and custom integrations should target\n`app-server --http`. `--stdio` and `--socket` expose compatibility controls.\nBare `app-server` selects initialized JSONL control over stdio. Local stdio,\nsocket and ACP owners do not bind a hidden TCP listener. Explicit `--http`\nretains HTTP/SSE and compatibility routes on the same Runtime owner.\n\n`codewhale exec` is the separate one-shot headless worker path (stream-json,\nfleet worker subprocess, CI primitive). It is not part of this API, but it\nshares the same runtime, provider/model resolution, permission profiles, and\nevent vocabulary.\n\nThis document is the stable integration contract for native workbench\napplications (and other local supervisors) that embed the Codewhale Engine.\n\n## Architecture\n\n```\nlocal supervisor / SDK / automation harness\n │\n ├─ codewhale app-server --http → HTTP/SSE runtime API (/v1/*) [canonical]\n ├─ codewhale app-server --mobile → runtime API + mobile control page\n ├─ codewhale app-server --stdio → JSON-RPC control transport over stdio\n ├─ codewhale app-server --socket → same JSON-RPC over a unix domain socket (desktop daemon)\n ├─ codewhale doctor --json → machine-readable health & capability\n ├─ codewhale serve --acp → ACP stdio agent for editors such as Zed\n ├─ codewhale serve --mcp → MCP stdio server\n ├─ codewhale serve --http/--mobile → legacy aliases for `app-server --http/--mobile`\n └─ codewhale exec [args] → one-shot headless worker (stream-json)\n```\n\nThe engine runs as a local-only process. All APIs bind to `localhost` by\ndefault. No hosted relay, no provider-token custody, no secret leakage.\n\nFor the read-only record of what a thread or turn did, see\n[`docs/RECEIPTS.md`](RECEIPTS.md): `codewhale receipts` on the CLI and the\n`/receipt` routes under **Threads** below.\n\n## Runtime API entrypoints\n\n| Entry | Transport | Use |\n|---|---|---|\n| `codewhale web [--port 7878]` | HTTP/SSE on `127.0.0.1:7878` + embedded client | First-class loopback-only browser client; opens the default browser |\n| `codewhale app-server --http` | HTTP/SSE on `127.0.0.1:7878` | Full `/v1/*` runtime API (canonical) |\n| `codewhale app-server --mobile` | HTTP/SSE on loopback + `/mobile` | Runtime API + local mobile control page |\n| `codewhale app-server --stdio` | Initialized JSONL over stdio | Compatibility controls and explicit tool-free turns through the held manager; no hidden TCP listener or full Codex interaction channel. |\n| `codewhale app-server --socket [--socket-path P]` | RPC text messages over WebSocket on a `0600` Unix domain socket (macOS/Linux) or an owner-only named pipe (Windows) | Desktop daemon: multi-client, kernel peer identity checked, `daemon/attach` claim handshake. Windows native-app qualification remains open. |\n| `codewhale app-server` | Initialized JSONL over stdio | Same selection as `--stdio`; HTTP requires explicit `--http`. |\n| `codewhale serve --http` / `--mobile` | same server as `app-server --http`/`--mobile` | Compatibility aliases |\n\n`app-server --http` and `--mobile` launch the same mature runtime API server\nhistorically reached through `serve --http` — no routes or behavior changed, so\nevery endpoint documented below is identical across both entrypoints. The\nruntime API token is read from `--auth-token`, then `CODEWHALE_RUNTIME_TOKEN`,\nthen `DEEPSEEK_RUNTIME_TOKEN`; use `--insecure-no-auth` only with a loopback\nbind. The `serve` compatibility aliases keep their `--insecure` flag.\nBare `app-server` does not open an HTTP port. Selected attached HTTP listeners\nretain their explicit bearer and non-loopback security checks.\n\nInitialized stdio/private-socket controls support `thread/loaded/list` with\n`{cursor?: string | null, limit?: number | null}`. The result is\n`{data: string[], nextCursor: string | null}` from the held manager's loaded\nengine snapshot, including idle and archived engines still in memory. It dLine truncated
"sha256": "e3040635df0cf62717f5acbe9aacf4909b6ad60db8468b06e90a5e9be1d76d1e",
"content": "# Codewhale Engine Runtime API & Integration Contract\n\n> 阅读简体中文版:[zh_hans/RUNTIME_API.md](zh_hans/RUNTIME_API.md)。\n\n`codewhale app-server --http` exposes the [Codewhale Engine](ARCHITECTURE.md) through\nits canonical local Runtime API and control plane.\nLocal SDKs, mobile/remote-control clients, and editor integrations talk to it\ninstead of screen-scraping terminal output. It serves the full HTTP/SSE runtime\nAPI (`/v1/*`), a JSON-RPC control transport over stdio, and the phone-friendly\nmobile page. `codewhale doctor --json` provides machine-readable health, and\n`codewhale serve --acp` speaks the Agent Client Protocol over stdio for editors\nsuch as Zed.\n\n`codewhale serve --http` / `serve --mobile` remain as **compatibility aliases**\nfor `codewhale app-server --http` / `--mobile`; both launch the identical\nserver. Desktop, web, SDK and custom integrations should target\n`app-server --http`. `--stdio` and `--socket` expose compatibility controls.\nBare `app-server` selects initialized JSONL control over stdio. Local stdio,\nsocket and ACP owners do not bind a hidden TCP listener. Explicit `--http`\nretains HTTP/SSE and compatibility routes on the same Runtime owner.\n\n`codewhale exec` is the separate one-shot headless worker path (stream-json,\nfleet worker subprocess, CI primitive). It is not part of this API, but it\nshares the same runtime, provider/model resolution, permission profiles, and\nevent vocabulary.\n\nThis document is the stable integration contract for native workbench\napplications (and other local supervisors) that embed the Codewhale Engine.\n\n## Architecture\n\n```\nlocal supervisor / SDK / automation harness\n │\n ├─ codewhale app-server --http → HTTP/SSE runtime API (/v1/*) [canonical]\n ├─ codewhale app-server --mobile → runtime API + mobile control page\n ├─ codewhale app-server --stdio → JSON-RPC control transport over stdio\n ├─ codewhale app-server --socket → same JSON-RPC over a unix domain socket (desktop daemon)\n ├─ codewhale doctor --json → machine-readable health & capability\n ├─ codewhale serve --acp → ACP stdio agent for editors such as Zed\n ├─ codewhale serve --mcp → MCP stdio server\n ├─ codewhale serve --http/--mobile → legacy aliases for `app-server --http/--mobile`\n └─ codewhale exec [args] → one-shot headless worker (stream-json)\n```\n\nThe engine runs as a local-only process. All APIs bind to `localhost` by\ndefault. No hosted relay, no provider-token custody, no secret leakage.\n\nFor the read-only record of what a thread or turn did, see\n[`docs/RECEIPTS.md`](RECEIPTS.md): `codewhale receipts` on the CLI and the\n`/receipt` routes under **Threads** below.\n\n## Runtime API entrypoints\n\n| Entry | Transport | Use |\n|---|---|---|\n| `codewhale web [--port 7878]` | HTTP/SSE on `127.0.0.1:7878` + embedded client | First-class loopback-only browser client; opens the default browser |\n| `codewhale app-server --http` | HTTP/SSE on `127.0.0.1:7878` | Full `/v1/*` runtime API (canonical) |\n| `codewhale app-server --mobile` | HTTP/SSE on loopback + `/mobile` | Runtime API + local mobile control page |\n| `codewhale app-server --stdio` | Initialized JSONL over stdio | Compatibility controls and explicit tool-free turns through the held manager; no hidden TCP listener or full Codex interaction channel. |\n| `codewhale app-server --socket [--socket-path P]` | RPC text messages over WebSocket on a `0600` Unix domain socket (macOS/Linux) or an owner-only named pipe (Windows) | Desktop daemon: multi-client, kernel peer identity checked, `daemon/attach` claim handshake. Windows native-app qualification remains open. |\n| `codewhale app-server` | Initialized JSONL over stdio | Same selection as `--stdio`; HTTP requires explicit `--http`. |\n| `codewhale serve --http` / `--mobile` | same server as `app-server --http`/`--mobile` | Compatibility aliases |\n\n`app-server --http` and `--mobile` launch the same mature runtime API server\nhistorically reached through `serve --http` — no routes or behavior changed, so\nevery endpoint documented below is identical across both entrypoints. The\nruntime API token is read from `--auth-token`, then `CODEWHALE_RUNTIME_TOKEN`,\nthen `DEEPSEEK_RUNTIME_TOKEN`; use `--insecure-no-auth` only with a loopback\nbind. The `serve` compatibility aliases keep their `--insecure` flag.\nBare `app-server` does not open an HTTP port. Selected attached HTTP listeners\nretain their explicit bearer and non-loopback security checks.\n\nInitialized stdio/private-socket controls support `thread/loaded/list` with\n`{cursor?: string | null, limit?: number | null}`. The result is\n`{data: string[], nextCursor: string | null}` from the held manager's loaded\nengine snapshot, including idle and archived engines still in memory. It dLine truncated
}
]
},
@@ -434,13 +434,13 @@
"sources": [
{
"path": "docs/RUNTIME_API.md",
"sha256": "6f3b8885978ef2e1b33d0fdf66ad6ccf5c25464c1951b59658aec00745390db2",
"content": "# Codewhale Engine Runtime API & Integration Contract\n\n> 阅读简体中文版:[zh_hans/RUNTIME_API.md](zh_hans/RUNTIME_API.md)。\n\n`codewhale app-server --http` exposes the [Codewhale Engine](ARCHITECTURE.md) through\nits canonical local Runtime API and control plane.\nLocal SDKs, mobile/remote-control clients, and editor integrations talk to it\ninstead of screen-scraping terminal output. It serves the full HTTP/SSE runtime\nAPI (`/v1/*`), a JSON-RPC control transport over stdio, and the phone-friendly\nmobile page. `codewhale doctor --json` provides machine-readable health, and\n`codewhale serve --acp` speaks the Agent Client Protocol over stdio for editors\nsuch as Zed.\n\n`codewhale serve --http` / `serve --mobile` remain as **compatibility aliases**\nfor `codewhale app-server --http` / `--mobile`; both launch the identical\nserver. Desktop, web, SDK and custom integrations should target\n`app-server --http`. `--stdio` and `--socket` expose compatibility controls.\nBare `app-server` selects initialized JSONL control over stdio. Local stdio,\nsocket and ACP owners do not bind a hidden TCP listener. Explicit `--http`\nretains HTTP/SSE and compatibility routes on the same Runtime owner.\n\n`codewhale exec` is the separate one-shot headless worker path (stream-json,\nfleet worker subprocess, CI primitive). It is not part of this API, but it\nshares the same runtime, provider/model resolution, permission profiles, and\nevent vocabulary.\n\nThis document is the stable integration contract for native workbench\napplications (and other local supervisors) that embed the Codewhale Engine.\n\n## Architecture\n\n```\nlocal supervisor / SDK / automation harness\n │\n ├─ codewhale app-server --http → HTTP/SSE runtime API (/v1/*) [canonical]\n ├─ codewhale app-server --mobile → runtime API + mobile control page\n ├─ codewhale app-server --stdio → JSON-RPC control transport over stdio\n ├─ codewhale app-server --socket → same JSON-RPC over a unix domain socket (desktop daemon)\n ├─ codewhale doctor --json → machine-readable health & capability\n ├─ codewhale serve --acp → ACP stdio agent for editors such as Zed\n ├─ codewhale serve --mcp → MCP stdio server\n ├─ codewhale serve --http/--mobile → legacy aliases for `app-server --http/--mobile`\n └─ codewhale exec [args] → one-shot headless worker (stream-json)\n```\n\nThe engine runs as a local-only process. All APIs bind to `localhost` by\ndefault. No hosted relay, no provider-token custody, no secret leakage.\n\nFor the read-only record of what a thread or turn did, see\n[`docs/RECEIPTS.md`](RECEIPTS.md): `codewhale receipts` on the CLI and the\n`/receipt` routes under **Threads** below.\n\n## Runtime API entrypoints\n\n| Entry | Transport | Use |\n|---|---|---|\n| `codewhale web [--port 7878]` | HTTP/SSE on `127.0.0.1:7878` + embedded client | First-class loopback-only browser client; opens the default browser |\n| `codewhale app-server --http` | HTTP/SSE on `127.0.0.1:7878` | Full `/v1/*` runtime API (canonical) |\n| `codewhale app-server --mobile` | HTTP/SSE on loopback + `/mobile` | Runtime API + local mobile control page |\n| `codewhale app-server --stdio` | Initialized JSONL over stdio | Compatibility controls and explicit tool-free turns through the held manager; no hidden TCP listener or full Codex interaction channel. |\n| `codewhale app-server --socket [--socket-path P]` | RPC text messages over WebSocket on a `0600` Unix domain socket (macOS/Linux) or an owner-only named pipe (Windows) | Desktop daemon: multi-client, kernel peer identity checked, `daemon/attach` claim handshake. Windows native-app qualification remains open. |\n| `codewhale app-server` | Initialized JSONL over stdio | Same selection as `--stdio`; HTTP requires explicit `--http`. |\n| `codewhale serve --http` / `--mobile` | same server as `app-server --http`/`--mobile` | Compatibility aliases |\n\n`app-server --http` and `--mobile` launch the same mature runtime API server\nhistorically reached through `serve --http` — no routes or behavior changed, so\nevery endpoint documented below is identical across both entrypoints. The\nruntime API token is read from `--auth-token`, then `CODEWHALE_RUNTIME_TOKEN`,\nthen `DEEPSEEK_RUNTIME_TOKEN`; use `--insecure-no-auth` only with a loopback\nbind. The `serve` compatibility aliases keep their `--insecure` flag.\nBare `app-server` does not open an HTTP port. Selected attached HTTP listeners\nretain their explicit bearer and non-loopback security checks.\n\nInitialized stdio/private-socket controls support `thread/loaded/list` with\n`{cursor?: string | null, limit?: number | null}`. The result is\n`{data: string[], nextCursor: string | null}` from the held manager's loaded\nengine snapshot, including idle and archived engines still in memory. It dLine truncated
"sha256": "e3040635df0cf62717f5acbe9aacf4909b6ad60db8468b06e90a5e9be1d76d1e",
"content": "# Codewhale Engine Runtime API & Integration Contract\n\n> 阅读简体中文版:[zh_hans/RUNTIME_API.md](zh_hans/RUNTIME_API.md)。\n\n`codewhale app-server --http` exposes the [Codewhale Engine](ARCHITECTURE.md) through\nits canonical local Runtime API and control plane.\nLocal SDKs, mobile/remote-control clients, and editor integrations talk to it\ninstead of screen-scraping terminal output. It serves the full HTTP/SSE runtime\nAPI (`/v1/*`), a JSON-RPC control transport over stdio, and the phone-friendly\nmobile page. `codewhale doctor --json` provides machine-readable health, and\n`codewhale serve --acp` speaks the Agent Client Protocol over stdio for editors\nsuch as Zed.\n\n`codewhale serve --http` / `serve --mobile` remain as **compatibility aliases**\nfor `codewhale app-server --http` / `--mobile`; both launch the identical\nserver. Desktop, web, SDK and custom integrations should target\n`app-server --http`. `--stdio` and `--socket` expose compatibility controls.\nBare `app-server` selects initialized JSONL control over stdio. Local stdio,\nsocket and ACP owners do not bind a hidden TCP listener. Explicit `--http`\nretains HTTP/SSE and compatibility routes on the same Runtime owner.\n\n`codewhale exec` is the separate one-shot headless worker path (stream-json,\nfleet worker subprocess, CI primitive). It is not part of this API, but it\nshares the same runtime, provider/model resolution, permission profiles, and\nevent vocabulary.\n\nThis document is the stable integration contract for native workbench\napplications (and other local supervisors) that embed the Codewhale Engine.\n\n## Architecture\n\n```\nlocal supervisor / SDK / automation harness\n │\n ├─ codewhale app-server --http → HTTP/SSE runtime API (/v1/*) [canonical]\n ├─ codewhale app-server --mobile → runtime API + mobile control page\n ├─ codewhale app-server --stdio → JSON-RPC control transport over stdio\n ├─ codewhale app-server --socket → same JSON-RPC over a unix domain socket (desktop daemon)\n ├─ codewhale doctor --json → machine-readable health & capability\n ├─ codewhale serve --acp → ACP stdio agent for editors such as Zed\n ├─ codewhale serve --mcp → MCP stdio server\n ├─ codewhale serve --http/--mobile → legacy aliases for `app-server --http/--mobile`\n └─ codewhale exec [args] → one-shot headless worker (stream-json)\n```\n\nThe engine runs as a local-only process. All APIs bind to `localhost` by\ndefault. No hosted relay, no provider-token custody, no secret leakage.\n\nFor the read-only record of what a thread or turn did, see\n[`docs/RECEIPTS.md`](RECEIPTS.md): `codewhale receipts` on the CLI and the\n`/receipt` routes under **Threads** below.\n\n## Runtime API entrypoints\n\n| Entry | Transport | Use |\n|---|---|---|\n| `codewhale web [--port 7878]` | HTTP/SSE on `127.0.0.1:7878` + embedded client | First-class loopback-only browser client; opens the default browser |\n| `codewhale app-server --http` | HTTP/SSE on `127.0.0.1:7878` | Full `/v1/*` runtime API (canonical) |\n| `codewhale app-server --mobile` | HTTP/SSE on loopback + `/mobile` | Runtime API + local mobile control page |\n| `codewhale app-server --stdio` | Initialized JSONL over stdio | Compatibility controls and explicit tool-free turns through the held manager; no hidden TCP listener or full Codex interaction channel. |\n| `codewhale app-server --socket [--socket-path P]` | RPC text messages over WebSocket on a `0600` Unix domain socket (macOS/Linux) or an owner-only named pipe (Windows) | Desktop daemon: multi-client, kernel peer identity checked, `daemon/attach` claim handshake. Windows native-app qualification remains open. |\n| `codewhale app-server` | Initialized JSONL over stdio | Same selection as `--stdio`; HTTP requires explicit `--http`. |\n| `codewhale serve --http` / `--mobile` | same server as `app-server --http`/`--mobile` | Compatibility aliases |\n\n`app-server --http` and `--mobile` launch the same mature runtime API server\nhistorically reached through `serve --http` — no routes or behavior changed, so\nevery endpoint documented below is identical across both entrypoints. The\nruntime API token is read from `--auth-token`, then `CODEWHALE_RUNTIME_TOKEN`,\nthen `DEEPSEEK_RUNTIME_TOKEN`; use `--insecure-no-auth` only with a loopback\nbind. The `serve` compatibility aliases keep their `--insecure` flag.\nBare `app-server` does not open an HTTP port. Selected attached HTTP listeners\nretain their explicit bearer and non-loopback security checks.\n\nInitialized stdio/private-socket controls support `thread/loaded/list` with\n`{cursor?: string | null, limit?: number | null}`. The result is\n`{data: string[], nextCursor: string | null}` from the held manager's loaded\nengine snapshot, including idle and archived engines still in memory. It dLine truncated
},
{
"path": "docs/RUNTIME_ROUTES.md",
"sha256": "1a21c05b0249c2ff225f7b554e1e610fc12122ab3da52e6ac7ceb06c90448984",
"content": "# Runtime HTTP route inventory\n\nThis table is generated from literal route registrations in\n`crates/app-server/src/runtime_api.rs`. It names the paths and methods that\nthe current source registers, including native-client and compatibility\nsurfaces. A registered path is not a grant of permission or a promise that\na feature is enabled in your installed version.\n\nThe [Runtime API contract](RUNTIME_API.md) explains request fields, events,\nauthorization and compatibility. This inventory is a navigation index, not\na replacement for those schemas; routes used by the native client can have\nclient-specific contracts. Provider calls, file mutations and connected-service\nactions remain subject to their documented authority boundaries.\n\nRun `node web/scripts/check-runtime-routes.mjs --write` after changing a\nregistration; `npm run check:docs` refuses a stale inventory. Dynamic nested\nrouters, fallback handlers and static-asset mounts are outside this literal\nregistration index.\n\n| HTTP method | Path |\n| --- | --- |\n| GET | `/` |\n| GET | `/__codewhale/bootstrap/{nonce}` |\n| GET | `/__codewhale/mobile/bootstrap/{nonce}` |\n| POST | `/__codewhale/mobile/session` |\n| POST | `/__codewhale/mobile/stream-ticket` |\n| POST | `/__codewhale/web/stream-ticket` |\n| GET | `/assets/codewhale-192.png` |\n| GET | `/assets/codewhale-web.css` |\n| GET | `/assets/codewhale-web.js` |\n| GET | `/health` |\n| GET | `/mobile` |\n| GET | `/mobile/` |\n| DELETE / GET / PUT | `/v1/account/model-access` |\n| POST | `/v1/agent-mail` |\n| GET | `/v1/agent-runs` |\n| GET | `/v1/agent-runs/{run_id}` |\n| POST | `/v1/agent-runs/{run_id}/cancel` |\n| GET | `/v1/agents/external` |\n| GET | `/v1/agents/external/{id}/options` |\n| GET | `/v1/agents/external/quota` |\n| GET | `/v1/approvals` |\n| POST | `/v1/approvals/{approval_id}` |\n| GET | `/v1/apps/avatars` |\n| GET | `/v1/apps/avatars/{handle}/atlases/{page}` |\n| GET / POST | `/v1/apps/marketplaces` |\n| DELETE / GET | `/v1/apps/marketplaces/{name}` |\n| POST | `/v1/apps/marketplaces/{name}/install` |\n| POST | `/v1/apps/mcp/desktop-browser` |\n| GET / POST | `/v1/apps/mcp/imports` |\n| GET / POST | `/v1/apps/mcp/servers` |\n| DELETE / GET / PATCH | `/v1/apps/mcp/servers/{name}` |\n| POST | `/v1/apps/mcp/servers/{name}/disable` |\n| POST | `/v1/apps/mcp/servers/{name}/enable` |\n| POST | `/v1/apps/mcp/servers/{name}/reconnect` |\n| GET | `/v1/apps/mcp/tools` |\n| GET | `/v1/apps/plugins` |\n| DELETE / GET | `/v1/apps/plugins/{selector}` |\n| POST | `/v1/apps/plugins/{selector}/disable` |\n| POST | `/v1/apps/plugins/{selector}/enable` |\n| POST | `/v1/apps/plugins/{selector}/restage` |\n| POST | `/v1/apps/plugins/{selector}/revoke` |\n| ANY | `/v1/apps/plugins/{selector}/routes/{*path}` |\n| POST | `/v1/apps/plugins/{selector}/trust` |\n| GET | `/v1/apps/plugins/{selector}/ui` |\n| GET | `/v1/apps/plugins/{selector}/ui/{*path}` |\n| POST | `/v1/apps/plugins/{selector}/update` |\n| GET | `/v1/apps/plugins/catalog` |\n| POST | `/v1/apps/plugins/catalog/{id}/disable` |\n| POST | `/v1/apps/plugins/catalog/{id}/enable` |\n| PUT | `/v1/apps/plugins/catalog/{id}/settings` |\n| POST | `/v1/apps/plugins/import/dsh/preview` |\n| POST | `/v1/apps/plugins/install` |\n| GET / POST | `/v1/automations` |\n| DELETE / GET / PATCH | `/v1/automations/{id}` |\n| GET | `/v1/automations/{id}/missed` |\n| PUT | `/v1/automations/{id}/missed/policy` |\n| POST | `/v1/automations/{id}/missed/run` |\n| POST | `/v1/automations/{id}/missed/skip` |\n| POST | `/v1/automations/{id}/pause` |\n| POST | `/v1/automations/{id}/resume` |\n| POST | `/v1/automations/{id}/run` |\n| GET | `/v1/automations/{id}/runs` |\n| GET | `/v1/automations/missed` |\n| GET | `/v1/changes` |\n| GET / POST | `/v1/cloud` |\n| POST | `/v1/cloud/attach` |\n| GET | `/v1/commands` |\n| GET / POST | `/v1/config` |\n| POST | `/v1/config/reload` |\n| GET | `/v1/constitution` |\n| POST | `/v1/constitution/preview` |\n| GET | `/v1/crashes` |\n| GET | `/v1/crashes/{name}` |\n| GET | `/v1/definition` |\n| GET | `/v1/diagnostics` |\n| GET | `/v1/diff` |\n| POST | `/v1/fleet/placement` |\n| GET | `/v1/fleet/profiles` |\n| GET | `/v1/fleet/profiles/{id}/document` |\n| GET / POST | `/v1/fleet/runs` |\n| GET | `/v1/fleet/runs/{run_id}` |\n| GET | `/v1/fleet/runs/{run_id}/events` |\n| GET | `/v1/fleet/runs/{run_id}/events/replay` |\n| GET | `/v1/fleet/runs/{run_id}/receipts` |\n| GET | `/v1/fleet/runs/{run_id}/receipts/{task_id}` |\n| GET | `/v1/fleet/runs/{run_id}/receipts/{task_id}/evidence` |\n| POST | `/v1/fleet/runs/{run_id}/start` |\n| POST | `/v1/fleet/runs/{run_id}/stop` |\n| GET | `/v1/fleet/runs/{run_id}/workers` |\n| GET | `/v1/fleet/workers/{worker_id}` |\n| POST | `/v1/fleet/workers/{worker_id}/interrupt` |\n| POST | `/v1/fleet/workers/{worker_id}/restart` |\n| POST | `/v1/fleet/workers/{worker_id}/stop` |\n| GET | `/v1/git` |\n| POST | `/v1/git/branch` |\n| POST | `/v1/git/commit` |\n| POST | `/v1/git/discard` |\nLine truncated
"sha256": "eae9cc4778f7c6f4ae2db5fe0d2d39d757521337a0818b2c2bf4a4f5dd686e0e",
"content": "# Runtime HTTP route inventory\n\nThis table is generated from literal route registrations in\n`crates/app-server/src/runtime_api.rs`. It names the paths and methods that\nthe current source registers, including native-client and compatibility\nsurfaces. A registered path is not a grant of permission or a promise that\na feature is enabled in your installed version.\n\nThe [Runtime API contract](RUNTIME_API.md) explains request fields, events,\nauthorization and compatibility. This inventory is a navigation index, not\na replacement for those schemas; routes used by the native client can have\nclient-specific contracts. Provider calls, file mutations and connected-service\nactions remain subject to their documented authority boundaries.\n\nRun `node web/scripts/check-runtime-routes.mjs --write` after changing a\nregistration; `npm run check:docs` refuses a stale inventory. Dynamic nested\nrouters, fallback handlers and static-asset mounts are outside this literal\nregistration index.\n\n| HTTP method | Path |\n| --- | --- |\n| GET | `/` |\n| GET | `/__codewhale/bootstrap/{nonce}` |\n| GET | `/__codewhale/mobile/bootstrap/{nonce}` |\n| POST | `/__codewhale/mobile/session` |\n| POST | `/__codewhale/mobile/stream-ticket` |\n| POST | `/__codewhale/web/stream-ticket` |\n| GET | `/assets/codewhale-192.png` |\n| GET | `/assets/codewhale-web.css` |\n| GET | `/assets/codewhale-web.js` |\n| GET | `/health` |\n| GET | `/mobile` |\n| GET | `/mobile/` |\n| DELETE / GET / PUT | `/v1/account/model-access` |\n| POST | `/v1/agent-mail` |\n| GET | `/v1/agent-runs` |\n| GET | `/v1/agent-runs/{run_id}` |\n| POST | `/v1/agent-runs/{run_id}/cancel` |\n| GET | `/v1/agents/external` |\n| GET | `/v1/agents/external/{id}/options` |\n| GET | `/v1/agents/external/quota` |\n| GET | `/v1/approvals` |\n| POST | `/v1/approvals/{approval_id}` |\n| GET | `/v1/apps/avatars` |\n| GET | `/v1/apps/avatars/{handle}/atlases/{page}` |\n| GET / POST | `/v1/apps/marketplaces` |\n| DELETE / GET | `/v1/apps/marketplaces/{name}` |\n| POST | `/v1/apps/marketplaces/{name}/install` |\n| POST | `/v1/apps/mcp/desktop-browser` |\n| GET / POST | `/v1/apps/mcp/imports` |\n| GET / POST | `/v1/apps/mcp/servers` |\n| DELETE / GET / PATCH | `/v1/apps/mcp/servers/{name}` |\n| POST | `/v1/apps/mcp/servers/{name}/disable` |\n| POST | `/v1/apps/mcp/servers/{name}/enable` |\n| POST | `/v1/apps/mcp/servers/{name}/reconnect` |\n| GET | `/v1/apps/mcp/tools` |\n| GET | `/v1/apps/plugin-index` |\n| GET | `/v1/apps/plugins` |\n| DELETE / GET | `/v1/apps/plugins/{selector}` |\n| POST | `/v1/apps/plugins/{selector}/disable` |\n| POST | `/v1/apps/plugins/{selector}/enable` |\n| POST | `/v1/apps/plugins/{selector}/restage` |\n| POST | `/v1/apps/plugins/{selector}/revoke` |\n| ANY | `/v1/apps/plugins/{selector}/routes/{*path}` |\n| POST | `/v1/apps/plugins/{selector}/trust` |\n| GET | `/v1/apps/plugins/{selector}/ui` |\n| GET | `/v1/apps/plugins/{selector}/ui/{*path}` |\n| POST | `/v1/apps/plugins/{selector}/update` |\n| GET | `/v1/apps/plugins/catalog` |\n| POST | `/v1/apps/plugins/catalog/{id}/disable` |\n| POST | `/v1/apps/plugins/catalog/{id}/enable` |\n| PUT | `/v1/apps/plugins/catalog/{id}/settings` |\n| POST | `/v1/apps/plugins/import/dsh/preview` |\n| POST | `/v1/apps/plugins/install` |\n| GET / POST | `/v1/automations` |\n| DELETE / GET / PATCH | `/v1/automations/{id}` |\n| GET | `/v1/automations/{id}/missed` |\n| PUT | `/v1/automations/{id}/missed/policy` |\n| POST | `/v1/automations/{id}/missed/run` |\n| POST | `/v1/automations/{id}/missed/skip` |\n| POST | `/v1/automations/{id}/pause` |\n| POST | `/v1/automations/{id}/resume` |\n| POST | `/v1/automations/{id}/run` |\n| GET | `/v1/automations/{id}/runs` |\n| GET | `/v1/automations/missed` |\n| GET | `/v1/changes` |\n| GET / POST | `/v1/cloud` |\n| POST | `/v1/cloud/attach` |\n| GET | `/v1/commands` |\n| GET / POST | `/v1/config` |\n| POST | `/v1/config/reload` |\n| GET | `/v1/constitution` |\n| POST | `/v1/constitution/preview` |\n| GET | `/v1/crashes` |\n| GET | `/v1/crashes/{name}` |\n| GET | `/v1/definition` |\n| GET | `/v1/diagnostics` |\n| GET | `/v1/diff` |\n| POST | `/v1/fleet/placement` |\n| GET | `/v1/fleet/profiles` |\n| GET | `/v1/fleet/profiles/{id}/document` |\n| GET / POST | `/v1/fleet/runs` |\n| GET | `/v1/fleet/runs/{run_id}` |\n| GET | `/v1/fleet/runs/{run_id}/events` |\n| GET | `/v1/fleet/runs/{run_id}/events/replay` |\n| GET | `/v1/fleet/runs/{run_id}/receipts` |\n| GET | `/v1/fleet/runs/{run_id}/receipts/{task_id}` |\n| GET | `/v1/fleet/runs/{run_id}/receipts/{task_id}/evidence` |\n| POST | `/v1/fleet/runs/{run_id}/start` |\n| POST | `/v1/fleet/runs/{run_id}/stop` |\n| GET | `/v1/fleet/runs/{run_id}/workers` |\n| GET | `/v1/fleet/workers/{worker_id}` |\n| POST | `/v1/fleet/workers/{worker_id}/interrupt` |\n| POST | `/v1/fleet/workers/{worker_id}/restart` |\n| POST | `/v1/fleet/workers/{worker_id}/stop` |\n| GET | `/v1/git` |\n| POST | `/v1/git/branch` |\n| POST | `/v1/git/commitLine truncated
}
]
},
+1 -2
View File
@@ -35,7 +35,6 @@ export const RELEASE_CONTRIBUTORS: string[] = [
"@luochen211",
"@gannp",
"@Felixkw12",
"@MannXo",
];
/**
@@ -43,7 +42,7 @@ export const RELEASE_CONTRIBUTORS: string[] = [
* one. scripts/check-contributor-credit.py requires them here now; the release
* cut moves them into RELEASE_CONTRIBUTORS with that release's changelog block.
*/
export const UNRELEASED_CONTRIBUTORS: string[] = ["@luochen211", "@Lstarsky0", "@SamhandsomeLee", "@gaord"];
export const UNRELEASED_CONTRIBUTORS: string[] = ["@luochen211", "@Lstarsky0", "@SamhandsomeLee", "@gaord", "@MannXo"];
/**
* Contributors who helped with reports, reproductions, and verification.