Compare commits

..
Author SHA1 Message Date
CodeWhale BotandClaude Sonnet 5.5 14cd110703 feat(telemetry): per-process spool and a once-only recovery send at the next start (CW-8190)
Replaces the shared buffer.jsonl, which every arming process truncated and
every flush drained under its own envelope, with telemetry/spool/<nonce>.jsonl:
a context header line, then events, owned by one process through an exclusive
<nonce>.lock held for its life. A clean flush sends only the process's own
spool and deletes it before the request.

A later TUI, exec or serve start recovers at most 4 dead spools (lock free),
newest first, on a separate thread: consent and tombstone re-checked, file
deleted before the POST, header validated (release version, 12-hex sha, schema
and notice equal to current), every event re-checked with is_bounded; older
than 7 days or surplus spools are deleted unsent. Opt-out removes spool/; arming
after a tombstone deletes spools and truncates the legacy buffer. The ring keeps
the header. Wire shape unchanged (schema 4). Drain removed; buffer.jsonl is
now only truncated.

Evidence: see the A5 review packet; hosted CI on this SHA.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 20:14:26 -08:00
CodeWhale BotandClaude Sonnet 5.5 ef02f41a8c chore(web): regenerate agent docs for the telemetry send-on-signal wording (CW-8190)
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 18:15:03 -08:00
CodeWhale BotandClaude Sonnet 5.5 b0361216c2 feat(telemetry): bounded send on a terminating signal and on a broken-pipe exit (CW-8190)
A session ended by a signal wrote its usage rows and session_end to the local
buffer only, and the next process to arm erased them, so killed serve and TUI
sessions never reported. The first terminating signal now records the session
end to disk at once, then makes one send attempt bounded at 1.5 s, run
alongside the runtime drain and the persistence flush inside the existing
select so a second signal still ends the process immediately. The send is the
ordinary flush: consent, the opt-out tombstone and the run kill switch are
re-checked first, and short CLI commands stay disk-only.

exec records the session end and makes the same bounded attempt before
restoring SIGPIPE's default disposition and re-raising it, the point the
earlier plan missed (the re-raise ends the process before exit(141)).

session_end construction moves from the TUI crate into the telemetry crate
(start time held in Armed, cold start reported through a setter by the event
loop). One session ends once: the clean path, the signal path and the broken
pipe share a claim, so a signal during shutdown cannot duplicate rows.

Wire shape is unchanged (schema 4). Docs describe the bounded send.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-11-12 17:43:44 -08:00
30 changed files with 1058 additions and 2232 deletions

No files matched your search

+3 -3
View File
@@ -48,7 +48,7 @@
tags: cnb:arch:amd64
cpus: 16
docker:
image: rust:1.100.0-bookworm
image: rust:1.88-bookworm
# codewhale-cnb-bridge GitHub App credentials from the CNB KeyStore
# (codewhale.net/codewhale-ci-secrets, github-bridge.yml), injected as
# environment variables. Values are never printed.
@@ -112,7 +112,7 @@
tags: cnb:arch:amd64
cpus: 16
docker:
image: rust:1.100.0-bookworm
image: rust:1.88-bookworm
stages:
- name: install release dependencies
script: |
@@ -163,7 +163,7 @@ main:
$:
tag_push:
- docker:
image: rust:1.100.0-bookworm
image: rust:1.88-bookworm
stages:
- name: build linux x64 release assets (static)
# The static full-LTO link can also outlive CNB's default no-output
+1 -1
View File
@@ -1,4 +1,4 @@
ARG RUST_VERSION=1.100.0
ARG RUST_VERSION=1.88
FROM rust:${RUST_VERSION}-slim-bookworm
# Native DBus headers and pkg-config are required by libdbus-sys during
+1 -1
View File
@@ -4,7 +4,7 @@
"context": "..",
"dockerfile": "Dockerfile",
"args": {
"RUST_VERSION": "1.100.0"
"RUST_VERSION": "1.88"
}
},
"customizations": {
-18
View File
@@ -1,18 +0,0 @@
name: ru08-image-probe
on:
push:
branches: ['ci/ru-08**']
jobs:
image:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
platform: [linux/amd64, linux/arm64]
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Build release Dockerfile
run: docker buildx build --platform ${{ matrix.platform }} -f Dockerfile --progress=plain .
- name: Build devcontainer image
run: docker buildx build -f .devcontainer/Dockerfile --build-arg RUST_VERSION=1.100.0 --progress=plain .
+1
View File
@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Non-ASCII file names:** `/diff`, the `@git` and `@diff` mentions and the Runtime API workspace diff now show paths with Chinese, emoji and other non-ASCII characters as written instead of as quoted octal escapes, and the turn context no longer lists the workspace root as an empty "Active paths" entry. Thanks [@luochen211](https://github.com/luochen211) (PR 7317).
- **Session names after a crash, and keep-forever snapshot retention:** a session name saved right after a crash that cut off the previous session-index line is no longer lost, because the new entry now starts on its own line; and a very large `[snapshots] max_age_days` (such as `9223372036854775807` for "keep forever") no longer wraps around and deletes every undo snapshot at session start. Thanks [@luochen211](https://github.com/luochen211) (PR 7316).
- **Blank environment variables:** exporting a variable empty (for example `CODEWHALE_MODEL=` from a compose `${VAR:-}` template) no longer hides its legacy alias. `DEEPSEEK_MODEL`, `DEEPSEEK_BASE_URL`, `KIMI_BASE_URL`, `DEEPSEEK_FORCE_HTTP1`, `DEEPSEEK_STREAM_OPEN_TIMEOUT_SECS` and the other provider alias variables now apply as they do when the primary is unset. Thanks [@luochen211](https://github.com/luochen211) (PR 7315).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (PR 7293).
- **MCP and language servers that send requests mid-call:** a tool call no longer fails with "neither a result nor an error" when an MCP server sends a `ping` or other request that reuses the call's id while the call is running, and a language server's `workspace/configuration` or similar request no longer ends a pending LSP request. Codewhale now answers `ping`, answers other server requests with "Method not found" (it never advertised them), keeps waiting for the real reply, and a refused answer over Streamable HTTP no longer drops the connection to legacy SSE. Thanks [@luochen211](https://github.com/luochen211) (PR 7319).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (#7293).
- **README and contributor credit:** the repository front page now opens the way the website does: the line "The model can be anyone's. The agent should be yours.", one sentence on what Codewhale is, the install command and one picture, with reference detail moved into the linked guides; the 18 translated READMEs follow the same structure and say they are machine-drafted. The contributor wall and the website now count the same 433 people (407 GitHub accounts and the commit authors whose email is not linked to an account), where the website had counted accounts only. `DEPENDENCY_SECURITY.md` moved into `docs/dependency-maintenance.md`, and `.github/SUPPORT.md` says where to ask for help.
+1 -1
View File
@@ -12,7 +12,7 @@
# Or mount an env file:
# docker run --rm -it --env-file .env codewhale
ARG RUST_VERSION=1.100.0
ARG RUST_VERSION=1.88
# ── Stage 1: Build ────────────────────────────────────────────────────
FROM --platform=$BUILDPLATFORM rust:${RUST_VERSION}-slim-bookworm AS builder
+4
View File
@@ -3115,6 +3115,10 @@ fn write_exec_stdout(text: &str) -> Result<()> {
.and_then(|()| stdout.flush())
{
Err(err) if err.kind() == io::ErrorKind::BrokenPipe => {
let _ = codewhale_telemetry::report_termination(
codewhale_telemetry::ExitClass::Signal,
codewhale_telemetry::SIGNAL_FLUSH_TIMEOUT,
);
// SAFETY: restores the default disposition and re-raises the
// signal the write would have delivered without SIG_IGN.
#[cfg(unix)]
@@ -293,11 +293,17 @@ fn dispatch_and_read_telemetry_with_endpoint(
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => panic!("read telemetry dry-run sink: {error}"),
};
let pending = match fs::read_to_string(telemetry_dir.join("buffer.jsonl")) {
Ok(contents) => Some(contents),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => panic!("read telemetry pending buffer: {error}"),
};
let pending = fs::read_dir(telemetry_dir.join("spool"))
.ok()
.and_then(|entries| {
let bodies: Vec<String> = entries
.filter_map(Result::ok)
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "jsonl"))
.filter_map(|path| fs::read_to_string(path).ok())
.collect();
(!bodies.is_empty()).then(|| bodies.join("\n"))
});
DispatchEvidence {
telemetry_dir_exists: telemetry_dir.exists(),
dry_run,
+131 -14
View File
@@ -23,6 +23,7 @@ use crate::client::{self, SendOutcome};
use crate::decision::{self, TelemetryDecision};
use crate::envelope;
use crate::event::{Batch, Event, SCHEMA_VERSION, Surface};
use crate::spool::{self, SpoolContext};
/// Events per batch.
pub const BATCH_MAX_EVENTS: usize = 200;
@@ -64,6 +65,19 @@ pub(crate) struct Context {
pub app_version: String,
pub git_sha: Option<String>,
pub tty: bool,
pub spool: PathBuf,
pub nonce: String,
}
impl Context {
fn header(&self) -> SpoolContext {
SpoolContext::new(
self.surface,
self.app_version.clone(),
self.git_sha.clone(),
self.tty,
)
}
}
/// A handle to the writer thread.
@@ -209,8 +223,7 @@ fn append(context: &Context, event: &Event) {
let Ok(line) = serde_json::to_string(event) else {
return;
};
let path = buffer::buffer_path(&context.root);
let _ = buffer::append(&context.root, &path, &line);
let _ = buffer::append(&context.root, &context.spool, &line);
}
/// Seal every event queued before this message without reaching the network.
@@ -219,7 +232,7 @@ fn append(context: &Context, event: &Event) {
/// runs. Re-deciding here preserves the same mid-session opt-out boundary as a
/// full flush. An explicitly empty endpoint is the local dry-run sink and can
/// therefore be finalized immediately; a configured endpoint leaves the
/// events in `buffer.jsonl` for the next interactive flush.
/// events in this process's spool for a later start to send.
fn persist_local(context: &Context) -> FlushOutcome {
match decision::re_decide(context.config_path.as_deref(), context.surface) {
TelemetryDecision::Enabled(_) => {}
@@ -233,7 +246,10 @@ fn persist_local(context: &Context) -> FlushOutcome {
if context.endpoint.is_none() {
return flush(context);
}
if buffer::read_lines(&buffer::buffer_path(&context.root)).is_empty() {
if buffer::read_lines(&context.spool)
.iter()
.all(|line| spool::is_header_line(line))
{
FlushOutcome::Empty
} else {
FlushOutcome::Buffered
@@ -257,7 +273,7 @@ fn flush(context: &Context) -> FlushOutcome {
return FlushOutcome::Suppressed;
}
let lines = buffer::drain(&context.root);
let lines = spool::take_own(&context.root, &context.spool);
if lines.is_empty() {
return FlushOutcome::Empty;
}
@@ -277,28 +293,94 @@ fn flush(context: &Context) -> FlushOutcome {
// attempts at most once per interval rather than on every launch.
let _ = envelope::write_state(&context.root, &state);
deliver(
&context.root,
context.endpoint.as_deref(),
&context.header(),
install.install_id,
events,
)
}
fn deliver(
root: &std::path::Path,
endpoint: Option<&str>,
header: &SpoolContext,
install_id: String,
events: Vec<Event>,
) -> FlushOutcome {
let batch = Batch {
schema_version: SCHEMA_VERSION,
notice_version: crate::event::NOTICE_VERSION,
schema_version: header.schema_version,
notice_version: header.notice_version,
sent_at: envelope::now_rfc3339(),
install_id: install.install_id,
app_version: context.app_version.clone(),
git_sha: context.git_sha.clone(),
surface: context.surface,
install_id,
app_version: header.app_version.clone(),
git_sha: header.git_sha.clone(),
surface: header.surface,
os: envelope::current_os(),
arch: envelope::current_arch(),
libc: envelope::current_libc(),
tty: context.tty,
tty: header.tty,
events,
};
match client::send(&context.root, context.endpoint.as_deref(), &batch) {
match client::send(root, endpoint, &batch) {
SendOutcome::DryRun => FlushOutcome::DryRun,
SendOutcome::Accepted => FlushOutcome::Sent,
SendOutcome::Dropped => FlushOutcome::Dropped,
}
}
/// Send what dead processes left behind, once each, on their own thread so a
/// slow request never holds up this session's writer.
///
/// Consent and the tombstone are decided before anything is read, and again by
/// the sender for every batch. Each spool is deleted before its request.
pub(crate) fn spawn_recovery(context: Context) {
if !matches!(
context.surface,
Surface::Tui | Surface::Exec | Surface::Serve
) {
return;
}
let _ = std::thread::Builder::new()
.name("codewhale-telemetry-recover".to_string())
.spawn(move || {
let _ = catch_unwind(AssertUnwindSafe(|| recover(&context)));
});
}
pub(crate) fn recover(context: &Context) -> Vec<FlushOutcome> {
match decision::re_decide(context.config_path.as_deref(), context.surface) {
TelemetryDecision::Enabled(_) => {}
TelemetryDecision::OptedOut | TelemetryDecision::ForcedOff => return Vec::new(),
}
send_dead(context)
}
pub(crate) fn send_dead(context: &Context) -> Vec<FlushOutcome> {
if buffer::tombstone_present(&context.root) {
return Vec::new();
}
let Ok(install) = envelope::read_or_create_install_id(&context.root) else {
return Vec::new();
};
spool::take_dead(&context.root, &context.nonce, std::time::SystemTime::now())
.into_iter()
.filter_map(|recovered| {
let events = parse_events(&recovered.lines);
(!events.is_empty()).then(|| {
deliver(
&context.root,
context.endpoint.as_deref(),
&recovered.context,
install.install_id.clone(),
events,
)
})
})
.collect()
}
/// Parse drained lines into events, capped at [`BATCH_MAX_EVENTS`] and
/// [`BATCH_MAX_BYTES`], skipping anything that does not parse **or does not
/// satisfy its declared string bounds**.
@@ -373,4 +455,39 @@ mod tests {
};
assert_eq!(handle.persist_local(), FlushOutcome::TimedOut);
}
#[test]
fn a_hung_writer_cannot_hold_a_termination_send_past_its_deadline() {
let (tx, rx) = sync_channel::<Message>(buffer::MAX_EVENTS + 1);
let handle = Handle {
tx,
pending: Arc::new(AtomicUsize::new(0)),
closing: AtomicBool::new(false),
};
let started = Instant::now();
let outcome = handle.shutdown(Duration::from_millis(50));
assert_eq!(outcome, FlushOutcome::TimedOut);
assert!(started.elapsed() < Duration::from_secs(5));
assert!(matches!(rx.try_recv(), Ok(Message::Shutdown(_))));
}
#[test]
fn a_second_termination_send_does_not_wait() {
let (tx, _rx) = sync_channel::<Message>(buffer::MAX_EVENTS + 1);
let handle = Handle {
tx,
pending: Arc::new(AtomicUsize::new(0)),
closing: AtomicBool::new(false),
};
assert_eq!(
handle.shutdown(Duration::from_millis(20)),
FlushOutcome::TimedOut
);
let started = Instant::now();
assert_eq!(
handle.shutdown(Duration::from_secs(30)),
FlushOutcome::TimedOut
);
assert!(started.elapsed() < Duration::from_secs(5));
}
}
+47 -33
View File
@@ -5,8 +5,10 @@
//!
//! | file | role |
//! |---|---|
//! | `buffer.jsonl` | one JSON event per line, awaiting a flush |
//! | `buffer.jsonl.lock` | a **sibling** lock file; never the data file |
//! | `spool/<nonce>.jsonl` | one process's header line, then one JSON event per line, awaiting its flush |
//! | `spool/<nonce>.lock` | held exclusively by that process while it lives |
//! | `buffer.jsonl` | the pre-spool sink, only ever truncated now |
//! | `buffer.jsonl.lock` | the **privacy** lock, a sibling file; never a data file |
//! | `dryrun.jsonl` | the sink when the endpoint resolves empty, same ring policy |
//! | `state.json` | last version seen and last flush attempt |
//! | `install_id.json` | the random install id |
@@ -218,6 +220,39 @@ fn append_under_lock(root: &Path, path: &Path, buf: &[u8]) -> Result<bool> {
Ok(true)
}
/// Write a spool header line, while the caller holds the blocking privacy lock.
pub(crate) fn append_header(root: &Path, path: &Path, line: &str) -> Result<bool> {
let mut buf = Vec::with_capacity(line.len() + 1);
buf.extend_from_slice(line.as_bytes());
buf.push(b'\n');
append_under_lock(root, path, &buf)
}
/// Re-attach the spool header that a ring cut dropped, keeping the newest events.
fn keep_spool_header(path: &Path, mut lines: Vec<String>) -> Vec<String> {
use std::io::BufRead as _;
let Ok(file) = File::open(path) else {
return lines;
};
let mut first = String::new();
if std::io::BufReader::new(file.take(MAX_LINE_BYTES as u64 + 2))
.read_line(&mut first)
.is_err()
{
return lines;
}
let first = first.trim_end().to_string();
if first.is_empty() || !crate::spool::is_header_line(&first) || lines.first() == Some(&first) {
return lines;
}
lines.insert(0, first);
let size = |lines: &[String]| lines.iter().map(|l| l.len() as u64 + 1).sum::<u64>();
while lines.len() > 1 && (lines.len() > MAX_EVENTS || size(&lines) > MAX_BYTES) {
lines.remove(1);
}
lines
}
/// Keep the newest [`MAX_EVENTS`] lines and at most [`MAX_BYTES`], under the
/// compaction lock. On lock contention this cycle is skipped: the next append
/// tries again, and the cap is a ceiling on disk footprint, not an invariant
@@ -243,7 +278,7 @@ fn enforce_ring(root: &Path, path: &Path) {
if len < PROBE_BYTES {
return Ok(());
}
let lines = read_ring_tail(path)?;
let lines = keep_spool_header(path, read_ring_tail(path)?);
let mut body = lines.join("\n");
if !body.is_empty() {
body.push('\n');
@@ -371,33 +406,6 @@ pub fn read_lines(path: &Path) -> Vec<String> {
read_ring_tail(path).unwrap_or_default()
}
/// Take every buffered line and truncate the buffer, under the compaction lock.
///
/// Returns an empty vector when the tombstone is present or the lock is held
/// elsewhere. Truncates rather than unlinks — `crates/engine/src/fleet/ledger.rs`
/// documents the rule: replacing the file leaves appenders holding the old
/// inode.
#[must_use]
pub fn drain(root: &Path) -> Vec<String> {
if tombstone_present(root) {
return Vec::new();
}
let path = buffer_path(root);
let drained = try_with_lock(root, || {
// Re-check under the lock: a wipe may have landed between the check
// above and the acquisition.
if tombstone_present(root) {
return Ok(Vec::new());
}
let lines = read_lines(&path);
if !lines.is_empty() {
truncate(&path)?;
}
Ok(lines)
});
drained.ok().flatten().unwrap_or_default()
}
/// Truncate a file to zero length, leaving the inode in place. A missing file
/// is not an error.
pub fn truncate(path: &Path) -> Result<()> {
@@ -420,7 +428,7 @@ pub fn truncate(path: &Path) -> Result<()> {
/// 1. take the blocking lock — this is not an exit path;
/// 2. write the tombstone **first**, and never remove it here;
/// 3. truncate `buffer.jsonl` and `dryrun.jsonl` — do **not** unlink them, and
/// never unlink the lock file;
/// never unlink the privacy lock file — and delete every spool;
/// 4. remove `install_id.json` and `state.json`.
///
/// If any step after the tombstone fails, the error is returned and the caller
@@ -466,6 +474,11 @@ pub fn wipe(root: &Path) -> Result<()> {
);
}
}
if let Err(error) = fs::remove_dir_all(crate::spool::dir(root))
&& error.kind() != std::io::ErrorKind::NotFound
{
failure.get_or_insert(anyhow::Error::new(error).context("failed to remove the spools"));
}
match failure {
Some(error) => Err(error),
None => Ok(()),
@@ -473,8 +486,8 @@ pub fn wipe(root: &Path) -> Result<()> {
})
}
/// Clear the tombstone and drop anything buffered before this process was
/// permitted.
/// Clear the tombstone, deleting every spool when there was one, and truncate
/// the pre-spool buffer.
///
/// Called by `init` on every arming. Both the exact tombstone generation and a
/// fresh durable permission check must still match while the wipe lock is held.
@@ -496,6 +509,7 @@ pub(crate) fn arm(
}
let tombstone = tombstone_path(root);
if tombstone.exists() {
let _ = fs::remove_dir_all(crate::spool::dir(root));
fs::remove_file(&tombstone)
.with_context(|| format!("failed to remove {}", tombstone.display()))?;
}
+136 -6
View File
@@ -21,8 +21,8 @@
//! construction empty until resolution completes. A disabled user's panic
//! therefore writes nothing and creates no directory.
//!
//! Arming also **truncates** any stale buffer before a newly permitted process
//! begins recording.
//! Arming also gives the process its own spool file and, when the machine was
//! opted out until now, deletes every spool left from before.
//!
//! # Failure posture
//!
@@ -41,13 +41,14 @@ pub mod decision;
pub mod envelope;
pub mod event;
pub mod notice;
pub mod spool;
#[cfg(test)]
mod tests;
use std::sync::OnceLock;
use std::sync::atomic::{AtomicU8, Ordering};
use std::time::Duration;
use std::sync::atomic::{AtomicBool, AtomicU8, AtomicU64, Ordering};
use std::time::{Duration, Instant};
pub use actor::{BATCH_MAX_BYTES, BATCH_MAX_EVENTS, FlushOutcome};
pub use counters::{Counter, ErrorCounter, SessionCounters, TurnOutcome, TurnTokens};
@@ -69,11 +70,27 @@ pub use event::{
/// handshake would hold a user's terminal past exit.
pub const SHUTDOWN_FLUSH_TIMEOUT: Duration = Duration::from_secs(3);
/// How long the send attempt on a terminating signal or a broken pipe may hold
/// the process.
///
/// Shorter than [`SHUTDOWN_FLUSH_TIMEOUT`] so that, run alongside the runtime's
/// two second signal drain, it fits inside the two second grace a supervisor
/// gives SIGTERM before it escalates.
pub const SIGNAL_FLUSH_TIMEOUT: Duration = Duration::from_millis(1500);
const COLD_START_UNSET: u64 = u64::MAX;
static COLD_START_MS: AtomicU64 = AtomicU64::new(COLD_START_UNSET);
/// Everything a write path needs once the process is armed.
struct Armed {
handle: actor::Handle,
root: std::path::PathBuf,
spool: std::path::PathBuf,
surface: Surface,
started: Instant,
exit_class: AtomicU8,
session_end_recorded: AtomicBool,
}
/// The one gate. Unset means every write path is a hard no-op.
@@ -107,6 +124,14 @@ pub fn init(consent: TelemetryConsent) {
notice::show_startup_disclosure(consent.surface());
let header = spool::SpoolContext::new(
consent.surface(),
env!("CARGO_PKG_VERSION").to_string(),
envelope::release_build_sha(),
envelope::current_tty(),
);
let (spool_path, nonce) = spool::open(&root, &header);
let context = actor::Context {
root: root.clone(),
endpoint: consent.endpoint().map(str::to_string),
@@ -115,12 +140,19 @@ pub fn init(consent: TelemetryConsent) {
app_version: env!("CARGO_PKG_VERSION").to_string(),
git_sha: envelope::release_build_sha(),
tty: envelope::current_tty(),
spool: spool_path.clone(),
nonce,
};
actor::spawn_recovery(context.clone());
let _ = ARMED.set(Armed {
handle: actor::Handle::spawn(context),
root: root.clone(),
spool: spool_path,
surface: consent.surface(),
started: Instant::now(),
exit_class: AtomicU8::new(ExitClass::Clean.as_u8()),
session_end_recorded: AtomicBool::new(false),
});
record_install_or_upgrade(&root);
@@ -241,8 +273,7 @@ pub fn record_blocking(event: Event) {
let Ok(line) = serde_json::to_string(&event) else {
return;
};
let path = buffer::buffer_path(&armed.root);
let _ = buffer::append(&armed.root, &path, &line);
let _ = buffer::append(&armed.root, &armed.spool, &line);
}
/// Record how this process is ending.
@@ -266,6 +297,105 @@ pub fn exit_class() -> ExitClass {
})
}
/// Report how long the interactive event loop took to start. The terminal UI is
/// the only caller; every other surface leaves the cold-start bucket absent.
pub fn set_cold_start_ms(millis: u64) {
let _ = COLD_START_MS.compare_exchange(
COLD_START_UNSET,
millis,
Ordering::Relaxed,
Ordering::Relaxed,
);
}
/// Build `session_end` from what this process actually accumulated.
///
/// The exit class is read from the process-wide atomic and never derived from
/// an exit code. Building the event writes nothing; an unarmed process reports a
/// zero duration and nothing reaches a sink.
#[must_use]
pub fn session_end_event() -> Event {
let counters = session_counters();
let cold_start = COLD_START_MS.load(Ordering::Relaxed);
Event::SessionEnd {
duration_bucket: DurationBucket::from_secs(
ARMED
.get()
.map_or(0, |armed| armed.started.elapsed().as_secs()),
),
exit_class: exit_class(),
cold_start_bucket: (cold_start != COLD_START_UNSET)
.then(|| ColdStartBucket::from_millis(cold_start)),
providers: counters.providers(),
counters: counters.counters(),
errors: counters.errors(),
turn_wall: counters.turn_wall(),
}
}
/// Queue this session's usage rows and `session_end` once.
///
/// Whichever of the clean path, the signal path or a broken pipe gets here
/// first reports; later callers do nothing, so one session never ends twice.
/// A no-op when unarmed.
pub fn record_session_end() {
let Some(armed) = ARMED.get() else {
return;
};
if armed.session_end_recorded.swap(true, Ordering::AcqRel) {
return;
}
for event in session_counters().usage_events() {
record(event);
}
record(session_end_event());
}
/// Close the session synchronously, for a path that cannot wait for the writer.
///
/// States `class`, then appends the usage rows and `session_end` to the local
/// buffer with the same lock-free-for-the-caller write as [`record_blocking`].
/// A no-op when unarmed, and a no-op for the events when the session already
/// ended.
pub fn record_session_end_blocking(class: ExitClass) {
let Some(armed) = ARMED.get() else {
return;
};
set_exit_class(class);
if armed.session_end_recorded.swap(true, Ordering::AcqRel) {
return;
}
for event in session_counters().usage_events() {
record_blocking(event);
}
record_blocking(session_end_event());
}
/// One bounded network attempt for a process that is being terminated.
///
/// Runs the ordinary flush, so consent, the opt-out tombstone and the run kill
/// switch are re-checked immediately beforehand and a person who has opted out
/// sends nothing. Short CLI commands stay disk-only. Never retried, and the
/// caller is released after `deadline` whether or not the request finished.
#[must_use]
pub fn flush_for_termination(deadline: Duration) -> FlushOutcome {
let Some(armed) = ARMED.get() else {
return FlushOutcome::Empty;
};
if armed.surface == Surface::Cli {
return FlushOutcome::Buffered;
}
armed.handle.shutdown(deadline)
}
/// Record the session end and make the one bounded send attempt, for a process
/// that is about to be ended by a signal. Returns [`FlushOutcome::Empty`] when
/// unarmed.
pub fn report_termination(class: ExitClass, deadline: Duration) -> FlushOutcome {
record_session_end_blocking(class);
flush_for_termination(deadline)
}
/// Final flush, then stop the writer thread.
///
/// Returns [`FlushOutcome::Empty`] when unarmed.
+277
View File
@@ -0,0 +1,277 @@
//! Per-process spools and the recovery of the ones a dead process left behind.
//!
//! Each armed process owns `telemetry/spool/<nonce>.jsonl`, whose first line is
//! a context header (the envelope facts that belong to that process), and holds
//! an exclusive lock on `<nonce>.lock` for as long as it lives. Recording and
//! the clean flush touch only the process's own spool, so one process can
//! neither truncate nor send another's events under the wrong surface or
//! version.
//!
//! A spool whose lock can be taken belongs to a process that is gone. A later
//! TUI, exec or serve start sends at most [`MAX_RECOVERED`] of them, each once:
//! the file is deleted **before** the request, so a crash or a failed send never
//! causes a second attempt.
//!
//! The nonce is local only. It names a file and is never sent.
//!
//! Known limitation: a spool is only as trustworthy as the user account, like
//! every other file here. Recovery therefore re-validates the header and every
//! event as untrusted input instead of inheriting any bound.
use std::collections::BTreeSet;
use std::fs::{self, File, OpenOptions};
use std::path::{Path, PathBuf};
use std::time::{Duration, SystemTime};
use serde::{Deserialize, Serialize};
use crate::buffer;
use crate::event::{NOTICE_VERSION, SCHEMA_VERSION, Surface, is_release_version_string};
/// Dead spools sent at one start. Older or surplus ones are deleted unsent.
pub const MAX_RECOVERED: usize = 4;
/// A dead spool older than this is deleted unsent.
pub const MAX_AGE: Duration = Duration::from_secs(7 * 24 * 60 * 60);
const HEADER_VERSION: u32 = 1;
const MAX_VERSION_BYTES: usize = 64;
/// The envelope facts one process stamps on everything it spools.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub(crate) struct SpoolContext {
spool: u32,
pub schema_version: u32,
pub notice_version: u32,
pub app_version: String,
pub git_sha: Option<String>,
pub surface: Surface,
pub tty: bool,
}
impl SpoolContext {
pub(crate) fn new(
surface: Surface,
app_version: String,
git_sha: Option<String>,
tty: bool,
) -> Self {
Self {
spool: HEADER_VERSION,
schema_version: SCHEMA_VERSION,
notice_version: NOTICE_VERSION,
app_version,
git_sha,
surface,
tty,
}
}
/// Whether a header read from disk may be sent as an envelope.
pub(crate) fn is_sendable(&self) -> bool {
self.spool == HEADER_VERSION
&& self.schema_version == SCHEMA_VERSION
&& self.notice_version == NOTICE_VERSION
&& self.app_version.len() <= MAX_VERSION_BYTES
&& is_release_version_string(&self.app_version)
&& self.git_sha.as_deref().is_none_or(|sha| {
sha.len() == 12 && sha.bytes().all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f'))
})
}
}
/// Whether `line` is a spool header rather than an event.
pub(crate) fn is_header_line(line: &str) -> bool {
serde_json::from_str::<SpoolContext>(line).is_ok()
}
/// `$CODEWHALE_HOME/telemetry/spool`.
#[must_use]
pub fn dir(root: &Path) -> PathBuf {
root.join("spool")
}
/// Every spool data file under `root`, for tests and diagnostics.
#[must_use]
pub fn data_files(root: &Path) -> Vec<PathBuf> {
let Ok(entries) = fs::read_dir(dir(root)) else {
return Vec::new();
};
let mut files: Vec<PathBuf> = entries
.filter_map(Result::ok)
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "jsonl"))
.collect();
files.sort();
files
}
fn data_path(root: &Path, nonce: &str) -> PathBuf {
dir(root).join(format!("{nonce}.jsonl"))
}
fn lock_file_path(root: &Path, nonce: &str) -> PathBuf {
dir(root).join(format!("{nonce}.lock"))
}
fn is_nonce(value: &str) -> bool {
value.len() == 32
&& value
.bytes()
.all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f'))
}
fn open_lock_file(path: &Path) -> Option<File> {
OpenOptions::new()
.create(true)
.read(true)
.write(true)
.truncate(false)
.open(path)
.ok()
}
/// Create this process's spool and hold its lock until the process ends.
///
/// Runs at arming, where blocking on the privacy lock is allowed, so the header
/// is on disk before any other writer, including the signal path, can append.
/// Returns the data path and the nonce.
pub(crate) fn open(root: &Path, context: &SpoolContext) -> (PathBuf, String) {
let nonce = uuid::Uuid::new_v4().simple().to_string();
let path = data_path(root, &nonce);
if buffer::ensure_dir(&dir(root)).is_ok() {
if let Some(file) = open_lock_file(&lock_file_path(root, &nonce)) {
let lock = Box::leak(Box::new(fd_lock::RwLock::new(file)));
if let Ok(guard) = lock.try_write() {
std::mem::forget(guard);
}
}
if let Ok(header) = serde_json::to_string(context) {
let _ = buffer::with_lock(root, || buffer::append_header(root, &path, &header));
}
}
(path, nonce)
}
/// Take this process's own spool for delivery: read it and delete it, under the
/// privacy lock.
///
/// Empty when the tombstone is present, the lock is held elsewhere, or the file
/// could not be removed. A spool that cannot be removed is never sent, because
/// a later start would send it again.
#[must_use]
pub(crate) fn take_own(root: &Path, path: &Path) -> Vec<String> {
if buffer::tombstone_present(root) {
return Vec::new();
}
buffer::try_with_lock(root, || {
if buffer::tombstone_present(root) {
return Ok(Vec::new());
}
let lines = buffer::read_lines(path);
if lines.is_empty() {
return Ok(lines);
}
if remove_or_truncate(path) {
Ok(lines)
} else {
Ok(Vec::new())
}
})
.ok()
.flatten()
.unwrap_or_default()
}
fn remove_or_truncate(path: &Path) -> bool {
fs::remove_file(path).is_ok() || buffer::truncate(path).is_ok_and(|()| !path.exists())
}
/// A dead process's spool, already deleted from disk.
#[derive(Debug)]
pub(crate) struct Recovered {
pub context: SpoolContext,
pub lines: Vec<String>,
}
/// Claim the dead spools that may be sent, newest first, deleting every dead
/// spool it looks at.
///
/// Live spools are skipped by their lock, and `own_nonce` is never touched.
/// Unknown file names, symbolic links and anything without a valid header are
/// deleted or ignored, never read as events.
pub(crate) fn take_dead(root: &Path, own_nonce: &str, now: SystemTime) -> Vec<Recovered> {
let Ok(entries) = fs::read_dir(dir(root)) else {
return Vec::new();
};
let mut stems = BTreeSet::new();
for entry in entries.filter_map(Result::ok) {
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
if let Some((stem, ext)) = name.rsplit_once('.')
&& matches!(ext, "jsonl" | "lock")
&& is_nonce(stem)
&& stem != own_nonce
{
stems.insert(stem.to_string());
}
}
let mut ranked: Vec<(SystemTime, String)> = stems
.into_iter()
.map(|stem| {
let modified = [data_path(root, &stem), lock_file_path(root, &stem)]
.iter()
.find_map(|path| fs::symlink_metadata(path).and_then(|m| m.modified()).ok())
.unwrap_or(SystemTime::UNIX_EPOCH);
(modified, stem)
})
.collect();
ranked.sort_by(|a, b| b.cmp(a));
let mut recovered = Vec::new();
for (modified, stem) in ranked {
if buffer::tombstone_present(root) {
break;
}
let data = data_path(root, &stem);
let lock_path = lock_file_path(root, &stem);
let eligible = recovered.len() < MAX_RECOVERED
&& now.duration_since(modified).is_ok_and(|age| age <= MAX_AGE);
let claimed = with_dead_lock(&lock_path, || {
let claimed = if eligible { claim(&data) } else { None };
if claimed.is_none() {
let _ = fs::remove_file(&data);
}
let _ = fs::remove_file(&lock_path);
claimed
});
if let Some(Some(claimed)) = claimed {
recovered.push(claimed);
}
}
recovered
}
fn with_dead_lock<T>(lock_path: &Path, operation: impl FnOnce() -> T) -> Option<T> {
let file = open_lock_file(lock_path)?;
let mut lock = fd_lock::RwLock::new(file);
let _guard = lock.try_write().ok()?;
Some(operation())
}
fn claim(data: &Path) -> Option<Recovered> {
let meta = fs::symlink_metadata(data).ok()?;
if !meta.is_file() {
return None;
}
let lines = buffer::read_lines(data);
let (header, events) = lines.split_first()?;
let context = serde_json::from_str::<SpoolContext>(header).ok()?;
if !context.is_sendable() || events.is_empty() || !remove_or_truncate(data) {
return None;
}
Some(Recovered {
context,
lines: events.to_vec(),
})
}
+330 -5
View File
@@ -1074,7 +1074,7 @@ fn failed_wipe_fails_closed() {
assert!(result.is_err(), "a failed truncate must be reported");
}
assert!(
buffer::drain(&root).is_empty(),
crate::spool::take_own(&root, &buffer_path).is_empty(),
"a tombstoned buffer must never drain, wipe failure or not"
);
assert!(
@@ -1262,14 +1262,14 @@ fn drain_skips_a_torn_trailing_line() {
.expect("open");
file.write_all(b"{\"event\":\"pan").expect("tear");
}
let drained = buffer::drain(&root);
let drained = crate::spool::take_own(&root, &path);
assert_eq!(drained.len(), 3, "the drain returns raw lines");
let parsed: Vec<Event> = drained
.iter()
.filter_map(|l| serde_json::from_str(l).ok())
.collect();
assert_eq!(parsed.len(), 2, "the torn line must not reach a batch");
assert!(buffer::read_lines(&path).is_empty(), "drain truncates");
assert!(!path.exists(), "taking a spool deletes it");
}
#[test]
@@ -1320,21 +1320,25 @@ fn a_tombstoned_buffer_never_appends_or_drains() {
assert!(buffer::append(&root, &buffer::buffer_path(&root), &line(2)).is_none());
assert!(buffer::append_locked(&root, &buffer::dryrun_path(&root), "{}").is_none());
assert!(buffer::drain(&root).is_empty());
assert!(crate::spool::take_own(&root, &buffer::buffer_path(&root)).is_empty());
}
#[test]
fn arming_truncates_a_pre_consent_buffer() {
fn arming_after_a_tombstone_deletes_spools_and_truncates_the_legacy_buffer() {
let home = temp_home();
let root = root_of(&home);
buffer::ensure_dir(&root).expect("create root");
buffer::append(&root, &buffer::buffer_path(&root), &line(1)).expect("append");
buffer::wipe(&root).expect("wipe");
std::fs::create_dir_all(crate::spool::dir(&root)).expect("spool dir");
let planted = crate::spool::dir(&root).join(format!("{}.jsonl", "a".repeat(32)));
std::fs::write(&planted, "{}\n").expect("plant");
let generation = buffer::tombstone_generation(&root).expect("read wipe generation");
buffer::arm(&root, generation.as_ref(), || true).expect("arm");
assert!(!buffer::tombstone_present(&root));
assert!(buffer::read_lines(&buffer::buffer_path(&root)).is_empty());
assert!(!planted.exists(), "re-enabling must not inherit old spools");
}
#[test]
@@ -1434,6 +1438,21 @@ fn completed_wipe_blocks_identity_and_state_recreation() {
// ------------------------------------------------------------ unarmed gate --
#[test]
fn termination_reporting_is_a_noop_when_unarmed() {
assert!(!crate::is_armed());
crate::record_session_end();
crate::record_session_end_blocking(ExitClass::Signal);
assert_eq!(
crate::report_termination(ExitClass::Signal, Duration::from_millis(10)),
crate::FlushOutcome::Empty
);
assert_eq!(
crate::flush_for_termination(Duration::from_millis(10)),
crate::FlushOutcome::Empty
);
}
#[test]
fn record_blocking_is_a_noop_when_unarmed() {
// The process panic hook is installed before the command line is parsed, so
@@ -1994,3 +2013,309 @@ fn the_notice_summarizes_what_the_schema_collects_and_states_every_red_line() {
assert!(!body.contains("CODEWHALE_TELEMETRY=0"));
assert!(body.contains("docs/TELEMETRY.md"));
}
// ------------------------------------------------------------- the spools --
mod spools {
use super::*;
use crate::actor::{Context, send_dead};
use crate::spool::{self, SpoolContext};
fn header(surface: Surface, version: &str) -> SpoolContext {
SpoolContext::new(
surface,
version.to_string(),
Some("abcdef123456".to_string()),
true,
)
}
fn event_line(source: SessionSource) -> String {
serde_json::to_string(&Event::SessionStart { source }).expect("serialize")
}
fn plant(root: &Path, nonce: char, header_line: &str, events: &[String]) -> PathBuf {
std::fs::create_dir_all(spool::dir(root)).expect("spool dir");
let path = spool::dir(root).join(format!("{}.jsonl", nonce.to_string().repeat(32)));
let mut body = format!("{header_line}\n");
for event in events {
body.push_str(event);
body.push('\n');
}
std::fs::write(&path, body).expect("plant spool");
path
}
fn header_line(surface: Surface, version: &str) -> String {
serde_json::to_string(&header(surface, version)).expect("serialize header")
}
fn current(root: &Path) -> Context {
let header = header(Surface::Tui, "0.10.3");
let (spool, nonce) = spool::open(root, &header);
Context {
root: root.to_path_buf(),
endpoint: None,
surface: Surface::Tui,
config_path: None,
app_version: "0.10.3".to_string(),
git_sha: None,
tty: false,
spool,
nonce,
}
}
fn dryrun_batches(root: &Path) -> Vec<Batch> {
buffer::read_lines(&buffer::dryrun_path(root))
.iter()
.map(|line| serde_json::from_str(line).expect("a dry-run batch"))
.collect()
}
#[test]
fn a_dead_spool_is_recovered_with_its_own_surface_and_version() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
let path = plant(
&root,
'b',
&header_line(Surface::Serve, "0.9.9"),
&[event_line(SessionSource::Interactive)],
);
let outcomes = send_dead(&context);
assert_eq!(outcomes, vec![crate::FlushOutcome::DryRun]);
let batches = dryrun_batches(&root);
assert_eq!(batches.len(), 1);
assert_eq!(batches[0].surface, Surface::Serve);
assert_eq!(batches[0].app_version, "0.9.9");
assert_eq!(batches[0].git_sha.as_deref(), Some("abcdef123456"));
assert_eq!(batches[0].events.len(), 1);
assert!(!path.exists());
}
#[test]
fn a_dead_spool_is_never_sent_twice() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
plant(
&root,
'b',
&header_line(Surface::Exec, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
assert_eq!(send_dead(&context).len(), 1);
assert!(send_dead(&context).is_empty());
assert_eq!(dryrun_batches(&root).len(), 1);
}
#[test]
fn a_live_locked_spool_is_untouched() {
let home = temp_home();
let root = root_of(&home);
let live = header(Surface::Serve, "0.10.3");
let (live_path, live_nonce) = spool::open(&root, &live);
buffer::append(&root, &live_path, &event_line(SessionSource::Interactive)).expect("append");
let taken = spool::take_dead(&root, &"c".repeat(32), std::time::SystemTime::now());
assert!(taken.is_empty());
assert_eq!(buffer::read_lines(&live_path).len(), 2);
assert!(
spool::dir(&root)
.join(format!("{live_nonce}.lock"))
.exists()
);
}
#[test]
fn the_running_process_never_recovers_its_own_spool() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
buffer::append(
&root,
&context.spool,
&event_line(SessionSource::Interactive),
)
.expect("append");
assert!(send_dead(&context).is_empty());
assert_eq!(buffer::read_lines(&context.spool).len(), 2);
}
#[test]
fn a_tombstone_blocks_recovery_and_append() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
buffer::wipe(&root).expect("wipe");
let planted = plant(
&root,
'b',
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
assert!(send_dead(&context).is_empty());
assert!(dryrun_batches(&root).is_empty());
assert!(
planted.exists(),
"nothing is read or sent under a tombstone"
);
assert!(buffer::append(&root, &context.spool, "{}").is_none());
}
#[test]
fn wipe_removes_every_spool() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
plant(
&root,
'b',
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
buffer::wipe(&root).expect("wipe");
assert!(!spool::dir(&root).exists());
assert!(!context.spool.exists());
}
#[test]
fn a_hostile_context_drops_the_whole_spool_unsent() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
let good = event_line(SessionSource::Interactive);
let hostile_headers = [
header_line(Surface::Serve, "/Users/victim/secret"),
header_line(Surface::Serve, "0.10.3").replace("abcdef123456", "ZZZZZZZZZZZZ"),
header_line(Surface::Serve, "0.10.3").replace("\"spool\":1", "\"spool\":2"),
header_line(Surface::Serve, "0.10.3")
.replace("\"schema_version\":", "\"x\":9,\"schema_version\":"),
header_line(Surface::Serve, "0.10.3").replace("\"serve\"", "\"not-a-surface\""),
format!(
"{{\"schema_version\":{},\"notice_version\":{},\"app_version\":\"0.10.3\"}}",
SCHEMA_VERSION + 1,
NOTICE_VERSION
),
good.clone(),
];
for (index, hostile) in hostile_headers.iter().enumerate() {
let nonce = char::from(b'1' + index as u8);
let path = plant(&root, nonce, hostile, std::slice::from_ref(&good));
assert!(send_dead(&context).is_empty(), "{hostile}");
assert!(!path.exists(), "{hostile}");
}
let mut stale = header(Surface::Serve, "0.10.3");
stale.schema_version = SCHEMA_VERSION + 1;
plant(
&root,
'f',
&serde_json::to_string(&stale).expect("serialize"),
std::slice::from_ref(&good),
);
assert!(send_dead(&context).is_empty());
assert!(dryrun_batches(&root).is_empty());
}
#[test]
fn a_hostile_event_in_a_dead_spool_is_dropped_and_the_rest_sent() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
plant(
&root,
'b',
&header_line(Surface::Serve, "0.10.3"),
&[
r#"{"event":"panic","site":"/Users/victim/src/secret-repo/main.rs"}"#.to_string(),
event_line(SessionSource::Interactive),
],
);
assert_eq!(send_dead(&context).len(), 1);
let batches = dryrun_batches(&root);
assert_eq!(batches[0].events.len(), 1);
assert!(!serde_json::to_string(&batches).unwrap().contains("victim"));
}
#[test]
fn the_count_and_age_caps_hold() {
let home = temp_home();
let root = root_of(&home);
let context = current(&root);
let now = std::time::SystemTime::now();
let mut paths = Vec::new();
for (index, nonce) in ['1', '2', '3', '4', '5', '6'].into_iter().enumerate() {
let path = plant(
&root,
nonce,
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
let age = Duration::from_secs(60 * (index as u64 + 1));
std::fs::File::options()
.write(true)
.open(&path)
.expect("open")
.set_modified(now - age)
.expect("set mtime");
paths.push(path);
}
let stale = plant(
&root,
'7',
&header_line(Surface::Serve, "0.10.3"),
&[event_line(SessionSource::Interactive)],
);
std::fs::File::options()
.write(true)
.open(&stale)
.expect("open")
.set_modified(now - Duration::from_secs(8 * 24 * 60 * 60))
.expect("set mtime");
assert_eq!(send_dead(&context).len(), spool::MAX_RECOVERED);
assert_eq!(dryrun_batches(&root).len(), spool::MAX_RECOVERED);
for path in paths.iter().chain([&stale]) {
assert!(!path.exists(), "{} must be gone", path.display());
}
}
#[test]
fn two_processes_never_mix_lines() {
let home = temp_home();
let root = root_of(&home);
let (tui_path, _) = spool::open(&root, &header(Surface::Tui, "0.10.3"));
let (exec_path, _) = spool::open(&root, &header(Surface::Exec, "0.10.3"));
assert_ne!(tui_path, exec_path);
buffer::append(&root, &tui_path, &event_line(SessionSource::Interactive)).expect("append");
buffer::append(&root, &exec_path, &event_line(SessionSource::Ci)).expect("append");
let tui = spool::take_own(&root, &tui_path);
assert!(tui.iter().any(|l| l.contains("interactive")));
assert!(
tui.iter()
.all(|l| !l.contains("\"ci\"") && !l.contains("exec"))
);
assert!(exec_path.exists(), "taking one spool leaves the other");
}
#[test]
fn the_ring_keeps_the_header() {
let home = temp_home();
let root = root_of(&home);
let (path, _) = spool::open(&root, &header(Surface::Serve, "0.10.3"));
for n in 0..700 {
buffer::append(&root, &path, &line(n)).expect("append");
}
let lines = buffer::read_lines(&path);
assert!(lines.len() <= buffer::MAX_EVENTS);
assert!(spool::is_header_line(&lines[0]));
assert!(lines.last().unwrap().contains(&line(699)));
}
}
+1
View File
@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Non-ASCII file names:** `/diff`, the `@git` and `@diff` mentions and the Runtime API workspace diff now show paths with Chinese, emoji and other non-ASCII characters as written instead of as quoted octal escapes, and the turn context no longer lists the workspace root as an empty "Active paths" entry. Thanks [@luochen211](https://github.com/luochen211) (PR 7317).
- **Session names after a crash, and keep-forever snapshot retention:** a session name saved right after a crash that cut off the previous session-index line is no longer lost, because the new entry now starts on its own line; and a very large `[snapshots] max_age_days` (such as `9223372036854775807` for "keep forever") no longer wraps around and deletes every undo snapshot at session start. Thanks [@luochen211](https://github.com/luochen211) (PR 7316).
- **Blank environment variables:** exporting a variable empty (for example `CODEWHALE_MODEL=` from a compose `${VAR:-}` template) no longer hides its legacy alias. `DEEPSEEK_MODEL`, `DEEPSEEK_BASE_URL`, `KIMI_BASE_URL`, `DEEPSEEK_FORCE_HTTP1`, `DEEPSEEK_STREAM_OPEN_TIMEOUT_SECS` and the other provider alias variables now apply as they do when the primary is unset. Thanks [@luochen211](https://github.com/luochen211) (PR 7315).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (PR 7293).
- **MCP and language servers that send requests mid-call:** a tool call no longer fails with "neither a result nor an error" when an MCP server sends a `ping` or other request that reuses the call's id while the call is running, and a language server's `workspace/configuration` or similar request no longer ends a pending LSP request. Codewhale now answers `ping`, answers other server requests with "Method not found" (it never advertised them), keeps waiting for the real reply, and a refused answer over Streamable HTTP no longer drops the connection to legacy SSE. Thanks [@luochen211](https://github.com/luochen211) (PR 7319).
- **Qwen Code as an external agent:** Codewhale now finds an installed Qwen Code CLI (`qwen`, 0.6.1 or newer) and can run it as a session over ACP with `qwen --acp`. `GET /v1/agents/external` lists it, an older install is reported as needing `npm i -g @qwen-code/qwen-code@latest`, and sign-in stays inside Qwen Code (`/auth`). Thanks [@MannXo](https://github.com/MannXo) (#7293).
- **README and contributor credit:** the repository front page now opens the way the website does: the line "The model can be anyone's. The agent should be yours.", one sentence on what Codewhale is, the install command and one picture, with reference detail moved into the linked guides; the 18 translated READMEs follow the same structure and say they are machine-drafted. The contributor wall and the website now count the same 433 people (407 GitHub accounts and the commit authors whose email is not linked to an account), where the website had counted accounts only. `DEPENDENCY_SECURITY.md` moved into `docs/dependency-maintenance.md`, and `.github/SUPPORT.md` says where to ask for help.
+8 -60
View File
@@ -862,6 +862,10 @@ fn spawn_signal_cleanup_task() {
let mut signals = TerminatingSignals::register();
tokio::spawn(async move {
let exit_code = signals.wait().await;
codewhale_telemetry::record_session_end_blocking(codewhale_telemetry::ExitClass::Signal);
let telemetry_send = tokio::task::spawn_blocking(|| {
codewhale_telemetry::flush_for_termination(codewhale_telemetry::SIGNAL_FLUSH_TIMEOUT)
});
// Persistence and a serving Runtime API share a bounded window to
// flush queued saves and end open streams with a typed `stream.end`.
// A second signal skips both waits.
@@ -870,6 +874,9 @@ fn spawn_signal_cleanup_task() {
tokio::join!(
codewhale_app_server::runtime_api::drain_for_signal_exit(),
crate::tui::persistence_actor::flush_for_signal_exit(),
async {
let _ = telemetry_send.await;
},
)
} => {}
_ = signals.wait() => {}
@@ -885,66 +892,11 @@ fn spawn_signal_cleanup_task() {
#[cfg(unix)]
codewhale_runtime::process_tree::kill_contained_trees_for_exit();
crate::tui::ui::emergency_restore_terminal();
// Nothing async survives the `exit` below, so this is the last
// chance to say how the session ended. `record_blocking` is one
// `O_APPEND` write with no lock: taking the compaction lock here
// would let a second Codewhale process sharing CODEWHALE_HOME hang
// Ctrl-C, and the second-signal short-circuit below has to stay
// reachable. A no-op unless this process was armed.
//
// The class is stated, not derived: `RunTerminationReason::Canceled`
// also exits 130, so `exit_code` cannot tell a signal from an
// Esc-cancelled turn.
record_signal_session_end();
}
std::process::exit(exit_code);
});
}
/// When this process's armed telemetry session began. Set once, at arming, and
/// read from both the ordinary teardown and the signal path.
static TELEMETRY_SESSION_START: std::sync::OnceLock<std::time::Instant> =
std::sync::OnceLock::new();
/// Build `session_end` from what this process actually accumulated.
///
/// The exit class is read from the process-wide atomic and never derived from
/// an exit code: `RunTerminationReason::Canceled` maps to 130, the same value
/// the SIGINT path uses, so a code-based derivation would report every
/// Esc-cancelled turn as a signal.
///
/// The cold-start bucket is `None` unless the interactive event loop actually
/// began, which is what keeps it absent rather than invented on the surfaces
/// that have no event loop.
fn telemetry_session_end() -> codewhale_telemetry::Event {
let counters = codewhale_telemetry::session_counters();
codewhale_telemetry::Event::SessionEnd {
duration_bucket: codewhale_telemetry::DurationBucket::from_secs(
TELEMETRY_SESSION_START
.get()
.map_or(0, |start| start.elapsed().as_secs()),
),
exit_class: codewhale_telemetry::exit_class(),
cold_start_bucket: codewhale_runtime::startup_trace::cold_start_ms()
.map(codewhale_telemetry::ColdStartBucket::from_millis),
providers: counters.providers(),
counters: counters.counters(),
errors: counters.errors(),
turn_wall: counters.turn_wall(),
}
}
/// Close the session synchronously, from the signal handler.
///
/// A no-op unless this process was armed.
fn record_signal_session_end() {
codewhale_telemetry::set_exit_class(codewhale_telemetry::ExitClass::Signal);
for event in codewhale_telemetry::session_counters().usage_events() {
codewhale_telemetry::record_blocking(event);
}
codewhale_telemetry::record_blocking(telemetry_session_end());
}
/// Terminating-signal streams, registered up front and awaited later.
///
/// Splitting registration from the await is the point: the OS disposition
@@ -2514,7 +2466,6 @@ fn arm_telemetry_with_setup(
return;
};
codewhale_telemetry::init(consent.with_config_path(Some(store.path().to_path_buf())));
let _ = TELEMETRY_SESSION_START.set(std::time::Instant::now());
codewhale_telemetry::record(codewhale_telemetry::Event::SessionStart { source });
}
@@ -2580,10 +2531,7 @@ async fn finish_telemetry(outcome: &Result<()>, surface: codewhale_telemetry::Su
{
codewhale_telemetry::set_exit_class(codewhale_telemetry::ExitClass::Error);
}
for event in codewhale_telemetry::session_counters().usage_events() {
codewhale_telemetry::record(event);
}
codewhale_telemetry::record(telemetry_session_end());
codewhale_telemetry::record_session_end();
if surface == codewhale_telemetry::Surface::Cli {
let persistence = codewhale_telemetry::persist_local_blocking();
logging::info(format!(
+1 -1
View File
@@ -227,7 +227,7 @@ fn the_session_source_distinguishes_resume_and_fork_from_a_fresh_launch() {
#[test]
fn a_session_end_built_without_arming_writes_nothing() {
let event = telemetry_session_end();
let event = codewhale_telemetry::session_end_event();
assert!(matches!(
event,
codewhale_telemetry::Event::SessionEnd { .. }
+3
View File
@@ -1418,6 +1418,9 @@ pub async fn run_tui(
// than through it. Only this path calls it, which is what keeps the
// cold-start bucket absent on surfaces with no event loop.
codewhale_runtime::startup_trace::mark_cold_start();
if let Some(millis) = codewhale_runtime::startup_trace::cold_start_ms() {
codewhale_telemetry::set_cold_start_ms(millis);
}
let result = run_event_loop(
&mut terminal,
&mut app,
@@ -51,7 +51,7 @@ const SENTINEL_FILENAME: &str = "tc-workspace-sentinel-file.txt";
const SENTINEL_PROVIDER_TABLE: &str = "tc_custom_provider_sentinel";
const SENTINEL_MCP_SERVER: &str = "tc-mcp-server-sentinel";
const SENTINEL_API_KEY: &str = "tc-api-key-sentinel-not-a-real-key";
/// Planted by writing to `buffer.jsonl` directly, which is what any other
/// Planted by writing to a spool file directly, which is what any other
/// process running as this user can do.
const SENTINEL_INJECTED: &str = "tc-injected-sentinel-/Users/victim/secret-repo";
/// The key lives only in the child's environment, never in a file, so the
@@ -279,22 +279,43 @@ async fn assert_no_batches(server: &MockServer, why: &str) {
);
}
fn spool_files(fixture: &Fixture) -> Vec<PathBuf> {
let Ok(entries) = std::fs::read_dir(fixture.telemetry_root().join("spool")) else {
return Vec::new();
};
let mut files: Vec<PathBuf> = entries
.filter_map(Result::ok)
.map(|entry| entry.path())
.filter(|path| path.extension().is_some_and(|ext| ext == "jsonl"))
.collect();
files.sort();
files
}
fn only_spool_file(fixture: &Fixture) -> Option<PathBuf> {
let mut files = spool_files(fixture);
(files.len() == 1).then(|| files.remove(0))
}
fn buffered_events(fixture: &Fixture) -> Vec<Value> {
let path = fixture.telemetry_root().join("buffer.jsonl");
let body = std::fs::read_to_string(&path).unwrap_or_else(|error| {
panic!(
"read locally buffered telemetry at {}: {error}",
path.display()
)
});
body.lines()
.filter(|line| !line.trim().is_empty())
.map(|line| {
serde_json::from_str::<Value>(line).unwrap_or_else(|error| {
let mut events = Vec::new();
for path in spool_files(fixture) {
let body = std::fs::read_to_string(&path).unwrap_or_else(|error| {
panic!(
"read locally buffered telemetry at {}: {error}",
path.display()
)
});
for line in body.lines().filter(|line| !line.trim().is_empty()) {
let value = serde_json::from_str::<Value>(line).unwrap_or_else(|error| {
panic!("buffered telemetry must be JSON: {error}\nline: {line}")
})
})
.collect()
});
if value.get("event").is_some() {
events.push(value);
}
}
}
events
}
async fn assert_short_cli_persistence_without_network(
@@ -380,8 +401,12 @@ async fn incomplete_short_cli_sessions_require_an_explicit_timeout() {
let server = start_recorder().await;
let fixture = Fixture::new().with_endpoint(&server.uri());
std::fs::create_dir_all(fixture.telemetry_root()).expect("create local buffer root");
std::fs::create_dir_all(fixture.telemetry_root().join("spool")).expect("create spool dir");
std::fs::write(
fixture.telemetry_root().join("buffer.jsonl"),
fixture
.telemetry_root()
.join("spool")
.join(format!("{}.jsonl", "a".repeat(32))),
"{\"event\":\"session_start\",\"source\":\"unknown\"}\n",
)
.expect("write incomplete session");
@@ -845,8 +870,10 @@ async fn a_hostile_buffer_line_never_reaches_a_batch() {
let stdout = read_in_background(child.stdout.take().expect("stdout pipe"));
let stderr = read_in_background(child.stderr.take().expect("stderr pipe"));
let buffer = fixture.telemetry_root().join("buffer.jsonl");
wait_until(Duration::from_secs(30), || buffer.exists());
wait_until(Duration::from_secs(30), || {
only_spool_file(&fixture).is_some()
});
let buffer = only_spool_file(&fixture).expect("this session's spool");
append_lines(
&buffer,
&[
@@ -929,11 +956,10 @@ async fn mid_session_opt_out_stops_the_shutdown_flush() {
// Wait until this session's event is actually buffered, then leave enough
// time for any accidental background flush to reach the recorder. Nothing
// may be sent before shutdown.
let buffer = fixture.telemetry_root().join("buffer.jsonl");
wait_until(Duration::from_secs(30), || {
std::fs::read_to_string(&buffer)
.map(|body| body.contains("\"event\":\"session_start\""))
.unwrap_or(false)
only_spool_file(&fixture)
.and_then(|buffer| std::fs::read_to_string(buffer).ok())
.is_some_and(|body| body.contains("\"event\":\"session_start\""))
});
tokio::time::sleep(Duration::from_millis(250)).await;
assert_no_batches(&server, "before the shutdown flush").await;
@@ -981,7 +1007,7 @@ async fn ctrl_c_exits_while_a_second_process_holds_the_lock() {
// Arming itself takes the lock, so the holder below must wait until the
// session is armed — otherwise this test would pin a deadlock it created.
wait_until(Duration::from_secs(30), || {
root.join("buffer.jsonl").exists()
only_spool_file(&fixture).is_some()
});
let _holder = LockHolder::take(&lock_path);
+49 -20
View File
@@ -65,11 +65,11 @@ codewhale --telemetry false # the same kill switch, for one command
**`telemetry = false` in the config file is the opt-out.** It is a floor:
`--telemetry true` and `CODEWHALE_TELEMETRY=1` both lose to it, because a
setting you can undo by accident from a wrapper script is not a setting. It
deletes the random install id, truncates every buffered event and every dry-run
deletes the random install id, deletes every spool, truncates every dry-run
record, and writes a tombstone. Appends, identity/state writes, and delivery all
share the wipe's ordering lock, so once opt-out returns no pre-opt-out write or
POST remains in flight. If any part of that wipe fails, the tombstone is still
there and the buffer is undrainable — a failed wipe fails closed. Every later
there and no spool can be sent — a failed wipe fails closed. Every later
run re-asserts the same tombstone for as long as the setting stands, so it
survives. Turning telemetry back on in `/settings` records that explicit preference
in both existing privacy registers; a fresh launch can then clear the tombstone. Nothing buffered before that point is
@@ -101,17 +101,21 @@ Everything is under `$CODEWHALE_HOME/telemetry/` (`0700`), every file `0600`:
| file | role |
|---|---|
| `buffer.jsonl` | pending events, one JSON object per line |
| `spool/<nonce>.jsonl` | one running process's pending events: a header line (schema, notice and app version, release sha, surface, whether a terminal), then one JSON object per line |
| `spool/<nonce>.lock` | held by that process while it runs; a spool whose lock is free belongs to a process that has ended |
| `buffer.jsonl.lock` | the sibling ordering lock shared by writes, delivery, arming, and wipe |
| `buffer.jsonl` | left over from older versions; only ever emptied, never written or sent |
| `dryrun.jsonl` | where batches go when the endpoint is configured empty |
| `state.json` | the last app version seen and the last flush attempt |
| `install_id.json` | the random install id and when it was minted |
| `disabled` | the tombstone; present means nothing is appended or sent |
Both `buffer.jsonl` and `dryrun.jsonl` are rings capped at 512 records or
256 KiB, whichever comes first, with the oldest dropped. The documented
footprint ceiling for the whole directory is therefore **512 KiB plus a few
hundred bytes of metadata**.
The nonce is random, names a file on your disk and is never sent. Each spool and
`dryrun.jsonl` is a ring capped at 512 records (the header counts as one) or
256 KiB, whichever comes first, with the oldest events dropped. The documented
footprint ceiling is therefore **256 KiB for each running process, 256 KiB for
`dryrun.jsonl`, and at most four leftover spools of ended processes kept after
each TUI, `exec` or `serve` start**, plus a few hundred bytes of metadata.
The install id is a random v4 UUID. It is never derived from your hostname,
MAC address, `machine-id`, home directory, username, or executable path — a
@@ -126,18 +130,43 @@ claim one.
## When anything is sent, and where
Nothing is collected or sent when a persistent opt-out or run-scoped kill
switch is in force, or when privacy state is unreadable. TUI and `exec` sessions have exactly one network flush point: an attempt
during shutdown, bounded at three seconds. Short CLI commands such as `config`,
`doctor`, and `auth` do not wait for that network request. They record
`session_end`, seal the event to the local buffer with a much shorter bound, and
return; a configured endpoint sends those buffered events with the next
interactive shutdown flush. An explicitly empty endpoint finalizes its local
dry-run batch immediately.
switch is in force, or when privacy state is unreadable. TUI and `exec` sessions have one network flush point at a normal end: an attempt
during shutdown, bounded at three seconds. That flush sends only the process's
own spool, under that process's own surface and version, and deletes it before
the request is made. Short CLI commands such as `config`, `doctor`, and `auth`
do not wait for that network request. They record `session_end`, seal the event
to their own spool with a much shorter bound, and return; the next TUI, `exec`
or `serve` start sends what they left (below). An explicitly empty endpoint
finalizes its local dry-run batch immediately.
There is no startup flush, mid-session flush, per-turn flush, or per-tool-call
flush. Every network shutdown flush re-resolves your setting from disk
A session that is ended early makes one more bounded attempt, so its usage is
not lost with it. When a terminating signal (interrupt, terminate, hang-up, or
console close) reaches a TUI, `exec` or `serve` session, and when `exec` ends
because the program reading its output has closed the pipe, the session records
`session_end` with exit class `signal`, then makes one send attempt bounded at
one and a half seconds, alongside the process's own shutdown work. A second
signal ends the process at once without waiting. This is the same single `POST`
of the same fields, never retried. Short CLI commands stay local-only on this
path too.
A session that ends without any report (killed outright, a power loss, a crash
before the handler runs) leaves its spool behind. When a later TUI, `exec` or
`serve` session starts, it sends the leftovers once: at most four, newest first,
one `POST` per leftover session, each carrying that session's own surface and
app version and this installation's current install id and send time. Each
leftover is deleted before its request is made and is never retried. A leftover
older than seven days, one beyond the newest four, one written by a build with a
different schema or notice version, and any line that fails the schema bounds is
deleted without being sent. Consent and the opt-out tombstone are checked
before anything is read and again before each request, and the request runs on
its own thread so it never delays your session. Opting out deletes every spool.
Opting back in starts without any spool from before.
There is no mid-session flush, per-turn flush, or per-tool-call
flush. Every network flush re-resolves your setting from disk
immediately beforehand, so `codewhale config set telemetry false` written from
another terminal stops the flush of a session that is already running.
another terminal stops the flush of a session that is already running, including
one that is being terminated.
A flush is **one `POST`** to the resolved endpoint — by default
`https://telemetry.codewhale.net/v1/telemetry`. The request carries a
@@ -206,7 +235,7 @@ new fields, surfaces, and `product_usage` events.
| `arch` | enum | `std::env::consts::ARCH` | `x86_64 \| aarch64 \| other`. |
| `libc` | enum | `cfg!(target_env)` — **compile time** | `gnu \| musl \| none`. Runtime detection reads distro vendor strings; compile-time is free and leaks nothing. |
| `tty` | bool | `std::io::IsTerminal`, as at `crates/telemetry/src/envelope.rs:196` | `stdin().is_terminal() && stdout().is_terminal()`. |
| `events` | array | the drained buffer | Every element is one of the six events below and nothing else. Capped at 200 events or 64 KiB per batch; a batch that would exceed either cap leaves the remainder buffered for the next flush. |
| `events` | array | the process's spool | Every element is one of the six events below and nothing else. Capped at 200 events or 64 KiB per batch; events beyond either cap are dropped with the spool, never carried to a later flush. |
**`os_major` is not collected.** Reading it costs unsafe FFI on two platforms plus a file parser on a third, in the one crate whose entire value is being small enough to audit — and `os`, `arch`, and `libc` are free and answer the platform question. It may be reconsidered if the stored data ever shows that the OS-version cut is what triage is missing; a hunch is not that evidence.
@@ -317,7 +346,7 @@ The workhorse. Everything a session accumulated ships here, once.
**`duration_bucket`** — `chrono` delta from `app.session_started_at` (`crates/tui/src/tui/app.rs:1889`). Half-open, seconds: `lt_1m` (`d < 60`), `1m_10m` (`60 ≤ d < 600`), `10m_60m` (`600 ≤ d < 3600`), `gt_60m` (`d ≥ 3600`).
**`exit_class`** — `clean | signal | panic | error`. **Derived from an explicit `AtomicU8`, never from an exit code.** `RunTerminationReason::Canceled` maps to exit 130 (`crates/engine/src/core/runtime_contract/termination.rs:53`), the same value the signal task uses (`crates/tui/src/lib.rs:682`, 128+SIGINT), so a code-based derivation would report every Esc-cancelled turn as a signal. The atomic is set by the panic hook (`crates/tui/src/lib.rs:1582`), by the signal task (`:678-696`) before `std::process::exit`, and on the clean path from `RunTerminationReason::is_success()` (`crates/engine/src/core/runtime_contract/termination.rs:44-46`) — `error` otherwise. Exit codes remain insufficient: an approval-required exit (3) and a terminal exec failure (1) need their actual termination classification.
**`exit_class`** — `clean | signal | panic | error`. **Derived from an explicit `AtomicU8`, never from an exit code.** `RunTerminationReason::Canceled` maps to exit 130 (`crates/engine/src/core/runtime_contract/termination.rs:53`), the same value the signal task uses (`crates/tui/src/lib.rs:682`, 128+SIGINT), so a code-based derivation would report every Esc-cancelled turn as a signal. The atomic is set by the panic hook (`crates/tui/src/lib.rs:1582`), by the signal task when the first signal arrives, by `exec` when its output pipe closes, and on the clean path from `RunTerminationReason::is_success()` (`crates/engine/src/core/runtime_contract/termination.rs:44-46`) — `error` otherwise. Exit codes remain insufficient: an approval-required exit (3) and a terminal exec failure (1) need their actual termination classification.
**`cold_start_bucket`** — from `startup_trace::elapsed_ms()`, which reads `PROCESS_START` directly and is independent of the startup summary's buffer clear (`crates/runtime/src/startup_trace.rs:39-45`). Boundaries: `lt_250`, `250_1000`, `1000_3000`, `gte_3000`. Absent on non-TUI surfaces.
@@ -541,7 +570,7 @@ is established by this source change.
`install_id` rotates client-side every 90 days (`rotated_at` in `install_id.json`), so no single identifier spans a long history. This costs longitudinal accuracy and the docs say so: **no count derived from `install_id` is a user count.** It is a lower bound on distinct machine-installs in a window, and it undercounts a returning user across a rotation.
**Turning it off deletes what was kept locally, not what was already sent.** `codewhale config set telemetry false` erases the install id, the buffer, and the dry-run records on your machine, and stops anything further. Rows already accepted by the endpoint are keyed only by a rotating random id that is now gone; first-party rows age out with the three-month window. If PostHog processing is activated, already-delivered processor records follow that project’s configured retention. The client has no remote deletion API and does not claim that local opt-out erases delivered records.
**Turning it off deletes what was kept locally, not what was already sent.** `codewhale config set telemetry false` erases the install id, every spool, and the dry-run records on your machine, and stops anything further. Rows already accepted by the endpoint are keyed only by a rotating random id that is now gone; first-party rows age out with the three-month window. If PostHog processing is activated, already-delivered processor records follow that project’s configured retention. The client has no remote deletion API and does not claim that local opt-out erases delivered records.
### What the owner reads back — observed active installs
-10
View File
@@ -2,16 +2,6 @@
## Unreleased — 0.2.0
- Add data displays: `DataSparkline`, `DataBars` and `DataTrend` (a braille
line chart). The words carry every value; the ink draws in from a caller
progress (`MotionStep::drawn`) and is at its end state under reduced or
still motion. Bars glide from their previous values.
- Add `DataDisplay`, the same displays as JSON an agent can send:
`DataDisplay::from_json` validates it (finite numbers, at most 1024 points
or 24 bars, labels up to 120 characters), `describe` says the same facts as
plain text, and `DATA_DISPLAY_SCHEMA` is its tool schema. Try
`cargo run --example data`.
- Reexport the shared avatar contract from Codewhale Protocol. Rendering and
artwork stay in this library. Protocol 0.10.3 registry proof must precede
this package publication.
-1
View File
@@ -57,7 +57,6 @@ Gallery names are the exact names accepted by the gallery example; the
| Engine `agent_roster.rs::render_agent_roster`, `widgets/workflow_panel.rs::row_receipt_text`, `gate_receipts.rs`; GPUI `usage.rs::TurnRow`, `workbar.rs::Readout`, `working_context.rs::ExecutionReceipt` | `Receipt`, `ReceiptTable`, `ReceiptValue`, `Cost`, formatting helpers | Primitive/composite | `receipt-row`, `receipt-table`, `receipt-table-compact`, `receipt-table-minimal`, `receipt-table-clipped` |
| Engine `diff_render.rs::BoundedDiffRender`, `render_diff_bounded`, `history/file_mutation.rs`; GPUI `review.rs::DiffRowKind`, `workspace/render/review.rs::ReviewRows` | `Diff`, `DiffLine`, `DiffKind`, `DiffGutter`, `DiffHighlight`, `parse_unified` | Primitive | `diff`, `diff-wrapped`, `diff-no-numbers`, `diff-highlighted`, `review-scene` |
| Engine `widgets/workflow_panel.rs::WorkflowPanel`, `WorkflowPanelRow`, `history/checklist.rs`; GPUI `plans.rs`, `workspace/render/workers.rs` | `WorkflowTree`, `TreeNode`, `TreeState`, `CountBar` | Primitive/composite | `workflow-tree`, `workflow-tree-selected`, `workflow-tree-clipped`, `workflow-tree-long`, `count-bars` |
| Kit-native; data from the host or an agent's JSON (CW-8202) | `DataSparkline`, `DataBars`, `DataBarItem`, `DataTrend`, `DataDisplay`, `DataDisplayView`, `DataWords`, `DATA_DISPLAY_SCHEMA`, `MotionStep::drawn` | Primitive | `data-sparkline`, `data-bars`, `data-bars-growing`, `data-trend`, `data-trend-drawing`, `data-from-agent` |
| Engine `approval/view.rs::ApprovalView`, `approval/elevation.rs::ElevationView`, `auto_review.rs`; GPUI `workspace/render/conversation.rs::Workspace::render_attention`, `workspace/render/review.rs::Workspace::render_restore_confirmation` | `ApprovalCard`, `ApprovalSubject`, `ApprovalState`, `ApprovalChoice`, `ApprovalPaint`, `DecisionBand`, `DecisionBandPlan`, `DecisionBandAction`, `DecisionBandSave`, `ReviewVerdict`, `ReviewAggregate` | Composite | `approval-command`, `approval-outside`, `approval-patch`, `approval-elevation`, `approval-clipped`, `approval-spoofed`, `approval-native-band`, `approval-native-band-collapsed`, `review-verdicts`, `review-aggregate` |
| GPUI `workspace/render/conversation.rs::Workspace::render_attention`, `workspace/render/files.rs::Workspace::render_artifact_column` | `AttentionQueue`, `AttentionItem`, `ArtifactShelf`, `Artifact` | Composite | `attention-queue`, `attention-focused`, `attention-narrow`, `attention-empty`, `artifact`, `artifact-shelf`, `artifact-narrow`, `artifact-unknown`, `artifact-empty` |
| Engine `views/mod.rs::ConfigView`, `ConfigRow`, `ConfigView::render_setting_detail`; GPUI `settings.rs::Spec`, `Applies`, `workspace/settings/mod.rs::Settings` | `SettingRow`, `SettingDetail`, `SettingWords` | Composite | `setting-row`, `setting-detail` |
-276
View File
@@ -1,276 +0,0 @@
//! Live data displays: `cargo run --example data`.
//!
//! A sparkline takes a new value every 400 ms, the trend draws in, and the
//! bars glide to new values every three seconds. r replays, m switches
//! between full and reduced motion, q or Escape exits.
//!
//! `--json FILE` shows one agent-sent [`DataDisplay`] instead.
//! `--text` prints the final frame and each display's plain description.
//! `--svg DIR` writes frames at fixed draw-in points.
use std::{
io,
path::PathBuf,
time::{Duration, Instant},
};
use codewhale_ratatui::{
DataBarItem, DataDisplay, DataWords, MotionEasing, MotionMode, MotionPolicy, MotionStep,
MotionTiming, Paint, Role, Theme,
testing::{self, Profile},
};
use crossterm::{
event::{self, Event, KeyCode, KeyEventKind},
execute,
terminal::{EnterAlternateScreen, LeaveAlternateScreen, disable_raw_mode, enable_raw_mode},
};
use ratatui::{Terminal, backend::CrosstermBackend, buffer::Buffer, layout::Rect};
const TICK: Duration = Duration::from_millis(400);
const RESHUFFLE: Duration = Duration::from_secs(3);
const DRAW_IN: MotionTiming =
MotionTiming::space(Duration::from_millis(1_400), MotionEasing::EaseOut);
const GLIDE: MotionTiming = MotionTiming::PANEL;
struct Noise(u64);
impl Noise {
fn next(&mut self) -> f64 {
self.0 = self
.0
.wrapping_mul(6_364_136_223_846_793_005)
.wrapping_add(1);
(self.0 >> 33) as f64 / f64::from(1u32 << 31)
}
}
struct Board {
noise: Noise,
requests: Vec<f64>,
latency: Vec<f64>,
bars: DataDisplay,
bars_before: DataDisplay,
draw: MotionStep,
glide: MotionStep,
agent: Option<DataDisplay>,
}
fn crate_bars(noise: &mut Noise) -> DataDisplay {
DataDisplay::Bars {
label: "Tests by crate".into(),
unit: "tests".into(),
items: ["engine", "tui", "runtime", "protocol", "config"]
.into_iter()
.map(|name| DataBarItem::new(name, (40.0 + noise.next() * 400.0).round()))
.collect(),
}
}
impl Board {
fn new(agent: Option<DataDisplay>) -> Self {
let mut noise = Noise(7);
let latency = (0..96)
.map(|i| 80.0 + 45.0 * (f64::from(i) / 9.0).sin() + noise.next() * 30.0)
.collect();
let requests = (0..40).map(|_| 20.0 + noise.next() * 60.0).collect();
let bars = crate_bars(&mut noise);
Self {
noise,
requests,
latency,
bars_before: DataDisplay::Bars {
label: String::new(),
unit: String::new(),
items: Vec::new(),
},
bars,
draw: MotionStep::at_rest(0.0),
glide: MotionStep::at_rest(1.0),
agent,
}
}
fn replay(&mut self, now: Instant, policy: MotionPolicy) {
self.draw = MotionStep::at_rest(0.0);
self.draw.go(1.0, now, DRAW_IN, policy);
}
fn tick(&mut self) {
let last = self.requests.last().copied().unwrap_or(50.0);
let next = (last + (self.noise.next() - 0.5) * 30.0).clamp(5.0, 120.0);
self.requests.push(next.round());
if self.requests.len() > 200 {
self.requests.remove(0);
}
}
fn reshuffle(&mut self, now: Instant, policy: MotionPolicy) {
self.bars_before = std::mem::replace(&mut self.bars, crate_bars(&mut self.noise));
self.glide = MotionStep::at_rest(0.0);
self.glide.go(1.0, now, GLIDE, policy);
}
fn paint(&self, area: Rect, buf: &mut Buffer, theme: &Theme, draw: f32, glide: f32) {
buf.set_style(area, theme.bg(Role::Background));
let inner = Rect::new(
area.x + 2,
area.y + 1,
area.width.saturating_sub(4),
area.height.saturating_sub(2),
);
if let Some(agent) = &self.agent {
agent.view().drawn(draw).paint(inner, buf, theme);
return;
}
let row =
|y: u16, h: u16| Rect::new(inner.x, inner.y + y, inner.width, h).intersection(inner);
let requests = DataDisplay::Sparkline {
label: "Requests".into(),
unit: "rpm".into(),
values: self.requests.clone(),
};
let latency = DataDisplay::Trend {
label: "Latency".into(),
unit: "ms".into(),
values: self.latency.clone(),
rows: Some(9),
};
requests.view().drawn(draw).paint(row(0, 1), buf, theme);
latency.view().drawn(draw).paint(row(2, 9), buf, theme);
self.bars
.view()
.from(&self.bars_before)
.drawn(glide.min(draw))
.paint(row(12, 6), buf, theme);
}
fn displays(&self) -> Vec<DataDisplay> {
if let Some(agent) = &self.agent {
return vec![agent.clone()];
}
vec![
DataDisplay::Sparkline {
label: "Requests".into(),
unit: "rpm".into(),
values: self.requests.clone(),
},
DataDisplay::Trend {
label: "Latency".into(),
unit: "ms".into(),
values: self.latency.clone(),
rows: Some(9),
},
self.bars.clone(),
]
}
}
fn main() -> io::Result<()> {
let args: Vec<String> = std::env::args().skip(1).collect();
let value = |flag: &str| {
args.iter()
.position(|a| a == flag)
.and_then(|i| args.get(i + 1))
.cloned()
};
let agent = match value("--json") {
Some(path) => {
let json = std::fs::read_to_string(&path)?;
match DataDisplay::from_json(&json) {
Ok(shown) => Some(shown),
Err(e) => {
eprintln!("{path}: {e}");
std::process::exit(2);
}
}
}
None => None,
};
let board = Board::new(agent);
if args.iter().any(|a| a == "--text") {
let buf = testing::render(84, 20, |area, buf| {
board.paint(area, buf, &Profile::DarkTrue.theme(), 1.0, 1.0);
});
println!("{}", testing::text(&buf));
for shown in board.displays() {
println!("{}", shown.describe(&DataWords::default()));
}
return Ok(());
}
if let Some(dir) = value("--svg") {
let dir = PathBuf::from(dir);
std::fs::create_dir_all(&dir)?;
let theme = Profile::DarkTrue.theme();
for (i, at) in [0.2_f32, 0.5, 0.8, 1.0].into_iter().enumerate() {
let buf = testing::render(84, 20, |area, buf| board.paint(area, buf, &theme, at, at));
let path = dir.join(format!("data-{i}.svg"));
std::fs::write(&path, testing::svg(&buf, &theme))?;
println!("{}", path.display());
}
return Ok(());
}
run(board)
}
fn run(mut board: Board) -> io::Result<()> {
enable_raw_mode()?;
let mut out = io::stdout();
execute!(out, EnterAlternateScreen)?;
let mut terminal = Terminal::new(CrosstermBackend::new(out))?;
let theme = Theme::detect().tui();
let mut mode = MotionMode::Full;
let mut policy = MotionPolicy::new(mode, &theme);
let start = Instant::now();
board.replay(start, policy);
let (mut next_tick, mut next_shuffle) = (start + TICK, start + RESHUFFLE);
let result = loop {
let now = Instant::now();
if now >= next_tick {
board.tick();
next_tick = now + TICK;
}
if now >= next_shuffle {
board.reshuffle(now, policy);
next_shuffle = now + RESHUFFLE;
}
let (draw, glide) = (
board.draw.drawn(now, policy),
board.glide.drawn(now, policy),
);
terminal.draw(|f| board.paint(f.area(), f.buffer_mut(), &theme, draw, glide))?;
let due = [
board.draw.next_frame_at(now, policy),
board.glide.next_frame_at(now, policy),
]
.into_iter()
.flatten()
.chain([next_tick, next_shuffle])
.min()
.unwrap_or(next_tick);
if event::poll(due.saturating_duration_since(Instant::now()))?
&& let Event::Key(key) = event::read()?
&& key.kind != KeyEventKind::Release
{
match key.code {
KeyCode::Char('q') | KeyCode::Esc => break Ok(()),
KeyCode::Char('r') => board.replay(Instant::now(), policy),
KeyCode::Char('m') => {
mode = if mode == MotionMode::Full {
MotionMode::Reduced
} else {
MotionMode::Full
};
policy = MotionPolicy::new(mode, &theme);
}
_ => {}
}
}
};
disable_raw_mode()?;
execute!(terminal.backend_mut(), LeaveAlternateScreen)?;
result
}
File diff suppressed because it is too large. Load diff
-2
View File
@@ -59,12 +59,10 @@ pub use tabs::*;
pub use toggle::*;
// Package Display.
mod chart;
mod diff;
mod progress;
mod receipt;
mod tree;
pub use chart::*;
pub use diff::*;
pub use progress::*;
pub use receipt::*;
-8
View File
@@ -366,14 +366,6 @@ impl MotionStep {
}
}
/// Progress for a draw-in, in `0..=1`: eased where `policy` lets space
/// move, the target where it does not. Feed it to a data display's
/// `drawn`.
#[must_use]
pub fn drawn(&self, now: Instant, policy: MotionPolicy) -> f32 {
self.shown(now, policy, MotionChannel::Space)
}
/// A column between `from` and `to`: slide a marker, nudge a panel.
#[must_use]
pub fn offset(&self, now: Instant, policy: MotionPolicy, from: u16, to: u16) -> u16 {
@@ -1,360 +0,0 @@
---
source: src/components/chart.rs
assertion_line: 1242
expression: "testing::snapshot(12, |area, buf, theme|\n{\n let row = |y: u16, h: u16|\n { Rect::new(area.x, area.y + y, area.width, h).intersection(area) };\n DataSparkline::new(\"Requests\",\n &values).unit(\"rpm\").paint(row(0, 1), buf, theme);\n DataBars::new(\"Tokens by model\",\n &items).unit(\"tokens\").paint(row(2, 5), buf, theme);\n DataTrend::new(\"Latency\",\n &values).unit(\"ms\").rows(5).paint(row(7, 5), buf, theme);\n})"
---
== dark-truecolor · 40x12
Requests ▁▃▅▄▅▆▇███▆▅▄▃▂▁ now 45.6 rpm
Tokens by model
opus ██████████████████ 12.4k tokens
sonnet ███████████▌ 7900 tokens
haiku ████▌ 3100 tokens
cache 0 tokens
Latency now 45.6 ms
69.7 ms │ ⡠⠒⠑⠒⠒⠢⣀ ⡀⡠⠔⠉⠉⢢⡀
│⢀⠔⠁ ⢇ ⢀⠔⠉ ⠈⠑⢄
│⠁ ⠑⠢⡀ ⡠⠊
17.2 ms │ ⠈⠱⣀⣀⠤⠊⠊
---
r0 c0..8 fg=Foreground bg=Background bold
r0 c8..10 bg=Background
r0 c10..26 fg=Accent bg=Background
r0 c26..28 bg=Background
r0 c28..32 fg=Muted bg=Background
r0 c32..40 fg=Foreground bg=Background
r1 c0..40 bg=Background
r2 c0..15 fg=Foreground bg=Background bold
r2 c15..40 bg=Background
r3 c0..6 fg=Foreground bg=Background
r3 c6..8 bg=Background
r3 c8..26 fg=Accent bg=Background
r3 c26..28 bg=Background
r3 c28..40 fg=Muted bg=Background
r4 c0..6 fg=Foreground bg=Background
r4 c6..8 bg=Background
r4 c8..20 fg=Accent bg=Background
r4 c20..28 bg=Background
r4 c28..39 fg=Muted bg=Background
r4 c39..40 bg=Background
r5 c0..6 fg=Foreground bg=Background
r5 c6..8 bg=Background
r5 c8..13 fg=Accent bg=Background
r5 c13..28 bg=Background
r5 c28..39 fg=Muted bg=Background
r5 c39..40 bg=Background
r6 c0..6 fg=Foreground bg=Background
r6 c6..28 bg=Background
r6 c28..36 fg=Muted bg=Background
r6 c36..40 bg=Background
r7 c0..7 fg=Foreground bg=Background bold
r7 c7..9 bg=Background
r7 c9..13 fg=Muted bg=Background
r7 c13..20 fg=Foreground bg=Background
r7 c20..40 bg=Background
r8 c0..8 fg=Muted bg=Background
r8 c8..9 fg=Dim bg=Background
r8 c9..11 bg=Background
r8 c11..18 fg=Accent bg=Background
r8 c18..31 bg=Background
r8 c31..38 fg=Accent bg=Background
r8 c38..40 bg=Background
r9 c0..8 fg=Muted bg=Background
r9 c8..9 fg=Dim bg=Background
r9 c9..12 fg=Accent bg=Background
r9 c12..18 bg=Background
r9 c18..19 fg=Accent bg=Background
r9 c19..29 bg=Background
r9 c29..32 fg=Accent bg=Background
r9 c32..37 bg=Background
r9 c37..40 fg=Accent bg=Background
r10 c0..8 fg=Muted bg=Background
r10 c8..9 fg=Dim bg=Background
r10 c9..10 fg=Accent bg=Background
r10 c10..19 bg=Background
r10 c19..22 fg=Accent bg=Background
r10 c22..28 bg=Background
r10 c28..30 fg=Accent bg=Background
r10 c30..40 bg=Background
r11 c0..8 fg=Muted bg=Background
r11 c8..9 fg=Dim bg=Background
r11 c9..21 bg=Background
r11 c21..28 fg=Accent bg=Background
r11 c28..40 bg=Background
== dark-truecolor · 80x12
Requests ▂▁▁▁▁▁▂▂▂▃▃▄▅▆▆▆▇▇████▇▇▆▆▅▅ now 45.6 rpm min 17.2 rpm max 69.7 rpm
Tokens by model
opus ██████████████████████████████████████████████████████████ 12.4k tokens
sonnet █████████████████████████████████████ 7900 tokens
haiku ██████████████▌ 3100 tokens
cache 0 tokens
Latency now 45.6 ms min 17.2 ms max 69.7 ms
69.7 ms │ ⣀⠤⠒⠊⠑⠒⠒⠒⠒⠒⠒⠢⠤⢄⣀ ⣀ ⣀⡠⠤⠒⠒⠉⠉⠉⠉⠢⢄⣀⣀
│ ⣀⠤⠒⠉ ⠑⠤⣀ ⢀⡠⠒⠉ ⠉ ⠉⠑⠒⠤⣀
│⠉ ⠉⠑⠒⠤⢄⣀ ⡠⠔⠊⠁
17.2 ms │ ⠉⠑⢄⣀⣀⣀⣀⣀⠤⠤⠒⠊⠒⠊⠉
---
r0 c0..8 fg=Foreground bg=Background bold
r0 c8..10 bg=Background
r0 c10..38 fg=Accent bg=Background
r0 c38..40 bg=Background
r0 c40..44 fg=Muted bg=Background
r0 c44..52 fg=Foreground bg=Background
r0 c52..54 bg=Background
r0 c54..58 fg=Muted bg=Background
r0 c58..66 fg=Foreground bg=Background
r0 c66..68 bg=Background
r0 c68..72 fg=Muted bg=Background
r0 c72..80 fg=Foreground bg=Background
r1 c0..80 bg=Background
r2 c0..15 fg=Foreground bg=Background bold
r2 c15..80 bg=Background
r3 c0..6 fg=Foreground bg=Background
r3 c6..8 bg=Background
r3 c8..66 fg=Accent bg=Background
r3 c66..68 bg=Background
r3 c68..80 fg=Muted bg=Background
r4 c0..6 fg=Foreground bg=Background
r4 c6..8 bg=Background
r4 c8..45 fg=Accent bg=Background
r4 c45..68 bg=Background
r4 c68..79 fg=Muted bg=Background
r4 c79..80 bg=Background
r5 c0..6 fg=Foreground bg=Background
r5 c6..8 bg=Background
r5 c8..23 fg=Accent bg=Background
r5 c23..68 bg=Background
r5 c68..79 fg=Muted bg=Background
r5 c79..80 bg=Background
r6 c0..6 fg=Foreground bg=Background
r6 c6..68 bg=Background
r6 c68..76 fg=Muted bg=Background
r6 c76..80 bg=Background
r7 c0..7 fg=Foreground bg=Background bold
r7 c7..9 bg=Background
r7 c9..13 fg=Muted bg=Background
r7 c13..20 fg=Foreground bg=Background
r7 c20..22 bg=Background
r7 c22..26 fg=Muted bg=Background
r7 c26..33 fg=Foreground bg=Background
r7 c33..35 bg=Background
r7 c35..39 fg=Muted bg=Background
r7 c39..46 fg=Foreground bg=Background
r7 c46..80 bg=Background
r8 c0..8 fg=Muted bg=Background
r8 c8..9 fg=Dim bg=Background
r8 c9..14 bg=Background
r8 c14..29 fg=Accent bg=Background
r8 c29..60 bg=Background
r8 c60..61 fg=Accent bg=Background
r8 c61..62 bg=Background
r8 c62..75 fg=Accent bg=Background
r8 c75..80 bg=Background
r9 c0..8 fg=Muted bg=Background
r9 c8..9 fg=Dim bg=Background
r9 c9..10 bg=Background
r9 c10..14 fg=Accent bg=Background
r9 c14..29 bg=Background
r9 c29..32 fg=Accent bg=Background
r9 c32..56 bg=Background
r9 c56..60 fg=Accent bg=Background
r9 c60..61 bg=Background
r9 c61..62 fg=Accent bg=Background
r9 c62..75 bg=Background
r9 c75..80 fg=Accent bg=Background
r10 c0..8 fg=Muted bg=Background
r10 c8..9 fg=Dim bg=Background
r10 c9..10 fg=Accent bg=Background
r10 c10..32 bg=Background
r10 c32..38 fg=Accent bg=Background
r10 c38..53 bg=Background
r10 c53..57 fg=Accent bg=Background
r10 c57..80 bg=Background
r11 c0..8 fg=Muted bg=Background
r11 c8..9 fg=Dim bg=Background
r11 c9..38 bg=Background
r11 c38..53 fg=Accent bg=Background
r11 c53..80 bg=Background
== dark-truecolor · 120x12
Requests ▄▅▆▆▇██▇██▇▇▇▇▅▅▄▄▃▃▂▁▁▁▁▁▂▂▂▃▃▄▅▆▆▆▇▇████▇▇▆▆▅▅ now 45.6 rpm min 17.2 rpm max 69.7 rpm
Tokens by model
opus ██████████████████████████████████████████████████████████████████████████████████████████████████ 12.4k tokens
sonnet ██████████████████████████████████████████████████████████████▍ 7900 tokens
haiku ████████████████████████▌ 3100 tokens
cache 0 tokens
Latency now 45.6 ms min 17.2 ms max 69.7 ms
69.7 ms │ ⢀⣀⠤⠤⠒⠒⠊⠉⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠢⠤⠤⠤⣀⣀ ⣀⡀ ⣀⣀⠤⠤⠒⠒⠒⠊⠉⠉⠉⠉⠉⠉⠒⠤⢄⣀⣀⣀
│ ⢀⣀⠤⠔⠒⠉⠁ ⠉⠢⠤⣀⣀ ⣀⡠⠤⠒⠊⠉ ⠈⠉ ⠉⠉⠑⠒⠒⠤⠤⣀
│⠉⠁ ⠉⠉⠒⠒⠢⠤⢄⣀⣀ ⢀⣀⠤⠔⠊⠉
17.2 ms │ ⠉⠉⠑⠢⢄⣀⣀⣀⣀⣀⣀⣀⡠⠤⠤⠤⠒⠒⠊⠑⠒⠒⠉⠁
---
r0 c0..8 fg=Foreground bg=Background bold
r0 c8..10 bg=Background
r0 c10..58 fg=Accent bg=Background
r0 c58..60 bg=Background
r0 c60..64 fg=Muted bg=Background
r0 c64..72 fg=Foreground bg=Background
r0 c72..74 bg=Background
r0 c74..78 fg=Muted bg=Background
r0 c78..86 fg=Foreground bg=Background
r0 c86..88 bg=Background
r0 c88..92 fg=Muted bg=Background
r0 c92..100 fg=Foreground bg=Background
r0 c100..120 bg=Background
r1 c0..120 bg=Background
r2 c0..15 fg=Foreground bg=Background bold
r2 c15..120 bg=Background
r3 c0..6 fg=Foreground bg=Background
r3 c6..8 bg=Background
r3 c8..106 fg=Accent bg=Background
r3 c106..108 bg=Background
r3 c108..120 fg=Muted bg=Background
r4 c0..6 fg=Foreground bg=Background
r4 c6..8 bg=Background
r4 c8..71 fg=Accent bg=Background
r4 c71..108 bg=Background
r4 c108..119 fg=Muted bg=Background
r4 c119..120 bg=Background
r5 c0..6 fg=Foreground bg=Background
r5 c6..8 bg=Background
r5 c8..33 fg=Accent bg=Background
r5 c33..108 bg=Background
r5 c108..119 fg=Muted bg=Background
r5 c119..120 bg=Background
r6 c0..6 fg=Foreground bg=Background
r6 c6..108 bg=Background
r6 c108..116 fg=Muted bg=Background
r6 c116..120 bg=Background
r7 c0..7 fg=Foreground bg=Background bold
r7 c7..9 bg=Background
r7 c9..13 fg=Muted bg=Background
r7 c13..20 fg=Foreground bg=Background
r7 c20..22 bg=Background
r7 c22..26 fg=Muted bg=Background
r7 c26..33 fg=Foreground bg=Background
r7 c33..35 bg=Background
r7 c35..39 fg=Muted bg=Background
r7 c39..46 fg=Foreground bg=Background
r7 c46..120 bg=Background
r8 c0..8 fg=Muted bg=Background
r8 c8..9 fg=Dim bg=Background
r8 c9..16 bg=Background
r8 c16..40 fg=Accent bg=Background
r8 c40..89 bg=Background
r8 c89..91 fg=Accent bg=Background
r8 c91..92 bg=Background
r8 c92..112 fg=Accent bg=Background
r8 c112..120 bg=Background
r9 c0..8 fg=Muted bg=Background
r9 c8..9 fg=Dim bg=Background
r9 c9..10 bg=Background
r9 c10..17 fg=Accent bg=Background
r9 c17..40 bg=Background
r9 c40..45 fg=Accent bg=Background
r9 c45..83 bg=Background
r9 c83..89 fg=Accent bg=Background
r9 c89..90 bg=Background
r9 c90..92 fg=Accent bg=Background
r9 c92..112 bg=Background
r9 c112..120 fg=Accent bg=Background
r10 c0..8 fg=Muted bg=Background
r10 c8..9 fg=Dim bg=Background
r10 c9..11 fg=Accent bg=Background
r10 c11..45 bg=Background
r10 c45..54 fg=Accent bg=Background
r10 c54..77 bg=Background
r10 c77..83 fg=Accent bg=Background
r10 c83..120 bg=Background
r11 c0..8 fg=Muted bg=Background
r11 c8..9 fg=Dim bg=Background
r11 c9..54 bg=Background
r11 c54..78 fg=Accent bg=Background
r11 c78..120 bg=Background
== no-color · 40x12
Requests ▁▃▅▄▅▆▇███▆▅▄▃▂▁ now 45.6 rpm
Tokens by model
opus ██████████████████ 12.4k tokens
sonnet ███████████▌ 7900 tokens
haiku ████▌ 3100 tokens
cache 0 tokens
Latency now 45.6 ms
69.7 ms │ ⡠⠒⠑⠒⠒⠢⣀ ⡀⡠⠔⠉⠉⢢⡀
│⢀⠔⠁ ⢇ ⢀⠔⠉ ⠈⠑⢄
│⠁ ⠑⠢⡀ ⡠⠊
17.2 ms │ ⠈⠱⣀⣀⠤⠊⠊
== no-color · 80x12
Requests ▂▁▁▁▁▁▂▂▂▃▃▄▅▆▆▆▇▇████▇▇▆▆▅▅ now 45.6 rpm min 17.2 rpm max 69.7 rpm
Tokens by model
opus ██████████████████████████████████████████████████████████ 12.4k tokens
sonnet █████████████████████████████████████ 7900 tokens
haiku ██████████████▌ 3100 tokens
cache 0 tokens
Latency now 45.6 ms min 17.2 ms max 69.7 ms
69.7 ms │ ⣀⠤⠒⠊⠑⠒⠒⠒⠒⠒⠒⠢⠤⢄⣀ ⣀ ⣀⡠⠤⠒⠒⠉⠉⠉⠉⠢⢄⣀⣀
│ ⣀⠤⠒⠉ ⠑⠤⣀ ⢀⡠⠒⠉ ⠉ ⠉⠑⠒⠤⣀
│⠉ ⠉⠑⠒⠤⢄⣀ ⡠⠔⠊⠁
17.2 ms │ ⠉⠑⢄⣀⣀⣀⣀⣀⠤⠤⠒⠊⠒⠊⠉
== no-color · 120x12
Requests ▄▅▆▆▇██▇██▇▇▇▇▅▅▄▄▃▃▂▁▁▁▁▁▂▂▂▃▃▄▅▆▆▆▇▇████▇▇▆▆▅▅ now 45.6 rpm min 17.2 rpm max 69.7 rpm
Tokens by model
opus ██████████████████████████████████████████████████████████████████████████████████████████████████ 12.4k tokens
sonnet ██████████████████████████████████████████████████████████████▍ 7900 tokens
haiku ████████████████████████▌ 3100 tokens
cache 0 tokens
Latency now 45.6 ms min 17.2 ms max 69.7 ms
69.7 ms │ ⢀⣀⠤⠤⠒⠒⠊⠉⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠢⠤⠤⠤⣀⣀ ⣀⡀ ⣀⣀⠤⠤⠒⠒⠒⠊⠉⠉⠉⠉⠉⠉⠒⠤⢄⣀⣀⣀
│ ⢀⣀⠤⠔⠒⠉⠁ ⠉⠢⠤⣀⣀ ⣀⡠⠤⠒⠊⠉ ⠈⠉ ⠉⠉⠑⠒⠒⠤⠤⣀
│⠉⠁ ⠉⠉⠒⠒⠢⠤⢄⣀⣀ ⢀⣀⠤⠔⠊⠉
17.2 ms │ ⠉⠉⠑⠢⢄⣀⣀⣀⣀⣀⣀⣀⡠⠤⠤⠤⠒⠒⠊⠑⠒⠒⠉⠁
== ascii · 40x12
Requests _.---=####=--.__ now 45.6 rpm
Tokens by model
opus ################## 12.4k tokens
sonnet ############ 7900 tokens
haiku ##### 3100 tokens
cache 0 tokens
Latency now 45.6 ms
69.7 ms | ....... .....:.
|... : ... ...
|. ... ..
17.2 ms | .:.....
== ascii · 80x12
Requests _________..--===########==-- now 45.6 rpm min 17.2 rpm max 69.7 rpm
Tokens by model
opus ########################################################## 12.4k tokens
sonnet ##################################### 7900 tokens
haiku ############### 3100 tokens
cache 0 tokens
Latency now 45.6 ms min 17.2 ms max 69.7 ms
69.7 ms | ............... . .............
| .... ... .... . .....
|. ...... ....
17.2 ms | ...............
== ascii · 120x12
Requests --==##########----.._________..--===########==-- now 45.6 rpm min 17.2 rpm max 69.7 rpm
Tokens by model
opus ################################################################################################## 12.4k tokens
sonnet ############################################################### 7900 tokens
haiku ######################### 3100 tokens
cache 0 tokens
Latency now 45.6 ms min 17.2 ms max 69.7 ms
69.7 ms | ........................ .. ....................
| ....... ..... ...... .. ........
|.. ......... ......
17.2 ms | ........................
-118
View File
@@ -1,118 +0,0 @@
//! Gallery: data displays fully drawn, mid draw-in, empty and from agent JSON.
use ratatui::{buffer::Buffer, layout::Rect};
use super::Entry;
use crate::{DataBarItem, DataBars, DataDisplay, DataSparkline, DataTrend, Paint, Theme};
fn series() -> Vec<f64> {
(0..64)
.map(|i| 120.0 + 60.0 * (f64::from(i) / 6.0).sin() + f64::from((i * 7) % 11) * 3.0)
.collect()
}
fn items() -> Vec<DataBarItem> {
vec![
DataBarItem::new("engine", 412.0),
DataBarItem::new("tui", 268.0),
DataBarItem::new("runtime", 131.0),
DataBarItem::new("protocol", 44.0),
]
}
fn row(area: Rect, y: u16, h: u16) -> Rect {
Rect::new(area.x, area.y.saturating_add(y), area.width, h).intersection(area)
}
fn sparklines(area: Rect, buf: &mut Buffer, theme: &Theme) {
let values = series();
DataSparkline::new("Requests", &values)
.unit("rpm")
.paint(row(area, 0, 1), buf, theme);
DataSparkline::new("Requests", &values)
.unit("rpm")
.drawn(0.45)
.paint(row(area, 1, 1), buf, theme);
DataSparkline::new("Requests", &[]).paint(row(area, 2, 1), buf, theme);
}
fn bars(area: Rect, buf: &mut Buffer, theme: &Theme) {
let items = items();
DataBars::new("Tests by crate", &items).paint(row(area, 0, 5), buf, theme);
}
fn bars_growing(area: Rect, buf: &mut Buffer, theme: &Theme) {
let items = items();
DataBars::new("Tests by crate", &items)
.previous(&[300.0, 300.0, 300.0, 300.0])
.drawn(0.5)
.paint(row(area, 0, 5), buf, theme);
}
fn trend(area: Rect, buf: &mut Buffer, theme: &Theme) {
let values = series();
DataTrend::new("Latency", &values)
.unit("ms")
.rows(7)
.paint(area, buf, theme);
}
fn trend_drawing(area: Rect, buf: &mut Buffer, theme: &Theme) {
let values = series();
DataTrend::new("Latency", &values)
.unit("ms")
.rows(7)
.drawn(0.6)
.paint(area, buf, theme);
}
fn from_agent(area: Rect, buf: &mut Buffer, theme: &Theme) {
let shown = DataDisplay::from_json(
r#"{"kind":"bars","label":"Tokens this turn","unit":"tokens",
"items":[{"label":"read","value":18200},{"label":"wrote","value":2400},
{"label":"cached","value":61000}]}"#,
)
.expect("gallery fixture is a valid data display");
shown.view().paint(area, buf, theme);
}
pub(crate) fn entries() -> Vec<Entry> {
vec![
Entry {
name: "data-sparkline",
width: 80,
height: 3,
draw: sparklines,
},
Entry {
name: "data-bars",
width: 60,
height: 5,
draw: bars,
},
Entry {
name: "data-bars-growing",
width: 60,
height: 5,
draw: bars_growing,
},
Entry {
name: "data-trend",
width: 72,
height: 7,
draw: trend,
},
Entry {
name: "data-trend-drawing",
width: 72,
height: 7,
draw: trend_drawing,
},
Entry {
name: "data-from-agent",
width: 60,
height: 4,
draw: from_agent,
},
]
}
-2
View File
@@ -31,7 +31,6 @@ mod approval;
mod artifacts;
mod atmosphere;
mod attention;
mod chart;
mod chrome;
mod display;
mod habitat;
@@ -105,7 +104,6 @@ pub fn entries() -> Vec<Entry> {
lists::entries(),
chrome::entries(),
display::entries(),
chart::entries(),
approval::entries(),
motion::entries(),
workspace::entries(),
+2 -2
View File
@@ -259,8 +259,8 @@
},
{
"path": "docs/TELEMETRY.md",
"sha256": "0ac65dbd0e1d9d196d1c4421116852c3ab6a855062c375c25b09db993084c563",
"content": "# Codewhale product telemetry\n\n> 阅读简体中文版:[zh_hans/TELEMETRY.md](zh_hans/TELEMETRY.md)\n\n**Usage analytics are on by default in the current source.** The first\ninteractive launch gives a localized, nonblocking disclosure naming **Codewhale\nand PostHog**, with a direct route to turn counting off. Notice version `6`\ndescribes this policy; displaying it is not recorded as human acceptance.\nMissing an older acceptance record does not turn a fresh installation off.\nExisting explicit opt-outs remain off across upgrades, and unreadable privacy\nstate fails closed. Environment and CLI kill switches still take precedence.\n\nRead the disclosure with `codewhale config telemetry`. Turn usage off in\n`/settings` or run `codewhale config set telemetry false`. Turning it back on\nin Settings or with `codewhale config set telemetry true` updates the existing\ndurable preference and privacy records for new sessions. The compatibility\ncommand `codewhale config telemetry --accept-notice 5` remains available, but\nis not required for a fresh installation. Collection never includes work content or credentials.\n\nThe PostHog sink is optional and inert until the ingest operator explicitly\nconfigures a project token and approved regional host, and records staging\nproof that the actual egress path does not forward the original client IP.\nSource readiness does not establish a deployment, an activated processor,\nan egress receipt, or a retention setting.\n\nCodewhale does not collect conversations, code, prompts, files, file/repo/branch\nnames, model content, or credentials. It sends no per-turn or per-tool timeline.\nIt does send the closed, aggregate schema below: version and platform classes,\nsession duration/outcome, feature/error counters, and a random install id that\nrotates every 90 days.\n\n**There is now a real endpoint.** An enabled session sends its batches to the\nfirst-party ingest service at `https://telemetry.codewhale.net/v1/telemetry`,\nwhich is the shipped default for `telemetry_endpoint`. What that service is,\nwhat it stores, and what it structurally cannot store is spelled out in \"What\nthe endpoint does\" below.\n\n**To send nothing anywhere, turn telemetry off** (see \"Turning it off\"). To stay\nenabled but contact nobody, set `telemetry_endpoint = \"\"`: batches are then\nappended to `$CODEWHALE_HOME/telemetry/dryrun.jsonl` on your own machine, byte\nfor byte what the server would have received, and no HTTP client is ever\nconstructed. That file is how you audit this document against reality.\n\nThis document is the schema. It is not a summary of the schema: a test in\n`crates/telemetry` parses the field names out of this file and asserts set\nequality against the structs the serializer actually uses, so a field that is\nhere and not in the code — or in the code and not here — fails the build.\n\nLocal saved sessions and `codewhale metrics` remain available with telemetry\ndisabled and require no account or hosted reporting service. Disabling telemetry\ndoes not disable local usage receipts.\n\n## Turning it off\n\nThere are two off switches and they do different things. Both stop collection\ncompletely; only one of them erases anything.\n\n```sh\ncodewhale config set telemetry false # opt out: stops collection and erases state\nCODEWHALE_TELEMETRY=0 codewhale # kill switch: stops collection, erases nothing\nDO_NOT_TRACK=1 codewhale # the same kill switch, by the common convention\ncodewhale --telemetry false # the same kill switch, for one command\n```\n\n**`telemetry = false` in the config file is the opt-out.** It is a floor:\n`--telemetry true` and `CODEWHALE_TELEMETRY=1` both lose to it, because a\nsetting you can undo by accident from a wrapper script is not a setting. It\ndeletes the random install id, truncates every buffered event and every dry-run\nrecord, and writes a tombstone. Appends, identity/state writes, and delivery all\nshare the wipe's ordering lock, so once opt-out returns no pre-opt-out write or\nPOST remains in flight. If any part of that wipe fails, the tombstone is still\nthere and the buffer is undrainable — a failed wipe fails closed. Every later\nrun re-asserts the same tombstone for as long as the setting stands, so it\nsurvives. Turning telemetry back on in `/settings` records that explicit preference\nin both existing privacy registers; a fresh launch can then clear the tombstone. Nothing buffered before that point is\never sent.\n\n**The environment variable and the flag are kill switches, not opt-outs.**\nTelemetry is off for the run, nothing is written, nothing is sent — and\nnothing on disk is touched or deleted. That is deliberate: a harness or agent\nthat sets `CODEWHALE_TELEMETRY=0` for one command must not silently discard the\ninstall id and the dry-run records of the person who owns the machine. If you\nwant the erasing kind, use the config file.\n\n`CODEWHALE_TELEMELine truncated
"sha256": "6ecccb66215a6aae5076cd00735028c10a50ce6e5902ce4151a9606ee86696a9",
"content": "# Codewhale product telemetry\n\n> 阅读简体中文版:[zh_hans/TELEMETRY.md](zh_hans/TELEMETRY.md)\n\n**Usage analytics are on by default in the current source.** The first\ninteractive launch gives a localized, nonblocking disclosure naming **Codewhale\nand PostHog**, with a direct route to turn counting off. Notice version `6`\ndescribes this policy; displaying it is not recorded as human acceptance.\nMissing an older acceptance record does not turn a fresh installation off.\nExisting explicit opt-outs remain off across upgrades, and unreadable privacy\nstate fails closed. Environment and CLI kill switches still take precedence.\n\nRead the disclosure with `codewhale config telemetry`. Turn usage off in\n`/settings` or run `codewhale config set telemetry false`. Turning it back on\nin Settings or with `codewhale config set telemetry true` updates the existing\ndurable preference and privacy records for new sessions. The compatibility\ncommand `codewhale config telemetry --accept-notice 5` remains available, but\nis not required for a fresh installation. Collection never includes work content or credentials.\n\nThe PostHog sink is optional and inert until the ingest operator explicitly\nconfigures a project token and approved regional host, and records staging\nproof that the actual egress path does not forward the original client IP.\nSource readiness does not establish a deployment, an activated processor,\nan egress receipt, or a retention setting.\n\nCodewhale does not collect conversations, code, prompts, files, file/repo/branch\nnames, model content, or credentials. It sends no per-turn or per-tool timeline.\nIt does send the closed, aggregate schema below: version and platform classes,\nsession duration/outcome, feature/error counters, and a random install id that\nrotates every 90 days.\n\n**There is now a real endpoint.** An enabled session sends its batches to the\nfirst-party ingest service at `https://telemetry.codewhale.net/v1/telemetry`,\nwhich is the shipped default for `telemetry_endpoint`. What that service is,\nwhat it stores, and what it structurally cannot store is spelled out in \"What\nthe endpoint does\" below.\n\n**To send nothing anywhere, turn telemetry off** (see \"Turning it off\"). To stay\nenabled but contact nobody, set `telemetry_endpoint = \"\"`: batches are then\nappended to `$CODEWHALE_HOME/telemetry/dryrun.jsonl` on your own machine, byte\nfor byte what the server would have received, and no HTTP client is ever\nconstructed. That file is how you audit this document against reality.\n\nThis document is the schema. It is not a summary of the schema: a test in\n`crates/telemetry` parses the field names out of this file and asserts set\nequality against the structs the serializer actually uses, so a field that is\nhere and not in the code — or in the code and not here — fails the build.\n\nLocal saved sessions and `codewhale metrics` remain available with telemetry\ndisabled and require no account or hosted reporting service. Disabling telemetry\ndoes not disable local usage receipts.\n\n## Turning it off\n\nThere are two off switches and they do different things. Both stop collection\ncompletely; only one of them erases anything.\n\n```sh\ncodewhale config set telemetry false # opt out: stops collection and erases state\nCODEWHALE_TELEMETRY=0 codewhale # kill switch: stops collection, erases nothing\nDO_NOT_TRACK=1 codewhale # the same kill switch, by the common convention\ncodewhale --telemetry false # the same kill switch, for one command\n```\n\n**`telemetry = false` in the config file is the opt-out.** It is a floor:\n`--telemetry true` and `CODEWHALE_TELEMETRY=1` both lose to it, because a\nsetting you can undo by accident from a wrapper script is not a setting. It\ndeletes the random install id, deletes every spool, truncates every dry-run\nrecord, and writes a tombstone. Appends, identity/state writes, and delivery all\nshare the wipe's ordering lock, so once opt-out returns no pre-opt-out write or\nPOST remains in flight. If any part of that wipe fails, the tombstone is still\nthere and no spool can be sent — a failed wipe fails closed. Every later\nrun re-asserts the same tombstone for as long as the setting stands, so it\nsurvives. Turning telemetry back on in `/settings` records that explicit preference\nin both existing privacy registers; a fresh launch can then clear the tombstone. Nothing buffered before that point is\never sent.\n\n**The environment variable and the flag are kill switches, not opt-outs.**\nTelemetry is off for the run, nothing is written, nothing is sent — and\nnothing on disk is touched or deleted. That is deliberate: a harness or agent\nthat sets `CODEWHALE_TELEMETRY=0` for one command must not silently discard the\ninstall id and the dry-run records of the person who owns the machine. If you\nwant the erasing kind, use the config file.\n\n`CODEWHALE_TELEMETRY` (andLine truncated
},
{
"path": "docs/public-surface-facts.json",
+2 -1
View File
@@ -35,6 +35,7 @@ export const RELEASE_CONTRIBUTORS: string[] = [
"@luochen211",
"@gannp",
"@Felixkw12",
"@MannXo",
];
/**
@@ -42,7 +43,7 @@ export const RELEASE_CONTRIBUTORS: string[] = [
* one. scripts/check-contributor-credit.py requires them here now; the release
* cut moves them into RELEASE_CONTRIBUTORS with that release's changelog block.
*/
export const UNRELEASED_CONTRIBUTORS: string[] = ["@luochen211", "@Lstarsky0", "@SamhandsomeLee", "@gaord", "@MannXo"];
export const UNRELEASED_CONTRIBUTORS: string[] = ["@luochen211", "@Lstarsky0", "@SamhandsomeLee", "@gaord"];
/**
* Contributors who helped with reports, reproductions, and verification.