mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-09 17:10:44 -10:00
fix(security): redact tool output as it enters the transcript; doctor and scrub for stored secrets
Session JSON stored tool output verbatim: redaction ran only when a model request was built (client.rs), so a `cat ~/.codex/auth.json` or a printed bearer token was written to ~/.codewhale/sessions live (sweep B1). - The engine masks credentials in ToolResult content (and text content_blocks) once, in add_session_message, with the same masking the request boundary applies. The transcript is what session files, checkpoints and later requests are built from. The double-confirmed `[redaction] model_bound = "disabled"` opt-out is honored the same way. - Runtime API threads mask tool output before it reaches the durable tool item and the event log built from it (always on: receipts fan out to UI clients). - `codewhale doctor` gains a "Stored Sessions" finding: it scans the newest 50 session/checkpoint files and says how many still hold credentials in stored tool output. It never rewrites anything. - `codewhale sessions scrub-secrets` reports every affected file; `--apply` rewrites them atomically, touching only tool_result text. Documented under docs/CONFIGURATION.md "Stored sessions", with the reminder that masking a copy does not un-leak a credential: rotate it. Not covered: runtime item/event files written by older builds are not scrubbed by the command (it walks the sessions store only). Tests: codewhale-tui core::engine::tests::tool_output_credentials_are_redacted_when_they_enter_the_transcript, session_secret_scrub (2), runtime_threads::tests::runtime_tool_items_store_tool_output_with_credentials_masked all pass (in the 638/1 focused run above; the runtime fixture failed once on a single-line JSON fixture and passes 1/0 after the fix); runtime_threads:: 226 passed / 0 failed / 2 ignored. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01R9rJEoMuUSRWznU6QjkE7h
This commit is contained in:
1 parent
9253eff1a6
commit
000ab6b65a
8 files changed
+519
-4
No files matched your search
@@ -1799,6 +1799,18 @@ impl CodewhaleClient {
|
||||
redact_model_bound_text(text, &self.model_bound_secret_values)
|
||||
}
|
||||
|
||||
/// Redact tool output as it enters the transcript. Same masking as the
|
||||
/// request boundary, including the confirmed `[redaction] model_bound`
|
||||
/// opt-out: a user who chose to let the model see file bytes verbatim
|
||||
/// keeps that, and everyone else never stores a live credential.
|
||||
pub(crate) fn redact_tool_output_for_transcript(&self, text: &str) -> String {
|
||||
if self.model_bound_masking {
|
||||
redact_model_bound_text(text, &self.model_bound_secret_values)
|
||||
} else {
|
||||
text.to_string()
|
||||
}
|
||||
}
|
||||
|
||||
/// Alternate models share this client's frozen endpoint and declarations.
|
||||
/// Resolution still owns protocol admission, including closed rosters.
|
||||
pub(crate) fn resolve_model_route(&self, model: &str) -> Result<ReadyRouteCandidate> {
|
||||
|
||||
@@ -3738,11 +3738,45 @@ impl Engine {
|
||||
Some(format!("Active goal token budget: {token_budget}"))
|
||||
}
|
||||
|
||||
async fn add_session_message(&mut self, message: Message) {
|
||||
async fn add_session_message(&mut self, mut message: Message) {
|
||||
self.redact_tool_results_for_transcript(&mut message);
|
||||
self.session.add_message(message);
|
||||
self.emit_session_updated().await;
|
||||
}
|
||||
|
||||
/// Scrub credentials from tool output once, as it enters the transcript
|
||||
/// (B1). The transcript is what session JSON, the journal and every
|
||||
/// later request are built from, so a token a tool printed is never
|
||||
/// written to disk live. Honors the confirmed `[redaction] model_bound`
|
||||
/// opt-out the same way the request boundary does.
|
||||
fn redact_tool_results_for_transcript(&self, message: &mut Message) {
|
||||
for block in &mut message.content {
|
||||
let ContentBlock::ToolResult {
|
||||
content,
|
||||
content_blocks,
|
||||
..
|
||||
} = block
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
*content = self.redact_tool_output_for_transcript(content);
|
||||
for value in content_blocks.iter_mut().flatten() {
|
||||
if value.get("type").and_then(serde_json::Value::as_str) == Some("text")
|
||||
&& let Some(serde_json::Value::String(text)) = value.get_mut("text")
|
||||
{
|
||||
*text = self.redact_tool_output_for_transcript(text);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn redact_tool_output_for_transcript(&self, text: &str) -> String {
|
||||
match self.codewhale_client.as_ref() {
|
||||
Some(client) => client.redact_tool_output_for_transcript(text),
|
||||
None => codewhale_config::persistence::redact_model_bound_secrets(text),
|
||||
}
|
||||
}
|
||||
|
||||
async fn add_interrupted_assistant_text(&mut self, text: &str) {
|
||||
if text.is_empty() {
|
||||
return;
|
||||
|
||||
@@ -5631,6 +5631,56 @@ async fn run_budgeted_read_turn(
|
||||
(status, error, completions)
|
||||
}
|
||||
|
||||
/// B1: tool output is redacted once, as it enters the transcript, so the
|
||||
/// session messages (and the session JSON built from them) never hold a live
|
||||
/// credential a tool printed.
|
||||
#[tokio::test]
|
||||
async fn tool_output_credentials_are_redacted_when_they_enter_the_transcript() {
|
||||
use crate::llm_client::mock::{MockLlmClient, canned};
|
||||
|
||||
const TOKEN: &str = "sk-ant-oat01-AbCdEfGhIjKlMnOpQrStUvWxYz0123456789abcdefghij";
|
||||
let workspace = tempdir().expect("tempdir");
|
||||
fs::write(
|
||||
workspace.path().join("auth.json"),
|
||||
format!("{{\n \"access_token\": \"{TOKEN}\",\n \"note\": \"keep me\"\n}}\n"),
|
||||
)
|
||||
.expect("write fixture");
|
||||
let mock = std::sync::Arc::new(MockLlmClient::new(vec![
|
||||
canned::tool_call_turn("call-read", "read_file", r#"{"path":"auth.json"}"#),
|
||||
canned::simple_text_turn("done"),
|
||||
]));
|
||||
let client: crate::core::model_client::SharedModelClient = mock;
|
||||
let (mut engine, _handle) = Engine::new_with_model_client(
|
||||
deterministic_engine_config(workspace.path()),
|
||||
&Config::default(),
|
||||
client,
|
||||
);
|
||||
let context = crate::tools::ToolContext::new(workspace.path().to_path_buf());
|
||||
let mut registry = crate::tools::ToolRegistry::new(context);
|
||||
registry.register(std::sync::Arc::new(crate::tools::file::ReadFileTool));
|
||||
let tools = Some(registry.to_api_tools_with_cache(true));
|
||||
let surface = test_tool_surface(&engine, registry, tools, AppMode::Agent);
|
||||
let mut turn = crate::core::turn::TurnContext::new(4);
|
||||
let (status, error) = engine.run_turn(&mut turn, surface, None, None).await;
|
||||
assert_eq!(status, TurnOutcomeStatus::Completed, "{error:?}");
|
||||
|
||||
let stored = engine
|
||||
.session
|
||||
.messages
|
||||
.iter()
|
||||
.flat_map(|message| message.content.iter())
|
||||
.find_map(|block| match block {
|
||||
ContentBlock::ToolResult { content, .. } => Some(content.clone()),
|
||||
_ => None,
|
||||
})
|
||||
.expect("the read result is in the transcript");
|
||||
assert!(!stored.contains(TOKEN), "{stored}");
|
||||
assert!(stored.contains("keep me"), "ordinary bytes stay: {stored}");
|
||||
let serialized = serde_json::to_string(&engine.session.messages.iter().collect::<Vec<_>>())
|
||||
.expect("serialize");
|
||||
assert!(!serialized.contains(TOKEN));
|
||||
}
|
||||
|
||||
/// #4415 AC(a): an 8-call cap admits exactly 8 calls; the 9th is rejected
|
||||
/// with the typed reason carrying `remaining=0` and is never executed.
|
||||
#[tokio::test]
|
||||
|
||||
@@ -128,6 +128,7 @@ mod session_manager;
|
||||
mod session_peek;
|
||||
mod session_projection;
|
||||
mod session_resume;
|
||||
mod session_secret_scrub;
|
||||
pub mod session_tree;
|
||||
mod settings;
|
||||
mod shell_dispatcher;
|
||||
@@ -399,6 +400,14 @@ enum SessionsCommand {
|
||||
#[arg(short, long)]
|
||||
search: Option<String>,
|
||||
},
|
||||
/// Mask credentials that older builds stored in saved sessions' tool
|
||||
/// output. Reports what it would change unless `--apply` is given. Run it
|
||||
/// while no Codewhale session is open.
|
||||
ScrubSecrets {
|
||||
/// Rewrite the affected session files (default: report only)
|
||||
#[arg(long, default_value_t = false)]
|
||||
apply: bool,
|
||||
},
|
||||
/// Export a session as a full-fidelity tar.xz archive (complete context:
|
||||
/// system prompt, messages, tool calls and results, plus artifacts)
|
||||
Export {
|
||||
@@ -2303,6 +2312,7 @@ async fn run_async_main_dispatch(
|
||||
} => match command {
|
||||
None => list_sessions(limit, search),
|
||||
Some(SessionsCommand::List { limit, search }) => list_sessions(limit, search),
|
||||
Some(SessionsCommand::ScrubSecrets { apply }) => run_sessions_scrub_secrets(apply),
|
||||
Some(SessionsCommand::Export {
|
||||
id,
|
||||
output,
|
||||
@@ -4611,6 +4621,7 @@ async fn run_doctor(
|
||||
(aqua_r, aqua_g, aqua_b),
|
||||
(sky_r, sky_g, sky_b),
|
||||
);
|
||||
print_doctor_stored_secrets_report().await;
|
||||
|
||||
let (setup_state, setup_source) = doctor_setup_state(config, workspace);
|
||||
print_doctor_setup_report(
|
||||
@@ -8107,6 +8118,88 @@ fn sessions_resume_command() -> &'static str {
|
||||
"codewhale resume"
|
||||
}
|
||||
|
||||
/// Newest session files `codewhale doctor` inspects for stored credentials;
|
||||
/// `codewhale sessions scrub-secrets` covers every file.
|
||||
const DOCTOR_SECRET_SCAN_FILES: usize = 50;
|
||||
|
||||
/// B1 finding: sessions written before tool output was redacted at the
|
||||
/// transcript boundary can still hold live credentials. Report, never
|
||||
/// rewrite — scrubbing is the explicit `scrub-secrets` command.
|
||||
async fn print_doctor_stored_secrets_report() {
|
||||
use colored::Colorize;
|
||||
|
||||
let scan = tokio::task::spawn_blocking(|| {
|
||||
let manager = session_manager::SessionManager::default_location().ok()?;
|
||||
let files = session_secret_scrub::session_files(manager.sessions_dir());
|
||||
let total = files.len();
|
||||
let checked: Vec<PathBuf> = files.into_iter().take(DOCTOR_SECRET_SCAN_FILES).collect();
|
||||
session_secret_scrub::scrub_files(&checked, false)
|
||||
.ok()
|
||||
.map(|report| (report, total))
|
||||
})
|
||||
.await
|
||||
.ok()
|
||||
.flatten();
|
||||
let Some((report, total)) = scan else {
|
||||
return;
|
||||
};
|
||||
println!();
|
||||
println!("{}", "Stored Sessions:".bold());
|
||||
let scope = if total > report.files_scanned {
|
||||
format!("newest {} of {total}", report.files_scanned)
|
||||
} else {
|
||||
format!("{total}")
|
||||
};
|
||||
if report.files_with_secrets.is_empty() {
|
||||
println!(" ✓ no credentials found in stored tool output ({scope} session files)");
|
||||
return;
|
||||
}
|
||||
println!(
|
||||
" ✗ {} session files hold credentials in stored tool output ({scope} checked)",
|
||||
report.files_with_secrets.len()
|
||||
);
|
||||
println!(
|
||||
" fix: `{}` to review, then `--apply` to mask them; rotate any exposed credential",
|
||||
session_secret_scrub::SCRUB_COMMAND
|
||||
);
|
||||
}
|
||||
|
||||
fn run_sessions_scrub_secrets(apply: bool) -> Result<()> {
|
||||
let manager = session_manager::SessionManager::default_location()?;
|
||||
let files = session_secret_scrub::session_files(manager.sessions_dir());
|
||||
let report = session_secret_scrub::scrub_files(&files, apply)?;
|
||||
let affected = report.files_with_secrets.len();
|
||||
if affected == 0 {
|
||||
println!(
|
||||
"No stored credentials found in tool output across {} session files.",
|
||||
report.files_scanned
|
||||
);
|
||||
} else {
|
||||
let verb = if apply { "Scrubbed" } else { "Found" };
|
||||
println!(
|
||||
"{verb} {} credential-bearing tool results in {affected} of {} session files:",
|
||||
report.tool_results_with_secrets, report.files_scanned
|
||||
);
|
||||
for path in &report.files_with_secrets {
|
||||
println!(" {}", path.display());
|
||||
}
|
||||
if !apply {
|
||||
println!(
|
||||
"Re-run with `{} --apply` to mask them (close open Codewhale sessions first). \
|
||||
Rotate any credential that was exposed: redaction cannot un-leak it.",
|
||||
session_secret_scrub::SCRUB_COMMAND
|
||||
);
|
||||
}
|
||||
}
|
||||
if !report.unreadable.is_empty() {
|
||||
println!(
|
||||
"{} files could not be read or parsed and were left untouched.",
|
||||
report.unreadable.len()
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn list_sessions(limit: usize, search: Option<String>) -> Result<()> {
|
||||
use codewhale_palette as palette;
|
||||
use colored::Colorize;
|
||||
|
||||
@@ -12531,11 +12531,18 @@ impl RuntimeThreadManager {
|
||||
"response_redacted": true,
|
||||
}));
|
||||
} else {
|
||||
// Durable receipt: credentials a tool
|
||||
// printed are masked before they reach
|
||||
// the item store or the event log (B1).
|
||||
let content =
|
||||
codewhale_config::persistence::redact_model_bound_secrets(
|
||||
&output.content,
|
||||
);
|
||||
item.summary = summarize_text(
|
||||
&format!("{name}: {}", output.content),
|
||||
&format!("{name}: {content}"),
|
||||
SUMMARY_LIMIT,
|
||||
);
|
||||
item.detail = Some(output.content.clone());
|
||||
item.detail = Some(content);
|
||||
// `detail` is now the tool output, so the
|
||||
// call identity persisted at start must be
|
||||
// carried through metadata. Mark the
|
||||
@@ -12580,9 +12587,12 @@ impl RuntimeThreadManager {
|
||||
}
|
||||
Err(err) => {
|
||||
item.status = TurnItemLifecycleStatus::Failed;
|
||||
let err = codewhale_config::persistence::redact_model_bound_secrets(
|
||||
&err.to_string(),
|
||||
);
|
||||
item.summary =
|
||||
summarize_text(&format!("{name} failed: {err}"), SUMMARY_LIMIT);
|
||||
item.detail = Some(err.to_string());
|
||||
item.detail = Some(err);
|
||||
}
|
||||
}
|
||||
self.store.save_item(&item)?;
|
||||
|
||||
@@ -18129,3 +18129,87 @@ async fn canonical_sessions_root_is_resolved_off_the_ui_runtime_and_cached() ->
|
||||
assert_ne!(cached, sessions, "the fixture spells the root two ways");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// B1: a credential a tool printed is masked in the durable tool item (and
|
||||
/// the event log built from it) on Runtime API threads, as it is in the
|
||||
/// engine transcript.
|
||||
#[tokio::test]
|
||||
async fn runtime_tool_items_store_tool_output_with_credentials_masked() -> Result<()> {
|
||||
const TOKEN: &str = "sk-ant-oat01-AbCdEfGhIjKlMnOpQrStUvWxYz0123456789abcdefghij";
|
||||
let manager = test_manager(test_runtime_dir())?;
|
||||
let thread = manager
|
||||
.create_thread(CreateThreadRequest {
|
||||
trust_mode: Some(true),
|
||||
auto_approve: Some(true),
|
||||
..Default::default()
|
||||
})
|
||||
.await?;
|
||||
let mut harness = install_mock_engine(&manager, &thread.id).await;
|
||||
let turn = manager
|
||||
.start_turn(
|
||||
&thread.id,
|
||||
StartTurnRequest {
|
||||
prompt: "print the auth file".to_string(),
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
assert!(matches!(
|
||||
harness.rx_op.recv().await,
|
||||
Some(Op::SendMessage(TurnSpec { .. }))
|
||||
));
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
})
|
||||
.await?;
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::ToolCallStarted {
|
||||
id: "tool-cat-auth".to_string(),
|
||||
name: "exec_command".to_string(),
|
||||
input: json!({"cmd": "cat auth.json"}),
|
||||
})
|
||||
.await?;
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::ToolCallComplete {
|
||||
id: "tool-cat-auth".to_string(),
|
||||
name: "exec_command".to_string(),
|
||||
result: Ok(crate::tools::spec::ToolResult::success(format!(
|
||||
"{{\n \"access_token\": \"{TOKEN}\",\n \"note\": \"keep me\"\n}}\n"
|
||||
))),
|
||||
})
|
||||
.await?;
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnComplete {
|
||||
usage: Usage::default(),
|
||||
parent_route_usage: Usage::default(),
|
||||
routed_usage_dropped_records: 0,
|
||||
status: TurnOutcomeStatus::Completed,
|
||||
error: None,
|
||||
tool_catalog: None,
|
||||
base_url: None,
|
||||
})
|
||||
.await?;
|
||||
wait_for_terminal_turn(&manager, &turn.id).await?;
|
||||
|
||||
let items = manager.store.list_items_for_turn(&turn.id)?;
|
||||
let tool_item = items
|
||||
.iter()
|
||||
.find(|item| {
|
||||
item.detail
|
||||
.as_deref()
|
||||
.is_some_and(|d| d.contains("keep me"))
|
||||
})
|
||||
.context("the tool item keeps its ordinary output")?;
|
||||
let stored = serde_json::to_string(tool_item)?;
|
||||
assert!(!stored.contains(TOKEN), "{stored}");
|
||||
let events = serde_json::to_string(&manager.events_since(&thread.id, None)?)?;
|
||||
assert!(!events.contains(TOKEN), "the event log holds no live token");
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
//! Find and scrub credentials that older builds stored in saved sessions.
|
||||
//!
|
||||
//! Tool output is redacted as it enters the transcript now (B1), so new
|
||||
//! session files never hold a live token. Sessions written before that fix
|
||||
//! can: the redaction used to run only when a request was built, while the
|
||||
//! transcript on disk kept the raw tool output (a `cat ~/.codex/auth.json`
|
||||
//! result, a printed bearer token). Nothing rewrites those files on its own —
|
||||
//! `codewhale doctor` reports them and `codewhale sessions scrub-secrets`
|
||||
//! rewrites them on request.
|
||||
//!
|
||||
//! Only `tool_result` text is touched (in `messages`, the journal, and
|
||||
//! checkpoints), using the same credential-shaped masking the model boundary
|
||||
//! applies, so file bytes the model quoted for exact edits stay intact.
|
||||
|
||||
use std::io;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
/// The documented command that rewrites affected sessions.
|
||||
pub(crate) const SCRUB_COMMAND: &str = "codewhale sessions scrub-secrets";
|
||||
|
||||
/// What a scan found (and, when applied, rewrote).
|
||||
#[derive(Debug, Default, Clone, PartialEq, Eq)]
|
||||
pub(crate) struct ScrubReport {
|
||||
/// Session and checkpoint files examined.
|
||||
pub files_scanned: usize,
|
||||
/// Files holding at least one unredacted credential in tool output.
|
||||
pub files_with_secrets: Vec<PathBuf>,
|
||||
/// Tool-result text fields that contained a credential.
|
||||
pub tool_results_with_secrets: usize,
|
||||
/// Files the scan could not read or parse, left untouched.
|
||||
pub unreadable: Vec<PathBuf>,
|
||||
}
|
||||
|
||||
/// Session JSON files under `sessions_dir`: `<id>.json` and
|
||||
/// `checkpoints/<id>.json`, newest first.
|
||||
pub(crate) fn session_files(sessions_dir: &Path) -> Vec<PathBuf> {
|
||||
let mut files: Vec<(std::time::SystemTime, PathBuf)> = Vec::new();
|
||||
for dir in [sessions_dir.to_path_buf(), sessions_dir.join("checkpoints")] {
|
||||
let Ok(entries) = std::fs::read_dir(&dir) else {
|
||||
continue;
|
||||
};
|
||||
for entry in entries.flatten() {
|
||||
let path = entry.path();
|
||||
if path.extension().and_then(|ext| ext.to_str()) != Some("json") {
|
||||
continue;
|
||||
}
|
||||
let Ok(meta) = entry.metadata() else {
|
||||
continue;
|
||||
};
|
||||
if !meta.is_file() {
|
||||
continue;
|
||||
}
|
||||
let modified = meta.modified().unwrap_or(std::time::UNIX_EPOCH);
|
||||
files.push((modified, path));
|
||||
}
|
||||
}
|
||||
files.sort_by(|a, b| b.0.cmp(&a.0).then_with(|| a.1.cmp(&b.1)));
|
||||
files.into_iter().map(|(_, path)| path).collect()
|
||||
}
|
||||
|
||||
/// Scan `files`; with `apply`, rewrite each affected file atomically with its
|
||||
/// tool-result credentials masked.
|
||||
pub(crate) fn scrub_files(files: &[PathBuf], apply: bool) -> io::Result<ScrubReport> {
|
||||
let mut report = ScrubReport::default();
|
||||
for path in files {
|
||||
report.files_scanned += 1;
|
||||
let Ok(raw) = std::fs::read(path) else {
|
||||
report.unreadable.push(path.clone());
|
||||
continue;
|
||||
};
|
||||
let Ok(mut value) = serde_json::from_slice::<Value>(&raw) else {
|
||||
report.unreadable.push(path.clone());
|
||||
continue;
|
||||
};
|
||||
let redacted = scrub_value(&mut value);
|
||||
if redacted == 0 {
|
||||
continue;
|
||||
}
|
||||
report.tool_results_with_secrets += redacted;
|
||||
report.files_with_secrets.push(path.clone());
|
||||
if apply {
|
||||
let bytes = serde_json::to_vec_pretty(&value).map_err(io::Error::other)?;
|
||||
codewhale_config::persistence::atomic_write(path, &bytes).map_err(io::Error::other)?;
|
||||
}
|
||||
}
|
||||
Ok(report)
|
||||
}
|
||||
|
||||
/// Mask credentials in every `tool_result` text field below `value`.
|
||||
/// Returns how many fields changed.
|
||||
fn scrub_value(value: &mut Value) -> usize {
|
||||
match value {
|
||||
Value::Object(map) => {
|
||||
let mut changed = 0;
|
||||
if map.get("type").and_then(Value::as_str) == Some("tool_result") {
|
||||
if let Some(Value::String(content)) = map.get_mut("content") {
|
||||
changed += usize::from(scrub_text(content));
|
||||
}
|
||||
if let Some(Value::Array(blocks)) = map.get_mut("content_blocks") {
|
||||
for block in blocks {
|
||||
if block.get("type").and_then(Value::as_str) == Some("text")
|
||||
&& let Some(Value::String(text)) = block.get_mut("text")
|
||||
{
|
||||
changed += usize::from(scrub_text(text));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// Strings below are only rewritten through a tool_result above,
|
||||
// so walking the rest (including this object's own fields) never
|
||||
// double-counts.
|
||||
changed + map.values_mut().map(scrub_value).sum::<usize>()
|
||||
}
|
||||
Value::Array(items) => items.iter_mut().map(scrub_value).sum(),
|
||||
_ => 0,
|
||||
}
|
||||
}
|
||||
|
||||
fn scrub_text(text: &mut String) -> bool {
|
||||
let redacted = codewhale_config::persistence::redact_model_bound_secrets(text);
|
||||
if redacted == *text {
|
||||
return false;
|
||||
}
|
||||
*text = redacted;
|
||||
true
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
const TOKEN: &str = "sk-ant-oat01-AbCdEfGhIjKlMnOpQrStUvWxYz0123456789abcdefghij";
|
||||
|
||||
fn session_with_tool_output(output: &str) -> Value {
|
||||
json!({
|
||||
"schema_version": 1,
|
||||
"metadata": {"id": "s1"},
|
||||
"messages": [
|
||||
{"role": "user", "content": [{"type": "text", "text": "show auth"}]},
|
||||
{"role": "user", "content": [{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": "call-1",
|
||||
"content": output,
|
||||
"content_blocks": [{"type": "text", "text": output}]
|
||||
}]}
|
||||
],
|
||||
"journal": {"entries": [{"message": {"role": "user", "content": [{
|
||||
"type": "tool_result", "tool_use_id": "call-1", "content": output
|
||||
}]}}]}
|
||||
})
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scan_reports_and_scrub_masks_stored_tool_output_tokens() {
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let output = format!("{{\"access_token\": \"{TOKEN}\"}}");
|
||||
let dirty = dir.path().join("dirty.json");
|
||||
std::fs::write(&dirty, session_with_tool_output(&output).to_string()).unwrap();
|
||||
let clean = dir.path().join("clean.json");
|
||||
std::fs::write(
|
||||
&clean,
|
||||
session_with_tool_output("nothing secret").to_string(),
|
||||
)
|
||||
.unwrap();
|
||||
std::fs::create_dir_all(dir.path().join("checkpoints")).unwrap();
|
||||
let checkpoint = dir.path().join("checkpoints").join("dirty.json");
|
||||
std::fs::write(&checkpoint, session_with_tool_output(&output).to_string()).unwrap();
|
||||
std::fs::write(dir.path().join("broken.json"), "{not json").unwrap();
|
||||
|
||||
let files = session_files(dir.path());
|
||||
assert_eq!(files.len(), 4, "{files:?}");
|
||||
|
||||
let report = scrub_files(&files, false).expect("scan");
|
||||
assert_eq!(report.files_scanned, 4);
|
||||
assert_eq!(report.files_with_secrets.len(), 2, "{report:?}");
|
||||
assert_eq!(report.tool_results_with_secrets, 6);
|
||||
assert_eq!(report.unreadable, vec![dir.path().join("broken.json")]);
|
||||
assert!(
|
||||
std::fs::read_to_string(&dirty).unwrap().contains(TOKEN),
|
||||
"a scan must not rewrite anything"
|
||||
);
|
||||
|
||||
let applied = scrub_files(&files, true).expect("scrub");
|
||||
assert_eq!(applied.files_with_secrets.len(), 2);
|
||||
for path in [&dirty, &checkpoint] {
|
||||
let text = std::fs::read_to_string(path).unwrap();
|
||||
assert!(!text.contains(TOKEN), "{text}");
|
||||
let value: Value = serde_json::from_str(&text).expect("still valid JSON");
|
||||
assert_eq!(value["metadata"]["id"], "s1");
|
||||
assert_eq!(value["messages"][0]["content"][0]["text"], "show auth");
|
||||
}
|
||||
assert_eq!(
|
||||
scrub_files(&files, false)
|
||||
.expect("rescan")
|
||||
.files_with_secrets,
|
||||
Vec::<PathBuf>::new(),
|
||||
"a scrubbed store scans clean"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_tool_result_text_is_left_alone() {
|
||||
let mut value = json!({
|
||||
"messages": [{"role": "user", "content": [{"type": "text", "text": TOKEN}]}]
|
||||
});
|
||||
assert_eq!(scrub_value(&mut value), 0);
|
||||
assert_eq!(value["messages"][0]["content"][0]["text"], TOKEN);
|
||||
}
|
||||
}
|
||||
@@ -3308,6 +3308,26 @@ Until a confirmation exists, the effective mode is always `"enabled"`:
|
||||
always-on redaction regardless of this switch; the opt-out exists so the
|
||||
model can quote file bytes for exact edits, not to relax stored state.
|
||||
|
||||
### Stored sessions
|
||||
|
||||
The masking runs once, when tool output enters the transcript, so saved
|
||||
session files (`~/.codewhale/sessions/*.json` and their checkpoints) never
|
||||
store a live credential from tool output. With the confirmed opt-out above,
|
||||
tool output is stored as the model saw it.
|
||||
|
||||
Sessions saved by builds before this change may still hold credentials in
|
||||
their tool output; nothing rewrites them automatically. `codewhale doctor`
|
||||
reports them under **Stored Sessions** (it checks the newest 50 files), and
|
||||
this command finds and masks them across every saved session:
|
||||
|
||||
```sh
|
||||
codewhale sessions scrub-secrets # report only
|
||||
codewhale sessions scrub-secrets --apply # rewrite the affected files
|
||||
```
|
||||
|
||||
Close open Codewhale sessions before `--apply`, and rotate any credential that
|
||||
was exposed — masking a stored copy cannot un-leak it.
|
||||
|
||||
The config value itself is forgiving: `true`/`false`, `"on"`/`"off"`, and
|
||||
`"enabled"`/`"disabled"` (any casing) all parse, with `false`/`"off"` meaning
|
||||
`"disabled"`.
|
||||
|
||||
Reference in new issue
Block a user