fix(workflows): route the stopship scout through tool_search and grep_files

The read-only scout brief commanded a `File` call with `action`
`search_content`, but `File` is a hidden replay-only alias
(`model_visible=false`): it never reaches a model-visible catalog, and
the child step loop fails any call outside the child's policy-filtered
catalog outright. The release-acceptance explore gate's first step
therefore could not succeed; dispatch-by-alias only worked below the
catalog gate (replay tests calling `registry.resolve` directly).

Route the scout through the live surface instead: response 1 activates
the deferred `grep_files` with one `tool_search` call, response 2 runs
the same alternation search through `grep_files`, and response 3
returns the verdict. The step and token caps are unchanged; the
evidence turn grows one small activation response. The leaf's explicit
`allowed_tools: ["File"]` is dropped — the read-only lowering already
scopes the leaf, and its alias-family intersection is what keeps
`grep_files` discoverable.

Guard both halves so neither side silently drifts again: a surface test
pins the scout catalog to an active `tool_search` plus a deferred,
searchable `grep_files` with no `File`; a behavioral test drives the
activation through the real dispatch gate; and the fixture guard now
pins the three-response contract and denies catalog-invisible tool
names in the brief.

Adapted from the Pinvou fork (Pinvou/CodeWhale 92427bd8d, PR #61).

Signed-off-by: asto18089 <[email protected]>
This commit is contained in:
asto18089 committed 2026-09-29 15:00:17 +08:00
1 parent 5f7ca50162
commit ec246ab063
3 files changed
+198 -18

No files matched your search

+174
View File
@@ -24627,6 +24627,180 @@ async fn agent_claim_is_withheld_from_a_role_with_no_write_authority() {
assert!(refusal.contains("no write authority to widen"), "{refusal}");
}
// Regression (stopship scout repair): the workflow's read-only scout
// activates `grep_files` with one `tool_search` call before searching. That
// two-step path only works while the scout surface keeps a first-turn-active
// `tool_search` and a deferred, searchable `grep_files`; the hidden `File`
// alias the brief cited before is filtered from every model-visible catalog
// (`to_api_tools`), which is what silently broke the release-acceptance
// explore gate. The registry is built with the scope the fixture scout really
// runs under (`leaf_allowed_tools` lowers a read-only leaf to `["File"]`), so
// the guard also pins the alias-family intersection that keeps `grep_files`
// discoverable under that legacy rule. If a surface reshape fails this test,
// re-work the fixture brief in the same change instead of leaving it
// instructing calls the child cannot make.
#[tokio::test]
async fn scout_surface_keeps_tool_search_grep_files_activation_path() {
let tmp = tempdir().expect("tempdir");
let mut runtime =
stub_runtime().with_agent_tool_surface_options(enabled_agent_surface_options());
runtime.context = ToolContext::new(tmp.path().to_path_buf());
let todo_list = crate::tools::todo::new_shared_todo_list();
let plan_state = crate::tools::plan::new_shared_plan_state();
let registry = SubAgentToolRegistry::new(
runtime,
FleetRole::Scout,
Some(vec!["File".to_string()]),
todo_list,
plan_state,
);
let catalog = registry.deferred_catalog_for_model(&FleetRole::Scout);
let search = catalog
.iter()
.find(|tool| tool.name == "tool_search")
.expect("scout must keep an active tool_search");
let grep = catalog
.iter()
.find(|tool| tool.name == "grep_files")
.expect("scout must keep a deferred, searchable grep_files");
assert_eq!(
search.defer_loading,
Some(false),
"tool_search must be first-turn active on the scout"
);
assert_eq!(
grep.defer_loading,
Some(true),
"grep_files must be deferred so the taught tool_search activation is required"
);
assert!(
!catalog.iter().any(|tool| tool.name == "File"),
"the hidden File alias must stay out of scout catalogs"
);
}
// Regression (stopship scout repair, phantom-tool class): fleet workflow
// briefs are model-facing text too. The scout brief must name tools the child
// catalog can actually see (`tool_search`, `grep_files`) and never cite the
// hidden `File` alias or its retired `search_content` action — that wording
// failed the release-acceptance explore gate exactly the way the
// bundled-skills `File` citations stalled real reasoning loops.
#[test]
fn workflow_briefs_name_catalog_visible_tools() {
const STOPSHIP: &str = include_str!("../../../../../workflows/stopship.workflow.js");
// Guard the fixture definition, not the maintainer header comment.
let body = STOPSHIP
.split_once("export default")
.expect("workflow module must export its definition")
.1;
assert!(
body.contains("`tool_search`") && body.contains("`grep_files`"),
"the scout brief must teach the two-step activation path:\n{body}"
);
// Every catalog-invisible execution name (hidden replay aliases and the
// retired `search_content` action) is denied as a whole word, so an
// unbackticked or renamed citation cannot slip past the guard. `list_dir`
// and the lowercase primitives stay legal: they are model-visible.
const HIDDEN_EXEC_NAMES: [&str; 8] = [
"File",
"Bash",
"TodoWrite",
"work_update",
"read_file",
"write_file",
"edit_file",
"search_content",
];
let cited: Vec<&str> = body
.split(|c: char| !(c.is_ascii_alphanumeric() || c == '_'))
.filter(|word| HIDDEN_EXEC_NAMES.iter().any(|hidden| hidden == word))
.collect();
assert!(
cited.is_empty(),
"workflow briefs must not command calls a catalog can never return: {cited:?}"
);
}
// Behavioral counterpart to the two composition guards above: on the scout's
// live surface, one `tool_search` call must make the deferred `grep_files`
// dispatchable through the same gate the child step loop uses, while the
// hidden `File` alias the old brief commanded must keep failing the catalog
// gate. Composition can drift from behavior; this cannot.
#[tokio::test]
async fn scout_activation_makes_grep_files_dispatchable() {
let tmp = tempdir().expect("tempdir");
let mut runtime =
stub_runtime().with_agent_tool_surface_options(enabled_agent_surface_options());
runtime.context = ToolContext::new(tmp.path().to_path_buf());
let todo_list = crate::tools::todo::new_shared_todo_list();
let plan_state = crate::tools::plan::new_shared_plan_state();
let registry = SubAgentToolRegistry::new(
runtime,
FleetRole::Scout,
Some(vec!["File".to_string()]),
todo_list,
plan_state,
);
// Mirror the spawn loop: filtered catalog, then a cold surface.
let mut surface =
SubAgentToolSurface::new(registry.deferred_catalog_for_model(&FleetRole::Scout), &[]);
let active_names: std::collections::HashSet<String> = surface.active_names.clone();
let file_error = registry
.execute_from_surface(
"agent_unknown",
"",
&mut surface,
&active_names,
"File",
serde_json::json!({"action": "search_content", "path": "."}),
)
.await
.expect_err("the hidden File alias must fail the scout catalog gate");
assert!(
file_error
.to_string()
.contains("not in this child's policy-filtered catalog"),
"{file_error}"
);
let active_names: std::collections::HashSet<String> = surface.active_names.clone();
registry
.execute_from_surface(
"agent_unknown",
"",
&mut surface,
&active_names,
"tool_search",
serde_json::json!({"query": "grep_files"}),
)
.await
.expect("tool_search activation must succeed on the scout surface");
assert!(
surface.active_names.contains("grep_files"),
"activation must admit grep_files into the same surface the dispatch gate reads"
);
let active_names: std::collections::HashSet<String> = surface.active_names.clone();
registry
.execute_from_surface(
"agent_unknown",
"",
&mut surface,
&active_names,
"grep_files",
serde_json::json!({
"path": ".",
"pattern": "stopship",
"max_results": 5,
"context_lines": 1
}),
)
.await
.expect("grep_files must dispatch through the real tool after the taught activation");
}
#[test]
fn agent_tool_description_names_only_schema_roles() {
// Every `type=<token>` and every role in the "type selects the Fleet role:"
+18 -16
View File
@@ -652,15 +652,12 @@ workflow({
assert_eq!(leaf.role.as_deref(), Some(expected_role));
assert_eq!(leaf.mode, TaskMode::ReadOnly);
assert!(!leaf.permissions.allow_write);
let expected_tools: &[&str] = if expected_role == "explore" {
&["File"]
} else {
&[]
};
assert_eq!(
leaf.permissions.allowed_tools, expected_tools,
"the fixture must explicitly restrict source gathering to File"
);
// No explicit `allowed_tools` anywhere: the read-only lowering
// alone scopes the source-gathering explore role, and its
// catalog-visible surface (tool_search + deferred grep_files) is
// pinned from the TUI side (scout_surface_keeps_tool_search_
// grep_files_activation_path).
assert!(leaf.permissions.allowed_tools.is_empty());
assert_eq!(
leaf.permissions.deny_all_tools,
expected_role != "explore",
@@ -680,12 +677,17 @@ workflow({
);
if expected_role == "explore" {
assert!(
leaf.prompt.contains("exactly one `File` call")
&& leaf.prompt.contains("Do not call `File` more than once")
&& leaf
.prompt
.contains("do not use any action except `search_content`"),
"the scout must finish discovery in one bounded tool round"
leaf.prompt.contains("exactly one `tool_search` call")
&& leaf.prompt.contains("exactly one `grep_files` call")
&& leaf.prompt.contains(
"the content-search tool is deferred, so this activation is required before it can be called"
),
"the scout must activate the deferred content-search tool before searching with it"
);
assert!(
leaf.prompt.contains("Make no other tool calls")
&& leaf.prompt.contains("nothing else"),
"the scout discovery must stay one bounded activation-plus-search round"
);
assert_eq!(
leaf.file_scope
@@ -705,7 +707,7 @@ workflow({
leaf.prompt.contains(
"`include` set exactly to [`fleets/stopship.toml`, `crates/cli/src/lib.rs`, `crates/workflow/src/role_resolve.rs`, `crates/tui/src/tools/workflow.rs`, `crates/lane/src/runtime.rs`]"
) && leaf.prompt.contains("Matches outside that exact include list do not count"),
"the one File search must constrain the actual tool input, not only File scope metadata"
"the grep_files search must constrain the actual tool input, not only file scope metadata"
);
assert!(
leaf.prompt.contains("if you can populate all seven")
+6 -2
View File
@@ -6,6 +6,11 @@
// tokens. Budget 24k per intended evidence turn, then add token-neutral
// max_steps headroom for the required final verdict. The token ceiling
// remains independent and the five role caps still total 360k.
// Scout repair: the read-only catalog carries no `File` (hidden replay-only
// alias, filtered from every model-visible surface), so the scout now
// activates `grep_files` with one `tool_search` call before searching. The
// evidence turn spans three responses (activate, search, verdict) inside the
// unchanged step/token caps.
export default workflow({
"id": "stopship-release-acceptance",
"goal": "Verify the current Codewhale Fleet, Workflow, Lane, Runtime, and gate receipt path without changing the workspace",
@@ -74,11 +79,10 @@ export default workflow({
{
"agent": {
"id": "explore-runtime",
"prompt": "Verify the runtime release-orchestration owners using only the five files in File scope. The host's typed run_started receipt already owns the compiled Workflow id and source path; do not re-verify the Workflow alias. You have at most six model responses and must reserve the verdict. Response 1 must make exactly one `File` call with `action` set to `search_content`, `path` set to `.`, and `include` set exactly to [`fleets/stopship.toml`, `crates/cli/src/lib.rs`, `crates/workflow/src/role_resolve.rs`, `crates/tui/src/tools/workflow.rs`, `crates/lane/src/runtime.rs`], using this high-signal alternation pattern: `name = \"stopship\"|load_named_fleet|start_lane|resolve_workflow_agent|record_task_started|WorkflowUiEventKind::GateUpdated|WorkflowUiEventKind::RunCompleted|terminal_completed_receipt|process_exit_receipt|lane_reconciled|tmux_reconcile_folds_detached_process_exit_into_lane_status|stopship_acceptance_fixture_emits_role_gate_and_terminal_receipts`. Set at most 80 results and 2 context lines. Matches outside that exact include list do not count. Do not add generic field names such as `resolved_profile` or `exit_code` to the pattern. Do not call `File` more than once and do not use any action except `search_content` or call any other tool. Response 2 must return the verdict with no tool calls; any later reserved response must do the same instead of gathering more evidence. Treat an exact match naming a call site, typed event constructor, reconciliation branch, or test assertion in a scoped file as source-owner evidence. Apply this decision rule literally: if you can populate all seven required SOURCE EVIDENCE entries from the search result, return APPROVE; never return BLOCK after citing all seven. Return BLOCK only when at least one named owner has no matching citation, and identify each missing owner as MISSING. The first non-empty line of your response must be exactly APPROVE or exactly BLOCK. Do not put any words before that verdict: no confirmation, summary, heading, or phrase such as `Here is the verdict`. After the verdict, include a `SOURCE EVIDENCE` section with concise `path: symbol` evidence for named Fleet loading, role-to-profile resolution, tmux Lane launch, typed task_started, gate_updated, terminal run_completed, and tmux process-exit reconciliation receipts. The terminal run_completed entry must carry both the `WorkflowUiEventKind::RunCompleted` constructor and the `terminal_completed_receipt` integration assertion. A bare verdict is invalid. Do not edit files, create branches, run shell commands, access GitHub, or infer success where source evidence is absent.",
"prompt": "Verify the runtime release-orchestration owners using only the five files in scope. The host's typed run_started receipt already owns the compiled Workflow id and source path; do not re-verify the Workflow alias. You have at most six model responses and must reserve the verdict. Response 1 must make exactly one `tool_search` call with `query` set to `grep_files`; the content-search tool is deferred, so this activation is required before it can be called. Response 2 must make exactly one `grep_files` call with `path` set to `.`, `include` set exactly to [`fleets/stopship.toml`, `crates/cli/src/lib.rs`, `crates/workflow/src/role_resolve.rs`, `crates/tui/src/tools/workflow.rs`, `crates/lane/src/runtime.rs`], and `pattern` set to this high-signal alternation: `name = \"stopship\"|load_named_fleet|start_lane|resolve_workflow_agent|record_task_started|WorkflowUiEventKind::GateUpdated|WorkflowUiEventKind::RunCompleted|terminal_completed_receipt|process_exit_receipt|lane_reconciled|tmux_reconcile_folds_detached_process_exit_into_lane_status|stopship_acceptance_fixture_emits_role_gate_and_terminal_receipts`. Set `max_results` to 80 and `context_lines` to 2. Matches outside that exact include list do not count. Do not add generic field names such as `resolved_profile` or `exit_code` to the pattern. Make no other tool calls: exactly one `tool_search` call, then exactly one `grep_files` call, nothing else. Response 3 must return the verdict with no tool calls; any later reserved response must do the same instead of gathering more evidence. Treat an exact match naming a call site, typed event constructor, reconciliation branch, or test assertion in a scoped file as source-owner evidence. Apply this decision rule literally: if you can populate all seven required SOURCE EVIDENCE entries from the search result, return APPROVE; never return BLOCK after citing all seven. Return BLOCK only when at least one named owner has no matching citation, and identify each missing owner as MISSING. The first non-empty line of your response must be exactly APPROVE or exactly BLOCK. Do not put any words before that verdict: no confirmation, summary, heading, or phrase such as `Here is the verdict`. After the verdict, include a `SOURCE EVIDENCE` section with concise `path: symbol` evidence for named Fleet loading, role-to-profile resolution, tmux Lane launch, typed task_started, gate_updated, terminal run_completed, and tmux process-exit reconciliation receipts. The terminal run_completed entry must carry both the `WorkflowUiEventKind::RunCompleted` constructor and the `terminal_completed_receipt` integration assertion. A bare verdict is invalid. Do not edit files, create branches, run shell commands, access GitHub, or infer success where source evidence is absent.",
"agent_type": "explore",
"role": "explore",
"mode": "read_only",
"permissions": { "allowed_tools": ["File"] },
"file_scope": [
"fleets/stopship.toml",
"crates/cli/src/lib.rs",