perf(core): defer workflow tool schemas by default (#16310)

* perf(core): defer workflow tool schemas by default

* test(core): preserve visible workflow in mod permission fixture

* test(integration): expect deferred workflow in headless requests
This commit is contained in:
callmeYe authored and GitHub committed 2026-11-05 02:09:34 +00:00
1 parent ef9c891b8f
commit 4b980c9636
7 files changed
+345 -8

No files matched your search

+68
View File
@@ -0,0 +1,68 @@
# Deferred Workflow tool
[English](deferred-workflow-tool.md) | [简体中文](deferred-workflow-tool.zh-CN.md)
## Problem
By default, when workflows are enabled, Direct-mode requests carry the complete
Workflow description and parameter schema, even when the user never runs a
workflow. The existing deferred-tool bridge already supports discovering a
tool's schema with `tool_search` and invoking it with `tool_call` without
changing the request's standing function declarations.
## Decision
Make Workflow deferred by default with `shouldDefer=true`. Keep its full
description, parameter schema, registration gates, authoring reference and
execution behavior unchanged. The startup catalog retains the existing short
description. Explicit workflow intent and persistent Workflow mode still use
on-demand discovery; neither automatically reveals the schema.
The per-turn `isToolHiddenBehindToolSearch` helper must recognize native
deferred tools through the registry's `isDeferredAndHidden` result. Retain its
permission-deferred lazy-factory fallback, because a factory need not have a
tool instance yet. Preserve the CodeModeOnly exclusion: that mode invokes
discovered tools through `exec`, not the Direct-mode `tool_call` bridge. Do not
change the separate construction-time Skill reference decision.
## Compatibility and scope
- `tools.visible` can still expose Workflow directly. `tools.eager` does not
override a tool's native deferral, consistent with other deferred tools.
- The existing schema-budget preload and incomplete-bridge fallback still
apply. An explicit permission deferral remains distinct from native deferral.
A nonzero preload budget remains all-or-nothing: Workflow now contributes to
the candidate total and can push the entire deferred set over budget. The
default zero budget does not preload tools.
Normal LLM initialization runs `startChat` and resolves the incomplete-bridge
fallback before Config marks Workflow mode ready or applies its initial mode,
so mode activation does not wait for the first model request.
- Old histories containing direct Workflow calls can reveal the tool during
replay; `/clear` retains its existing reset behavior.
- Named workflows, name-only sessions, host-owned runs and `/review` activation
keep their existing registration and execution paths. Deferred calls retain
target-tool approval, hooks and subagent restrictions.
- No public API, settings, parameter or persistence changes are required. Other
tools, context-usage accounting and deployment are outside this change.
## Verification and acceptance
Use a real Workflow tool in registry and discovery tests. A default request with
workflows enabled must omit its full declaration and retain a short catalog
entry. Exact and keyword search must return the unchanged schema; reviewing it
must not expand the standing declarations. A reviewed call must execute through
the existing bridge, and unreviewed or policy-forbidden calls must remain
blocked.
Check native-deferred workflow intent and mode reminders, visible/revealed
tools, CodeModeOnly, permission-deferred lazy factories, schema-budget preload,
disabled workflows, missing bridge halves, name-only schemas, history replay,
`/clear` and mid-session `/review` activation. Run focused package tests, build,
typecheck and bundle.
Dry-run the headless E2E procedure with the global CLI, then compare actual
requests from the exact unmodified base and changed build with identical input
and configuration. Report token estimates with a fixed tokenizer separately
from provider usage. Mock-provider usage is synthetic and cannot establish
billed savings. The acceptance criterion is request-content reduction and a
working discovery/invocation path, not a fixed token-saving threshold.
@@ -0,0 +1,53 @@
# Workflow 工具延迟加载
[English](deferred-workflow-tool.md) | [简体中文](deferred-workflow-tool.zh-CN.md)
## 问题
默认情况下,启用 workflow 后,即使用户没有运行 workflow,Direct 模式请求仍携带完整的
Workflow 描述和参数 schema。已有延迟工具桥接机制支持通过 `tool_search` 查阅
schema,再通过 `tool_call` 调用,无需改变请求中常驻的函数声明。
## 决策
通过 `shouldDefer=true` 将 Workflow 改为默认延迟加载。保持完整描述、参数 schema、
注册条件、编写参考和执行行为不变。启动目录保留现有简短描述。明确的 workflow 意图和
持久 Workflow 模式也按需发现工具,均不自动展示完整 schema。
逐轮调用的 `isToolHiddenBehindToolSearch` 应通过注册表的 `isDeferredAndHidden`
结果识别工具原生的延迟状态。同时保留 permission-deferred lazy factory 的兜底判断,
因为此时工具实例可能尚未构造。保留 CodeModeOnly 排除逻辑:该模式通过 `exec` 调用
发现的工具,不使用 Direct 模式的 `tool_call` 桥。保持独立的 Skill 编写参考构造期
判断不变。
## 兼容性和范围
- `tools.visible` 仍可直接展示 Workflow。与其他延迟工具一致,`tools.eager` 不覆盖
工具自身的延迟状态。
- 保留现有 schema 预算预加载和桥接工具缺失时的兜底。配置明确要求的延迟与工具原生
延迟仍分别处理。
非零预加载预算仍采用全有或全无规则:Workflow 现在计入候选总量,可能使整组延迟工具
超出预算。默认零预算不预加载工具。
正常 LLM 初始化会执行 `startChat` 并完成缺桥兜底,随后 Config 才将 Workflow 模式
标记为就绪、应用初始模式,因此模式激活无需等待首个模型请求。
- 包含直接 Workflow 调用的旧会话可在回放时展示工具;`/clear` 保持现有重置行为。
- 已保存的具名 workflow、name-only 会话、宿主运行和 `/review` 激活保持现有注册及
执行路径。延迟调用仍遵守目标工具审批、hooks 和子代理限制。
- 无需修改公开 API、配置、参数或持久化格式。其他工具、context-usage 统计和部署不在
此次改动范围内。
## 验证和验收
使用真实 Workflow 工具进行注册表和发现测试。启用 workflow 的默认请求必须省略完整
声明,保留简短目录条目。精确搜索和关键词搜索必须返回原有 schema,查阅后不能扩展
常驻声明。查阅后的调用必须通过现有桥接执行,未经查阅或被策略禁止的调用仍须拦截。
检查原生延迟 workflow 的意图及模式提示、visible/revealed 工具、CodeModeOnly、
permission-deferred lazy factory、schema 预算预加载、workflow 禁用、桥接任一端
缺失、name-only schema、历史回放、`/clear` 和会话中途 `/review` 激活。执行相关
包内测试、build、typecheck 和 bundle。
先用全局 CLI dry-run 无界面 E2E 流程,再使用完全相同的输入和配置,对比精确未修改
基线与修改版构建的实际请求。使用固定 tokenizer 的估算与 Provider 用量分别报告。
Mock Provider 的 usage 是合成值,不能证明计费用量收益。验收依据是请求内容减少且
发现、调用链路可用,不设固定 token 节省阈值。
@@ -77,7 +77,12 @@ describe('DW03: headless prompts remain unknown', () => {
const body = JSON.stringify(requests[0].body['messages']);
expect(body).toContain('workflow ultracode');
expect(body).not.toContain("The user's message includes the");
expect(JSON.stringify(requests[0].body['tools'])).toContain('workflow');
expect(body).toContain('- \\"workflow\\":');
expect(requests[0].body['tools']).not.toContainEqual(
expect.objectContaining({
function: expect.objectContaining({ name: 'workflow' }),
}),
);
};
it.each(['text', 'json', 'stream-json'])(
@@ -575,6 +575,7 @@ describe('headless-owned Mod tool.check restriction', () => {
chatRecording: true,
workflowsEnabled: true,
workflowNameOnly: true,
visibleTools: ['workflow'],
approvalMode,
interactive: approver === 'person',
inputFormat:
@@ -217,20 +217,21 @@ function isToolDeferredBehindToolSearch(config: Config, name: string): boolean {
}
/**
* Whether a registered tool's schema is withheld from the request right now:
* deferred as above and not revealed through ToolSearch yet. For a question
* asked again on every turn, such as the Workflow keyword reminder's.
* Whether a registered tool's schema is withheld from the Direct-mode request
* right now. Used each turn by the Workflow keyword and mode reminders.
*
* Mirrors `ToolRegistry.isDeferredAndHidden`, which cannot be used here: it
* answers false for a tool that is still only a lazy factory, which is exactly
* the state while the tool asking the question is being constructed.
* The registry covers native deferral, but a permission-deferred lazy factory
* has no instance yet and still needs the construction-time fallback.
*/
export function isToolHiddenBehindToolSearch(
config: Config,
name: string,
): boolean {
if (config.getToolMode?.() === ToolMode.CodeModeOnly) return false;
const registry = config.getToolRegistry?.();
if (registry?.isDeferredAndHidden?.(name)) return true;
if (!isToolDeferredBehindToolSearch(config, name)) return false;
return !config.getToolRegistry?.()?.isDeferredToolRevealed?.(name);
return !registry?.isDeferredToolRevealed?.(name);
}
/**
@@ -0,0 +1,208 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { afterEach, describe, expect, it, vi } from 'vitest';
import { Config, type ConfigParameters } from '../../config/config.js';
import { buildDeferredToolsReminder } from '../../core/environmentContext.js';
import {
resolveWorkflowContext,
resolveWorkflowIntent,
} from '../../core/workflow-intent.js';
import { WorkflowModeContext } from '../../core/workflow-mode.js';
import { runWithAgentContext } from '../../agents/runtime/agent-context.js';
import { MockTool } from '../../test-utils/mock-tool.js';
import { ToolRegistry } from '../tool-registry.js';
import { ToolSearchTool } from '../tool-search.js';
import { resolveDeferredToolCall, ToolCallTool } from '../tool-call.js';
import { ToolErrorType } from '../tool-error.js';
import { ToolNames } from '../tool-names.js';
import { WorkflowTool } from './workflow.js';
afterEach(() => vi.restoreAllMocks());
function fixture(params: Partial<ConfigParameters> = {}) {
const config = new Config({
cwd: '/tmp',
targetDir: '/tmp',
model: 'test-model',
embeddingModel: 'test-embedding-model',
sandbox: undefined,
debugMode: false,
workflowsEnabled: true,
...params,
});
const registry = new ToolRegistry(config);
vi.spyOn(config, 'getToolRegistry').mockReturnValue(registry);
const search = new ToolSearchTool(config);
registry.registerTool(search);
registry.registerTool(new ToolCallTool());
registry.registerTool(new MockTool({ name: ToolNames.EXEC }));
const workflow = new WorkflowTool(config);
registry.registerTool(workflow);
const lookup = (query = 'select:workflow') =>
search.build({ query }).execute(new AbortController().signal);
return { config, registry, workflow, lookup };
}
describe('deferred Workflow discovery', () => {
it('keeps only the short catalog entry in a fresh Direct request', () => {
const { registry, workflow } = fixture();
expect(registry.getFunctionDeclarations()).not.toContainEqual(
workflow.schema,
);
expect(
registry.getFunctionDeclarations({ includeDeferred: true }),
).toContainEqual(workflow.schema);
const reminder = buildDeferredToolsReminder(registry);
expect(reminder).toContain('"workflow"');
expect(reminder).toContain(workflow.description.split('\n')[0]);
expect(reminder).not.toContain('**Runtime**');
expect(reminder).not.toContain('parametersJsonSchema');
});
it.each(['select:workflow', 'workflow orchestration'])(
'returns the full schema without expanding declarations: %s',
async (query) => {
const { registry, workflow, lookup } = fixture();
const declarations = registry.getFunctionDeclarations();
const envelope = {
name: 'workflow',
arguments: { script: 'return 42;' },
};
expect(await resolveDeferredToolCall(registry, envelope)).toMatchObject({
errorType: ToolErrorType.INVALID_TOOL_PARAMS,
error: expect.objectContaining({
message: expect.stringContaining('no verified schema review'),
}),
});
const result = await lookup(query);
expect(result.error).toBeUndefined();
const block = String(result.llmContent).match(
/<function>([\s\S]*?)<\/function>/,
);
expect(block).not.toBeNull();
expect(JSON.parse(block![1])).toEqual(workflow.schema);
expect(await resolveDeferredToolCall(registry, envelope)).toEqual({
tool: workflow,
arguments: envelope.arguments,
});
expect(registry.getFunctionDeclarations()).toEqual(declarations);
expect(registry.isDeferredAndHidden('workflow')).toBe(true);
},
);
it('directs native-deferred keyword and mode requests through the bridge', () => {
const { config, registry } = fixture();
const intent = resolveWorkflowIntent(config, { text: 'workflow' });
const context = resolveWorkflowContext(config);
const mode = new WorkflowModeContext();
mode.setRequested(true);
for (const reminder of [
intent?.reminder,
mode.prepare(context, null).reminder,
]) {
expect(reminder).toContain('`tool_search`');
expect(reminder).toContain('`tool_call`');
}
expect(registry.isDeferredAndHidden('workflow')).toBe(true);
});
it.each([ToolNames.TOOL_SEARCH, ToolNames.TOOL_CALL])(
'does not advertise a hidden tool when %s is missing',
(name) => {
const { config, registry } = fixture();
registry.unregisterTool(name);
expect(resolveWorkflowContext(config)).toBeNull();
registry.revealDeferredTool('workflow');
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
},
);
it('honors explicit visibility without suggesting bridge discovery', () => {
const { config, registry, workflow } = fixture({
visibleTools: ['workflow'],
});
expect(registry.getFunctionDeclarations()).toContainEqual(workflow.schema);
expect(buildDeferredToolsReminder(registry)).toBeNull();
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
});
it('preserves budget preload and restores deferral after clearing reveals', () => {
const { config, registry, workflow } = fixture();
expect(registry.preloadDeferredToolsWithinBudget(0)).toBe(0);
expect(registry.preloadDeferredToolsWithinBudget(100_000)).toBe(1);
expect(registry.getFunctionDeclarations()).toContainEqual(workflow.schema);
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
registry.clearRevealedDeferredTools();
expect(registry.getFunctionDeclarations()).not.toContainEqual(
workflow.schema,
);
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(true);
});
it('keeps CodeMode discovery behind exec without Direct bridge instructions', async () => {
const { config, registry, workflow, lookup } = fixture({
codeModeOnly: true,
});
const declarations = registry.getFunctionDeclarations();
expect(JSON.stringify(declarations)).not.toContain('tools.workflow(args:');
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
const result = await lookup();
const block = String(result.llmContent).match(
/<function>([\s\S]*?)<\/function>/,
);
expect(block).not.toBeNull();
expect(JSON.parse(block![1])).toMatchObject({
...workflow.schema,
jsName: 'workflow',
});
expect(result.llmContent).toContain('through exec');
expect(registry.getFunctionDeclarations()).toEqual(declarations);
});
it('retains the named-only schema through discovery', async () => {
const { workflow, lookup } = fixture({ workflowNameOnly: true });
const result = await lookup();
const block = String(result.llmContent).match(
/<function>([\s\S]*?)<\/function>/,
);
expect(JSON.parse(block![1])).toEqual(workflow.schema);
expect(workflow.parameterSchema).toHaveProperty('properties.name');
expect(workflow.parameterSchema).not.toHaveProperty('properties.script');
expect(() => workflow.build({ script: 'return 42;' })).toThrow(
'named workflows',
);
});
it('does not let a reviewed deferred Workflow bypass subagent restrictions', async () => {
const { registry, lookup } = fixture();
await lookup();
const result = await runWithAgentContext('child', () =>
resolveDeferredToolCall(registry, {
name: 'workflow',
arguments: { script: 'return 42;' },
}),
);
expect(result).toMatchObject({ errorType: ToolErrorType.EXECUTION_DENIED });
expect(result).not.toHaveProperty('tool');
});
it('keeps mid-session review activation deferred and discoverable', async () => {
const { config, registry, lookup } = fixture({
workflowsEnabled: undefined,
});
registry.unregisterTool('workflow');
const declarations = registry.getFunctionDeclarations();
const setTools = vi.fn(async () => registry.warmAll());
vi.spyOn(config, 'getLlmClient').mockReturnValue({ setTools } as never);
await config.enableReviewWorkflow();
expect(config.isWorkflowsEnabled()).toBe(true);
expect(setTools).toHaveBeenCalledOnce();
expect(registry.isDeferredAndHidden('workflow')).toBe(true);
expect((await lookup()).llmContent).toContain('"name":"workflow"');
expect(registry.getFunctionDeclarations()).toEqual(declarations);
});
});
@@ -1806,6 +1806,7 @@ export class WorkflowTool extends BaseDeclarativeTool<
buildWorkflowParamSchema(surface, nameOnly),
/* isOutputMarkdown */ true,
/* canUpdateOutput */ true,
/* shouldDefer */ true,
);
this.authoringSurface = surface;
}