mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
perf(core): defer workflow tool schemas by default (#16310)
* perf(core): defer workflow tool schemas by default * test(core): preserve visible workflow in mod permission fixture * test(integration): expect deferred workflow in headless requests
This commit is contained in:
1 parent
ef9c891b8f
commit
4b980c9636
7 files changed
+345
-8
No files matched your search
@@ -0,0 +1,68 @@
|
||||
# Deferred Workflow tool
|
||||
|
||||
[English](deferred-workflow-tool.md) | [简体中文](deferred-workflow-tool.zh-CN.md)
|
||||
|
||||
## Problem
|
||||
|
||||
By default, when workflows are enabled, Direct-mode requests carry the complete
|
||||
Workflow description and parameter schema, even when the user never runs a
|
||||
workflow. The existing deferred-tool bridge already supports discovering a
|
||||
tool's schema with `tool_search` and invoking it with `tool_call` without
|
||||
changing the request's standing function declarations.
|
||||
|
||||
## Decision
|
||||
|
||||
Make Workflow deferred by default with `shouldDefer=true`. Keep its full
|
||||
description, parameter schema, registration gates, authoring reference and
|
||||
execution behavior unchanged. The startup catalog retains the existing short
|
||||
description. Explicit workflow intent and persistent Workflow mode still use
|
||||
on-demand discovery; neither automatically reveals the schema.
|
||||
|
||||
The per-turn `isToolHiddenBehindToolSearch` helper must recognize native
|
||||
deferred tools through the registry's `isDeferredAndHidden` result. Retain its
|
||||
permission-deferred lazy-factory fallback, because a factory need not have a
|
||||
tool instance yet. Preserve the CodeModeOnly exclusion: that mode invokes
|
||||
discovered tools through `exec`, not the Direct-mode `tool_call` bridge. Do not
|
||||
change the separate construction-time Skill reference decision.
|
||||
|
||||
## Compatibility and scope
|
||||
|
||||
- `tools.visible` can still expose Workflow directly. `tools.eager` does not
|
||||
override a tool's native deferral, consistent with other deferred tools.
|
||||
- The existing schema-budget preload and incomplete-bridge fallback still
|
||||
apply. An explicit permission deferral remains distinct from native deferral.
|
||||
A nonzero preload budget remains all-or-nothing: Workflow now contributes to
|
||||
the candidate total and can push the entire deferred set over budget. The
|
||||
default zero budget does not preload tools.
|
||||
Normal LLM initialization runs `startChat` and resolves the incomplete-bridge
|
||||
fallback before Config marks Workflow mode ready or applies its initial mode,
|
||||
so mode activation does not wait for the first model request.
|
||||
- Old histories containing direct Workflow calls can reveal the tool during
|
||||
replay; `/clear` retains its existing reset behavior.
|
||||
- Named workflows, name-only sessions, host-owned runs and `/review` activation
|
||||
keep their existing registration and execution paths. Deferred calls retain
|
||||
target-tool approval, hooks and subagent restrictions.
|
||||
- No public API, settings, parameter or persistence changes are required. Other
|
||||
tools, context-usage accounting and deployment are outside this change.
|
||||
|
||||
## Verification and acceptance
|
||||
|
||||
Use a real Workflow tool in registry and discovery tests. A default request with
|
||||
workflows enabled must omit its full declaration and retain a short catalog
|
||||
entry. Exact and keyword search must return the unchanged schema; reviewing it
|
||||
must not expand the standing declarations. A reviewed call must execute through
|
||||
the existing bridge, and unreviewed or policy-forbidden calls must remain
|
||||
blocked.
|
||||
|
||||
Check native-deferred workflow intent and mode reminders, visible/revealed
|
||||
tools, CodeModeOnly, permission-deferred lazy factories, schema-budget preload,
|
||||
disabled workflows, missing bridge halves, name-only schemas, history replay,
|
||||
`/clear` and mid-session `/review` activation. Run focused package tests, build,
|
||||
typecheck and bundle.
|
||||
|
||||
Dry-run the headless E2E procedure with the global CLI, then compare actual
|
||||
requests from the exact unmodified base and changed build with identical input
|
||||
and configuration. Report token estimates with a fixed tokenizer separately
|
||||
from provider usage. Mock-provider usage is synthetic and cannot establish
|
||||
billed savings. The acceptance criterion is request-content reduction and a
|
||||
working discovery/invocation path, not a fixed token-saving threshold.
|
||||
@@ -0,0 +1,53 @@
|
||||
# Workflow 工具延迟加载
|
||||
|
||||
[English](deferred-workflow-tool.md) | [简体中文](deferred-workflow-tool.zh-CN.md)
|
||||
|
||||
## 问题
|
||||
|
||||
默认情况下,启用 workflow 后,即使用户没有运行 workflow,Direct 模式请求仍携带完整的
|
||||
Workflow 描述和参数 schema。已有延迟工具桥接机制支持通过 `tool_search` 查阅
|
||||
schema,再通过 `tool_call` 调用,无需改变请求中常驻的函数声明。
|
||||
|
||||
## 决策
|
||||
|
||||
通过 `shouldDefer=true` 将 Workflow 改为默认延迟加载。保持完整描述、参数 schema、
|
||||
注册条件、编写参考和执行行为不变。启动目录保留现有简短描述。明确的 workflow 意图和
|
||||
持久 Workflow 模式也按需发现工具,均不自动展示完整 schema。
|
||||
|
||||
逐轮调用的 `isToolHiddenBehindToolSearch` 应通过注册表的 `isDeferredAndHidden`
|
||||
结果识别工具原生的延迟状态。同时保留 permission-deferred lazy factory 的兜底判断,
|
||||
因为此时工具实例可能尚未构造。保留 CodeModeOnly 排除逻辑:该模式通过 `exec` 调用
|
||||
发现的工具,不使用 Direct 模式的 `tool_call` 桥。保持独立的 Skill 编写参考构造期
|
||||
判断不变。
|
||||
|
||||
## 兼容性和范围
|
||||
|
||||
- `tools.visible` 仍可直接展示 Workflow。与其他延迟工具一致,`tools.eager` 不覆盖
|
||||
工具自身的延迟状态。
|
||||
- 保留现有 schema 预算预加载和桥接工具缺失时的兜底。配置明确要求的延迟与工具原生
|
||||
延迟仍分别处理。
|
||||
非零预加载预算仍采用全有或全无规则:Workflow 现在计入候选总量,可能使整组延迟工具
|
||||
超出预算。默认零预算不预加载工具。
|
||||
正常 LLM 初始化会执行 `startChat` 并完成缺桥兜底,随后 Config 才将 Workflow 模式
|
||||
标记为就绪、应用初始模式,因此模式激活无需等待首个模型请求。
|
||||
- 包含直接 Workflow 调用的旧会话可在回放时展示工具;`/clear` 保持现有重置行为。
|
||||
- 已保存的具名 workflow、name-only 会话、宿主运行和 `/review` 激活保持现有注册及
|
||||
执行路径。延迟调用仍遵守目标工具审批、hooks 和子代理限制。
|
||||
- 无需修改公开 API、配置、参数或持久化格式。其他工具、context-usage 统计和部署不在
|
||||
此次改动范围内。
|
||||
|
||||
## 验证和验收
|
||||
|
||||
使用真实 Workflow 工具进行注册表和发现测试。启用 workflow 的默认请求必须省略完整
|
||||
声明,保留简短目录条目。精确搜索和关键词搜索必须返回原有 schema,查阅后不能扩展
|
||||
常驻声明。查阅后的调用必须通过现有桥接执行,未经查阅或被策略禁止的调用仍须拦截。
|
||||
|
||||
检查原生延迟 workflow 的意图及模式提示、visible/revealed 工具、CodeModeOnly、
|
||||
permission-deferred lazy factory、schema 预算预加载、workflow 禁用、桥接任一端
|
||||
缺失、name-only schema、历史回放、`/clear` 和会话中途 `/review` 激活。执行相关
|
||||
包内测试、build、typecheck 和 bundle。
|
||||
|
||||
先用全局 CLI dry-run 无界面 E2E 流程,再使用完全相同的输入和配置,对比精确未修改
|
||||
基线与修改版构建的实际请求。使用固定 tokenizer 的估算与 Provider 用量分别报告。
|
||||
Mock Provider 的 usage 是合成值,不能证明计费用量收益。验收依据是请求内容减少且
|
||||
发现、调用链路可用,不设固定 token 节省阈值。
|
||||
@@ -77,7 +77,12 @@ describe('DW03: headless prompts remain unknown', () => {
|
||||
const body = JSON.stringify(requests[0].body['messages']);
|
||||
expect(body).toContain('workflow ultracode');
|
||||
expect(body).not.toContain("The user's message includes the");
|
||||
expect(JSON.stringify(requests[0].body['tools'])).toContain('workflow');
|
||||
expect(body).toContain('- \\"workflow\\":');
|
||||
expect(requests[0].body['tools']).not.toContainEqual(
|
||||
expect.objectContaining({
|
||||
function: expect.objectContaining({ name: 'workflow' }),
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
it.each(['text', 'json', 'stream-json'])(
|
||||
|
||||
@@ -575,6 +575,7 @@ describe('headless-owned Mod tool.check restriction', () => {
|
||||
chatRecording: true,
|
||||
workflowsEnabled: true,
|
||||
workflowNameOnly: true,
|
||||
visibleTools: ['workflow'],
|
||||
approvalMode,
|
||||
interactive: approver === 'person',
|
||||
inputFormat:
|
||||
|
||||
@@ -217,20 +217,21 @@ function isToolDeferredBehindToolSearch(config: Config, name: string): boolean {
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a registered tool's schema is withheld from the request right now:
|
||||
* deferred as above and not revealed through ToolSearch yet. For a question
|
||||
* asked again on every turn, such as the Workflow keyword reminder's.
|
||||
* Whether a registered tool's schema is withheld from the Direct-mode request
|
||||
* right now. Used each turn by the Workflow keyword and mode reminders.
|
||||
*
|
||||
* Mirrors `ToolRegistry.isDeferredAndHidden`, which cannot be used here: it
|
||||
* answers false for a tool that is still only a lazy factory, which is exactly
|
||||
* the state while the tool asking the question is being constructed.
|
||||
* The registry covers native deferral, but a permission-deferred lazy factory
|
||||
* has no instance yet and still needs the construction-time fallback.
|
||||
*/
|
||||
export function isToolHiddenBehindToolSearch(
|
||||
config: Config,
|
||||
name: string,
|
||||
): boolean {
|
||||
if (config.getToolMode?.() === ToolMode.CodeModeOnly) return false;
|
||||
const registry = config.getToolRegistry?.();
|
||||
if (registry?.isDeferredAndHidden?.(name)) return true;
|
||||
if (!isToolDeferredBehindToolSearch(config, name)) return false;
|
||||
return !config.getToolRegistry?.()?.isDeferredToolRevealed?.(name);
|
||||
return !registry?.isDeferredToolRevealed?.(name);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { Config, type ConfigParameters } from '../../config/config.js';
|
||||
import { buildDeferredToolsReminder } from '../../core/environmentContext.js';
|
||||
import {
|
||||
resolveWorkflowContext,
|
||||
resolveWorkflowIntent,
|
||||
} from '../../core/workflow-intent.js';
|
||||
import { WorkflowModeContext } from '../../core/workflow-mode.js';
|
||||
import { runWithAgentContext } from '../../agents/runtime/agent-context.js';
|
||||
import { MockTool } from '../../test-utils/mock-tool.js';
|
||||
import { ToolRegistry } from '../tool-registry.js';
|
||||
import { ToolSearchTool } from '../tool-search.js';
|
||||
import { resolveDeferredToolCall, ToolCallTool } from '../tool-call.js';
|
||||
import { ToolErrorType } from '../tool-error.js';
|
||||
import { ToolNames } from '../tool-names.js';
|
||||
import { WorkflowTool } from './workflow.js';
|
||||
|
||||
afterEach(() => vi.restoreAllMocks());
|
||||
|
||||
function fixture(params: Partial<ConfigParameters> = {}) {
|
||||
const config = new Config({
|
||||
cwd: '/tmp',
|
||||
targetDir: '/tmp',
|
||||
model: 'test-model',
|
||||
embeddingModel: 'test-embedding-model',
|
||||
sandbox: undefined,
|
||||
debugMode: false,
|
||||
workflowsEnabled: true,
|
||||
...params,
|
||||
});
|
||||
const registry = new ToolRegistry(config);
|
||||
vi.spyOn(config, 'getToolRegistry').mockReturnValue(registry);
|
||||
const search = new ToolSearchTool(config);
|
||||
registry.registerTool(search);
|
||||
registry.registerTool(new ToolCallTool());
|
||||
registry.registerTool(new MockTool({ name: ToolNames.EXEC }));
|
||||
const workflow = new WorkflowTool(config);
|
||||
registry.registerTool(workflow);
|
||||
const lookup = (query = 'select:workflow') =>
|
||||
search.build({ query }).execute(new AbortController().signal);
|
||||
return { config, registry, workflow, lookup };
|
||||
}
|
||||
|
||||
describe('deferred Workflow discovery', () => {
|
||||
it('keeps only the short catalog entry in a fresh Direct request', () => {
|
||||
const { registry, workflow } = fixture();
|
||||
expect(registry.getFunctionDeclarations()).not.toContainEqual(
|
||||
workflow.schema,
|
||||
);
|
||||
expect(
|
||||
registry.getFunctionDeclarations({ includeDeferred: true }),
|
||||
).toContainEqual(workflow.schema);
|
||||
const reminder = buildDeferredToolsReminder(registry);
|
||||
expect(reminder).toContain('"workflow"');
|
||||
expect(reminder).toContain(workflow.description.split('\n')[0]);
|
||||
expect(reminder).not.toContain('**Runtime**');
|
||||
expect(reminder).not.toContain('parametersJsonSchema');
|
||||
});
|
||||
|
||||
it.each(['select:workflow', 'workflow orchestration'])(
|
||||
'returns the full schema without expanding declarations: %s',
|
||||
async (query) => {
|
||||
const { registry, workflow, lookup } = fixture();
|
||||
const declarations = registry.getFunctionDeclarations();
|
||||
const envelope = {
|
||||
name: 'workflow',
|
||||
arguments: { script: 'return 42;' },
|
||||
};
|
||||
expect(await resolveDeferredToolCall(registry, envelope)).toMatchObject({
|
||||
errorType: ToolErrorType.INVALID_TOOL_PARAMS,
|
||||
error: expect.objectContaining({
|
||||
message: expect.stringContaining('no verified schema review'),
|
||||
}),
|
||||
});
|
||||
const result = await lookup(query);
|
||||
expect(result.error).toBeUndefined();
|
||||
const block = String(result.llmContent).match(
|
||||
/<function>([\s\S]*?)<\/function>/,
|
||||
);
|
||||
expect(block).not.toBeNull();
|
||||
expect(JSON.parse(block![1])).toEqual(workflow.schema);
|
||||
expect(await resolveDeferredToolCall(registry, envelope)).toEqual({
|
||||
tool: workflow,
|
||||
arguments: envelope.arguments,
|
||||
});
|
||||
expect(registry.getFunctionDeclarations()).toEqual(declarations);
|
||||
expect(registry.isDeferredAndHidden('workflow')).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
it('directs native-deferred keyword and mode requests through the bridge', () => {
|
||||
const { config, registry } = fixture();
|
||||
const intent = resolveWorkflowIntent(config, { text: 'workflow' });
|
||||
const context = resolveWorkflowContext(config);
|
||||
const mode = new WorkflowModeContext();
|
||||
mode.setRequested(true);
|
||||
for (const reminder of [
|
||||
intent?.reminder,
|
||||
mode.prepare(context, null).reminder,
|
||||
]) {
|
||||
expect(reminder).toContain('`tool_search`');
|
||||
expect(reminder).toContain('`tool_call`');
|
||||
}
|
||||
expect(registry.isDeferredAndHidden('workflow')).toBe(true);
|
||||
});
|
||||
|
||||
it.each([ToolNames.TOOL_SEARCH, ToolNames.TOOL_CALL])(
|
||||
'does not advertise a hidden tool when %s is missing',
|
||||
(name) => {
|
||||
const { config, registry } = fixture();
|
||||
registry.unregisterTool(name);
|
||||
expect(resolveWorkflowContext(config)).toBeNull();
|
||||
registry.revealDeferredTool('workflow');
|
||||
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
|
||||
},
|
||||
);
|
||||
|
||||
it('honors explicit visibility without suggesting bridge discovery', () => {
|
||||
const { config, registry, workflow } = fixture({
|
||||
visibleTools: ['workflow'],
|
||||
});
|
||||
expect(registry.getFunctionDeclarations()).toContainEqual(workflow.schema);
|
||||
expect(buildDeferredToolsReminder(registry)).toBeNull();
|
||||
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
|
||||
});
|
||||
|
||||
it('preserves budget preload and restores deferral after clearing reveals', () => {
|
||||
const { config, registry, workflow } = fixture();
|
||||
expect(registry.preloadDeferredToolsWithinBudget(0)).toBe(0);
|
||||
expect(registry.preloadDeferredToolsWithinBudget(100_000)).toBe(1);
|
||||
expect(registry.getFunctionDeclarations()).toContainEqual(workflow.schema);
|
||||
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
|
||||
registry.clearRevealedDeferredTools();
|
||||
expect(registry.getFunctionDeclarations()).not.toContainEqual(
|
||||
workflow.schema,
|
||||
);
|
||||
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(true);
|
||||
});
|
||||
|
||||
it('keeps CodeMode discovery behind exec without Direct bridge instructions', async () => {
|
||||
const { config, registry, workflow, lookup } = fixture({
|
||||
codeModeOnly: true,
|
||||
});
|
||||
const declarations = registry.getFunctionDeclarations();
|
||||
expect(JSON.stringify(declarations)).not.toContain('tools.workflow(args:');
|
||||
expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false);
|
||||
const result = await lookup();
|
||||
const block = String(result.llmContent).match(
|
||||
/<function>([\s\S]*?)<\/function>/,
|
||||
);
|
||||
expect(block).not.toBeNull();
|
||||
expect(JSON.parse(block![1])).toMatchObject({
|
||||
...workflow.schema,
|
||||
jsName: 'workflow',
|
||||
});
|
||||
expect(result.llmContent).toContain('through exec');
|
||||
expect(registry.getFunctionDeclarations()).toEqual(declarations);
|
||||
});
|
||||
|
||||
it('retains the named-only schema through discovery', async () => {
|
||||
const { workflow, lookup } = fixture({ workflowNameOnly: true });
|
||||
const result = await lookup();
|
||||
const block = String(result.llmContent).match(
|
||||
/<function>([\s\S]*?)<\/function>/,
|
||||
);
|
||||
expect(JSON.parse(block![1])).toEqual(workflow.schema);
|
||||
expect(workflow.parameterSchema).toHaveProperty('properties.name');
|
||||
expect(workflow.parameterSchema).not.toHaveProperty('properties.script');
|
||||
expect(() => workflow.build({ script: 'return 42;' })).toThrow(
|
||||
'named workflows',
|
||||
);
|
||||
});
|
||||
|
||||
it('does not let a reviewed deferred Workflow bypass subagent restrictions', async () => {
|
||||
const { registry, lookup } = fixture();
|
||||
await lookup();
|
||||
const result = await runWithAgentContext('child', () =>
|
||||
resolveDeferredToolCall(registry, {
|
||||
name: 'workflow',
|
||||
arguments: { script: 'return 42;' },
|
||||
}),
|
||||
);
|
||||
expect(result).toMatchObject({ errorType: ToolErrorType.EXECUTION_DENIED });
|
||||
expect(result).not.toHaveProperty('tool');
|
||||
});
|
||||
|
||||
it('keeps mid-session review activation deferred and discoverable', async () => {
|
||||
const { config, registry, lookup } = fixture({
|
||||
workflowsEnabled: undefined,
|
||||
});
|
||||
registry.unregisterTool('workflow');
|
||||
const declarations = registry.getFunctionDeclarations();
|
||||
const setTools = vi.fn(async () => registry.warmAll());
|
||||
vi.spyOn(config, 'getLlmClient').mockReturnValue({ setTools } as never);
|
||||
await config.enableReviewWorkflow();
|
||||
expect(config.isWorkflowsEnabled()).toBe(true);
|
||||
expect(setTools).toHaveBeenCalledOnce();
|
||||
expect(registry.isDeferredAndHidden('workflow')).toBe(true);
|
||||
expect((await lookup()).llmContent).toContain('"name":"workflow"');
|
||||
expect(registry.getFunctionDeclarations()).toEqual(declarations);
|
||||
});
|
||||
});
|
||||
@@ -1806,6 +1806,7 @@ export class WorkflowTool extends BaseDeclarativeTool<
|
||||
buildWorkflowParamSchema(surface, nameOnly),
|
||||
/* isOutputMarkdown */ true,
|
||||
/* canUpdateOutput */ true,
|
||||
/* shouldDefer */ true,
|
||||
);
|
||||
this.authoringSurface = surface;
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user