diff --git a/docs/design/deferred-workflow-tool.md b/docs/design/deferred-workflow-tool.md new file mode 100644 index 00000000000..ca344f67422 --- /dev/null +++ b/docs/design/deferred-workflow-tool.md @@ -0,0 +1,68 @@ +# Deferred Workflow tool + +[English](deferred-workflow-tool.md) | [简体中文](deferred-workflow-tool.zh-CN.md) + +## Problem + +By default, when workflows are enabled, Direct-mode requests carry the complete +Workflow description and parameter schema, even when the user never runs a +workflow. The existing deferred-tool bridge already supports discovering a +tool's schema with `tool_search` and invoking it with `tool_call` without +changing the request's standing function declarations. + +## Decision + +Make Workflow deferred by default with `shouldDefer=true`. Keep its full +description, parameter schema, registration gates, authoring reference and +execution behavior unchanged. The startup catalog retains the existing short +description. Explicit workflow intent and persistent Workflow mode still use +on-demand discovery; neither automatically reveals the schema. + +The per-turn `isToolHiddenBehindToolSearch` helper must recognize native +deferred tools through the registry's `isDeferredAndHidden` result. Retain its +permission-deferred lazy-factory fallback, because a factory need not have a +tool instance yet. Preserve the CodeModeOnly exclusion: that mode invokes +discovered tools through `exec`, not the Direct-mode `tool_call` bridge. Do not +change the separate construction-time Skill reference decision. + +## Compatibility and scope + +- `tools.visible` can still expose Workflow directly. `tools.eager` does not + override a tool's native deferral, consistent with other deferred tools. +- The existing schema-budget preload and incomplete-bridge fallback still + apply. An explicit permission deferral remains distinct from native deferral. + A nonzero preload budget remains all-or-nothing: Workflow now contributes to + the candidate total and can push the entire deferred set over budget. The + default zero budget does not preload tools. + Normal LLM initialization runs `startChat` and resolves the incomplete-bridge + fallback before Config marks Workflow mode ready or applies its initial mode, + so mode activation does not wait for the first model request. +- Old histories containing direct Workflow calls can reveal the tool during + replay; `/clear` retains its existing reset behavior. +- Named workflows, name-only sessions, host-owned runs and `/review` activation + keep their existing registration and execution paths. Deferred calls retain + target-tool approval, hooks and subagent restrictions. +- No public API, settings, parameter or persistence changes are required. Other + tools, context-usage accounting and deployment are outside this change. + +## Verification and acceptance + +Use a real Workflow tool in registry and discovery tests. A default request with +workflows enabled must omit its full declaration and retain a short catalog +entry. Exact and keyword search must return the unchanged schema; reviewing it +must not expand the standing declarations. A reviewed call must execute through +the existing bridge, and unreviewed or policy-forbidden calls must remain +blocked. + +Check native-deferred workflow intent and mode reminders, visible/revealed +tools, CodeModeOnly, permission-deferred lazy factories, schema-budget preload, +disabled workflows, missing bridge halves, name-only schemas, history replay, +`/clear` and mid-session `/review` activation. Run focused package tests, build, +typecheck and bundle. + +Dry-run the headless E2E procedure with the global CLI, then compare actual +requests from the exact unmodified base and changed build with identical input +and configuration. Report token estimates with a fixed tokenizer separately +from provider usage. Mock-provider usage is synthetic and cannot establish +billed savings. The acceptance criterion is request-content reduction and a +working discovery/invocation path, not a fixed token-saving threshold. diff --git a/docs/design/deferred-workflow-tool.zh-CN.md b/docs/design/deferred-workflow-tool.zh-CN.md new file mode 100644 index 00000000000..efa46592b33 --- /dev/null +++ b/docs/design/deferred-workflow-tool.zh-CN.md @@ -0,0 +1,53 @@ +# Workflow 工具延迟加载 + +[English](deferred-workflow-tool.md) | [简体中文](deferred-workflow-tool.zh-CN.md) + +## 问题 + +默认情况下,启用 workflow 后,即使用户没有运行 workflow,Direct 模式请求仍携带完整的 +Workflow 描述和参数 schema。已有延迟工具桥接机制支持通过 `tool_search` 查阅 +schema,再通过 `tool_call` 调用,无需改变请求中常驻的函数声明。 + +## 决策 + +通过 `shouldDefer=true` 将 Workflow 改为默认延迟加载。保持完整描述、参数 schema、 +注册条件、编写参考和执行行为不变。启动目录保留现有简短描述。明确的 workflow 意图和 +持久 Workflow 模式也按需发现工具,均不自动展示完整 schema。 + +逐轮调用的 `isToolHiddenBehindToolSearch` 应通过注册表的 `isDeferredAndHidden` +结果识别工具原生的延迟状态。同时保留 permission-deferred lazy factory 的兜底判断, +因为此时工具实例可能尚未构造。保留 CodeModeOnly 排除逻辑:该模式通过 `exec` 调用 +发现的工具,不使用 Direct 模式的 `tool_call` 桥。保持独立的 Skill 编写参考构造期 +判断不变。 + +## 兼容性和范围 + +- `tools.visible` 仍可直接展示 Workflow。与其他延迟工具一致,`tools.eager` 不覆盖 + 工具自身的延迟状态。 +- 保留现有 schema 预算预加载和桥接工具缺失时的兜底。配置明确要求的延迟与工具原生 + 延迟仍分别处理。 + 非零预加载预算仍采用全有或全无规则:Workflow 现在计入候选总量,可能使整组延迟工具 + 超出预算。默认零预算不预加载工具。 + 正常 LLM 初始化会执行 `startChat` 并完成缺桥兜底,随后 Config 才将 Workflow 模式 + 标记为就绪、应用初始模式,因此模式激活无需等待首个模型请求。 +- 包含直接 Workflow 调用的旧会话可在回放时展示工具;`/clear` 保持现有重置行为。 +- 已保存的具名 workflow、name-only 会话、宿主运行和 `/review` 激活保持现有注册及 + 执行路径。延迟调用仍遵守目标工具审批、hooks 和子代理限制。 +- 无需修改公开 API、配置、参数或持久化格式。其他工具、context-usage 统计和部署不在 + 此次改动范围内。 + +## 验证和验收 + +使用真实 Workflow 工具进行注册表和发现测试。启用 workflow 的默认请求必须省略完整 +声明,保留简短目录条目。精确搜索和关键词搜索必须返回原有 schema,查阅后不能扩展 +常驻声明。查阅后的调用必须通过现有桥接执行,未经查阅或被策略禁止的调用仍须拦截。 + +检查原生延迟 workflow 的意图及模式提示、visible/revealed 工具、CodeModeOnly、 +permission-deferred lazy factory、schema 预算预加载、workflow 禁用、桥接任一端 +缺失、name-only schema、历史回放、`/clear` 和会话中途 `/review` 激活。执行相关 +包内测试、build、typecheck 和 bundle。 + +先用全局 CLI dry-run 无界面 E2E 流程,再使用完全相同的输入和配置,对比精确未修改 +基线与修改版构建的实际请求。使用固定 tokenizer 的估算与 Provider 用量分别报告。 +Mock Provider 的 usage 是合成值,不能证明计费用量收益。验收依据是请求内容减少且 +发现、调用链路可用,不设固定 token 节省阈值。 diff --git a/integration-tests/cli/workflow-intent.test.ts b/integration-tests/cli/workflow-intent.test.ts index 4870ff63342..36380b2baf2 100644 --- a/integration-tests/cli/workflow-intent.test.ts +++ b/integration-tests/cli/workflow-intent.test.ts @@ -77,7 +77,12 @@ describe('DW03: headless prompts remain unknown', () => { const body = JSON.stringify(requests[0].body['messages']); expect(body).toContain('workflow ultracode'); expect(body).not.toContain("The user's message includes the"); - expect(JSON.stringify(requests[0].body['tools'])).toContain('workflow'); + expect(body).toContain('- \\"workflow\\":'); + expect(requests[0].body['tools']).not.toContainEqual( + expect.objectContaining({ + function: expect.objectContaining({ name: 'workflow' }), + }), + ); }; it.each(['text', 'json', 'stream-json'])( diff --git a/packages/core/src/core/mod-tool-check.test.ts b/packages/core/src/core/mod-tool-check.test.ts index 99205ffe696..bff89cdb043 100644 --- a/packages/core/src/core/mod-tool-check.test.ts +++ b/packages/core/src/core/mod-tool-check.test.ts @@ -575,6 +575,7 @@ describe('headless-owned Mod tool.check restriction', () => { chatRecording: true, workflowsEnabled: true, workflowNameOnly: true, + visibleTools: ['workflow'], approvalMode, interactive: approver === 'person', inputFormat: diff --git a/packages/core/src/skills/bundled-reference.ts b/packages/core/src/skills/bundled-reference.ts index 4f3d6ea97fc..093fb6df6f1 100644 --- a/packages/core/src/skills/bundled-reference.ts +++ b/packages/core/src/skills/bundled-reference.ts @@ -217,20 +217,21 @@ function isToolDeferredBehindToolSearch(config: Config, name: string): boolean { } /** - * Whether a registered tool's schema is withheld from the request right now: - * deferred as above and not revealed through ToolSearch yet. For a question - * asked again on every turn, such as the Workflow keyword reminder's. + * Whether a registered tool's schema is withheld from the Direct-mode request + * right now. Used each turn by the Workflow keyword and mode reminders. * - * Mirrors `ToolRegistry.isDeferredAndHidden`, which cannot be used here: it - * answers false for a tool that is still only a lazy factory, which is exactly - * the state while the tool asking the question is being constructed. + * The registry covers native deferral, but a permission-deferred lazy factory + * has no instance yet and still needs the construction-time fallback. */ export function isToolHiddenBehindToolSearch( config: Config, name: string, ): boolean { + if (config.getToolMode?.() === ToolMode.CodeModeOnly) return false; + const registry = config.getToolRegistry?.(); + if (registry?.isDeferredAndHidden?.(name)) return true; if (!isToolDeferredBehindToolSearch(config, name)) return false; - return !config.getToolRegistry?.()?.isDeferredToolRevealed?.(name); + return !registry?.isDeferredToolRevealed?.(name); } /** diff --git a/packages/core/src/tools/workflow/workflow-deferred.test.ts b/packages/core/src/tools/workflow/workflow-deferred.test.ts new file mode 100644 index 00000000000..479a164cfca --- /dev/null +++ b/packages/core/src/tools/workflow/workflow-deferred.test.ts @@ -0,0 +1,208 @@ +/** + * @license + * Copyright 2026 Qwen Team + * SPDX-License-Identifier: Apache-2.0 + */ + +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { Config, type ConfigParameters } from '../../config/config.js'; +import { buildDeferredToolsReminder } from '../../core/environmentContext.js'; +import { + resolveWorkflowContext, + resolveWorkflowIntent, +} from '../../core/workflow-intent.js'; +import { WorkflowModeContext } from '../../core/workflow-mode.js'; +import { runWithAgentContext } from '../../agents/runtime/agent-context.js'; +import { MockTool } from '../../test-utils/mock-tool.js'; +import { ToolRegistry } from '../tool-registry.js'; +import { ToolSearchTool } from '../tool-search.js'; +import { resolveDeferredToolCall, ToolCallTool } from '../tool-call.js'; +import { ToolErrorType } from '../tool-error.js'; +import { ToolNames } from '../tool-names.js'; +import { WorkflowTool } from './workflow.js'; + +afterEach(() => vi.restoreAllMocks()); + +function fixture(params: Partial = {}) { + const config = new Config({ + cwd: '/tmp', + targetDir: '/tmp', + model: 'test-model', + embeddingModel: 'test-embedding-model', + sandbox: undefined, + debugMode: false, + workflowsEnabled: true, + ...params, + }); + const registry = new ToolRegistry(config); + vi.spyOn(config, 'getToolRegistry').mockReturnValue(registry); + const search = new ToolSearchTool(config); + registry.registerTool(search); + registry.registerTool(new ToolCallTool()); + registry.registerTool(new MockTool({ name: ToolNames.EXEC })); + const workflow = new WorkflowTool(config); + registry.registerTool(workflow); + const lookup = (query = 'select:workflow') => + search.build({ query }).execute(new AbortController().signal); + return { config, registry, workflow, lookup }; +} + +describe('deferred Workflow discovery', () => { + it('keeps only the short catalog entry in a fresh Direct request', () => { + const { registry, workflow } = fixture(); + expect(registry.getFunctionDeclarations()).not.toContainEqual( + workflow.schema, + ); + expect( + registry.getFunctionDeclarations({ includeDeferred: true }), + ).toContainEqual(workflow.schema); + const reminder = buildDeferredToolsReminder(registry); + expect(reminder).toContain('"workflow"'); + expect(reminder).toContain(workflow.description.split('\n')[0]); + expect(reminder).not.toContain('**Runtime**'); + expect(reminder).not.toContain('parametersJsonSchema'); + }); + + it.each(['select:workflow', 'workflow orchestration'])( + 'returns the full schema without expanding declarations: %s', + async (query) => { + const { registry, workflow, lookup } = fixture(); + const declarations = registry.getFunctionDeclarations(); + const envelope = { + name: 'workflow', + arguments: { script: 'return 42;' }, + }; + expect(await resolveDeferredToolCall(registry, envelope)).toMatchObject({ + errorType: ToolErrorType.INVALID_TOOL_PARAMS, + error: expect.objectContaining({ + message: expect.stringContaining('no verified schema review'), + }), + }); + const result = await lookup(query); + expect(result.error).toBeUndefined(); + const block = String(result.llmContent).match( + /([\s\S]*?)<\/function>/, + ); + expect(block).not.toBeNull(); + expect(JSON.parse(block![1])).toEqual(workflow.schema); + expect(await resolveDeferredToolCall(registry, envelope)).toEqual({ + tool: workflow, + arguments: envelope.arguments, + }); + expect(registry.getFunctionDeclarations()).toEqual(declarations); + expect(registry.isDeferredAndHidden('workflow')).toBe(true); + }, + ); + + it('directs native-deferred keyword and mode requests through the bridge', () => { + const { config, registry } = fixture(); + const intent = resolveWorkflowIntent(config, { text: 'workflow' }); + const context = resolveWorkflowContext(config); + const mode = new WorkflowModeContext(); + mode.setRequested(true); + for (const reminder of [ + intent?.reminder, + mode.prepare(context, null).reminder, + ]) { + expect(reminder).toContain('`tool_search`'); + expect(reminder).toContain('`tool_call`'); + } + expect(registry.isDeferredAndHidden('workflow')).toBe(true); + }); + + it.each([ToolNames.TOOL_SEARCH, ToolNames.TOOL_CALL])( + 'does not advertise a hidden tool when %s is missing', + (name) => { + const { config, registry } = fixture(); + registry.unregisterTool(name); + expect(resolveWorkflowContext(config)).toBeNull(); + registry.revealDeferredTool('workflow'); + expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false); + }, + ); + + it('honors explicit visibility without suggesting bridge discovery', () => { + const { config, registry, workflow } = fixture({ + visibleTools: ['workflow'], + }); + expect(registry.getFunctionDeclarations()).toContainEqual(workflow.schema); + expect(buildDeferredToolsReminder(registry)).toBeNull(); + expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false); + }); + + it('preserves budget preload and restores deferral after clearing reveals', () => { + const { config, registry, workflow } = fixture(); + expect(registry.preloadDeferredToolsWithinBudget(0)).toBe(0); + expect(registry.preloadDeferredToolsWithinBudget(100_000)).toBe(1); + expect(registry.getFunctionDeclarations()).toContainEqual(workflow.schema); + expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false); + registry.clearRevealedDeferredTools(); + expect(registry.getFunctionDeclarations()).not.toContainEqual( + workflow.schema, + ); + expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(true); + }); + + it('keeps CodeMode discovery behind exec without Direct bridge instructions', async () => { + const { config, registry, workflow, lookup } = fixture({ + codeModeOnly: true, + }); + const declarations = registry.getFunctionDeclarations(); + expect(JSON.stringify(declarations)).not.toContain('tools.workflow(args:'); + expect(resolveWorkflowContext(config)?.bridgeWorkflowTool).toBe(false); + const result = await lookup(); + const block = String(result.llmContent).match( + /([\s\S]*?)<\/function>/, + ); + expect(block).not.toBeNull(); + expect(JSON.parse(block![1])).toMatchObject({ + ...workflow.schema, + jsName: 'workflow', + }); + expect(result.llmContent).toContain('through exec'); + expect(registry.getFunctionDeclarations()).toEqual(declarations); + }); + + it('retains the named-only schema through discovery', async () => { + const { workflow, lookup } = fixture({ workflowNameOnly: true }); + const result = await lookup(); + const block = String(result.llmContent).match( + /([\s\S]*?)<\/function>/, + ); + expect(JSON.parse(block![1])).toEqual(workflow.schema); + expect(workflow.parameterSchema).toHaveProperty('properties.name'); + expect(workflow.parameterSchema).not.toHaveProperty('properties.script'); + expect(() => workflow.build({ script: 'return 42;' })).toThrow( + 'named workflows', + ); + }); + + it('does not let a reviewed deferred Workflow bypass subagent restrictions', async () => { + const { registry, lookup } = fixture(); + await lookup(); + const result = await runWithAgentContext('child', () => + resolveDeferredToolCall(registry, { + name: 'workflow', + arguments: { script: 'return 42;' }, + }), + ); + expect(result).toMatchObject({ errorType: ToolErrorType.EXECUTION_DENIED }); + expect(result).not.toHaveProperty('tool'); + }); + + it('keeps mid-session review activation deferred and discoverable', async () => { + const { config, registry, lookup } = fixture({ + workflowsEnabled: undefined, + }); + registry.unregisterTool('workflow'); + const declarations = registry.getFunctionDeclarations(); + const setTools = vi.fn(async () => registry.warmAll()); + vi.spyOn(config, 'getLlmClient').mockReturnValue({ setTools } as never); + await config.enableReviewWorkflow(); + expect(config.isWorkflowsEnabled()).toBe(true); + expect(setTools).toHaveBeenCalledOnce(); + expect(registry.isDeferredAndHidden('workflow')).toBe(true); + expect((await lookup()).llmContent).toContain('"name":"workflow"'); + expect(registry.getFunctionDeclarations()).toEqual(declarations); + }); +}); diff --git a/packages/core/src/tools/workflow/workflow.ts b/packages/core/src/tools/workflow/workflow.ts index c32ad17c60f..24807d4add3 100644 --- a/packages/core/src/tools/workflow/workflow.ts +++ b/packages/core/src/tools/workflow/workflow.ts @@ -1806,6 +1806,7 @@ export class WorkflowTool extends BaseDeclarativeTool< buildWorkflowParamSchema(surface, nameOnly), /* isOutputMarkdown */ true, /* canUpdateOutput */ true, + /* shouldDefer */ true, ); this.authoringSurface = surface; }