mirror of
https://github.com/MervinPraison/PraisonAI.git
synced 2026-11-07 17:33:33 -10:00
feat(ts): honour all 21 Agent and Agent.chat options, and keep them off the webview graph (#4841)
* feat(ts): honour all 21 Agent and Agent.chat options, and keep them off the webview graph Agent accepted 15 constructor options and 6 chat options and acted on none of them. All 21 now work: reflection's critique loop, templates, tool search, toolsets, learn, runtime, backend, runOn, toolsRunOn, sandbox, autonomy, messageSteering, selfImprove, toolConfig, auth; and per call attachments, reasoningSteps, taskName, taskDescription, taskId and config. Five feature modules already existed -- reflection, templates, autonomy, learn, tool-config -- complete and with no importer anywhere in the repo. They are called now. Three options resolve through registries a host fills in: runOn, toolsRunOn and a non-subprocess sandbox. Naming an unregistered one THROWS rather than falling back to running locally. That is a behaviour change and it is the parity-correct one: placement.py:118-150 raises TypeError for exactly these cases, and its own comment gives the reason -- "A typo here used to survive construction and only surface on the first model turn -- after the prompt was built and, for a real run, after the API spend." The webview gate caught a regression the feature work introduced, which is what it is for. Four Node builtins reached the Agent's browser graph -- child_process, fs, os and path -- through the sandbox runner and the learn store, and two require() calls earned their files the esm-shim createRequire banner, a top-level await esbuild refuses at the chrome58 floor the Android app ships against. Left alone this is issue #4437 again: a blank screen at import time with no error boundary. Fixed the way this package already does it. The sandbox loads its builtins through computed specifiers; the learn manager is built on first use instead of in the constructor, behind a type-only import; the attachment reader and the global-tool registry lookup became async dynamic imports. require() is not enough for either, because the gate counts a require-call as a static import. The gate now reports OK for all four entries including the floor check, and the mobile bundle is unchanged at 1528.0kB of its 1600kB budget. Two options become partial rather than closed, on the AI SDK path only: attachments, because that adapter converts user messages to text, and toolConfig.parallel, because Python's parallel_tool_calls is a request field that needs the fetch wrapper. Same shape as the existing seed and reasoningEffort rows. Also rewritten: a test asserting the exact list of ignored chat options, which had to change every time one was closed. It derives its expectation from the ledger now. Behaviour parity: 76 options outstanding, now 55. 77 new tests; 2,112 passing. Co-Authored-By: Claude Opus 5 <[email protected]> * fix(ts): repair four agent-feature defects flagged in PR #4841 review - sandbox: resolve process.execPath lazily via globalThis so the Agent browser/mobile graph no longer evaluates a Node global at import time (#4437) - sandbox: honour workingDir and persistFiles — run the child with cwd set to the workspace and reuse it across calls; never delete a persisted/user dir - learn: base the periodic nudge on the previous turn's tool count, captured before the per-turn reset, so nudgeMinToolIters can actually be satisfied - auth: forward the wrapped fetch (carrying subscription headers) through getBackend -> resolveBackend into the AI SDK provider options Adds unit tests for the sandbox workspace/persist path and the nudge threshold. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Mervin Praison <[email protected]> --------- Co-authored-by: Claude Opus 5 <[email protected]> Co-authored-by: praisonai-triage-agent[bot] <272766704+praisonai-triage-agent[bot]@users.noreply.github.com> Co-authored-by: Mervin Praison <[email protected]>
This commit is contained in:
20 files changed
+4010
-130
No files matched your search
@@ -17,47 +17,18 @@ The check is a ratchet: the total may fall, never rise.
|
||||
|
||||
| Surface | Options not yet acted on |
|
||||
|---|---|
|
||||
| `Agent.__init__` | 15 |
|
||||
| `Agent.chat` | 6 |
|
||||
| `AgentTeam.__init__` | 15 |
|
||||
| `Handoff` | 8 |
|
||||
| `Task.__init__` | 32 |
|
||||
| **Total** | **76** |
|
||||
| **Total** | **55** |
|
||||
|
||||
Plus 10 options that work for some inputs and announce themselves for the rest.
|
||||
Plus 12 options that work for some inputs and announce themselves for the rest.
|
||||
|
||||
## The queue
|
||||
|
||||
Each row is one unit of work: implement it, delete its entry from the ledger,
|
||||
add a test proving the option changes what the code does, and regenerate.
|
||||
|
||||
### `Agent.__init__` (15)
|
||||
|
||||
- `auth`
|
||||
- `toolsets`
|
||||
- `reflection`
|
||||
- `autonomy`
|
||||
- `templates`
|
||||
- `selfImprove`
|
||||
- `toolConfig`
|
||||
- `learn`
|
||||
- `backend`
|
||||
- `runOn`
|
||||
- `toolsRunOn`
|
||||
- `runtime`
|
||||
- `toolSearch`
|
||||
- `messageSteering`
|
||||
- `sandbox`
|
||||
|
||||
### `Agent.chat` (6)
|
||||
|
||||
- `reasoningSteps`
|
||||
- `taskName`
|
||||
- `taskDescription`
|
||||
- `taskId`
|
||||
- `config`
|
||||
- `attachments`
|
||||
|
||||
### `AgentTeam.__init__` (15)
|
||||
|
||||
- `managerLlm`
|
||||
@@ -131,7 +102,9 @@ add a test proving the option changes what the code does, and regenerate.
|
||||
| `Agent` | `knowledge` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `Agent` | `memory` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `Agent` | `reasoningEffort` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `Agent` | `toolConfig` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `Agent` | `web` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `Agent.chat` | `attachments` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `Agent.chat` | `outputPydantic` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `Agent.chat` | `seed` | `src/praisonai-ts/src/agent/simple.ts` |
|
||||
| `ChromaMemory` | `ragDbPath` | `src/praisonai-ts/src/memory/adapters.ts` |
|
||||
|
||||
@@ -42,7 +42,7 @@ This complements `PARITY.md`, which only tracks whether an export exists.
|
||||
### `Agent.__init__`
|
||||
|
||||
- Python: `src/praisonai-agents/praisonaiagents/agent/agent.py:609`
|
||||
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:113` (ctor `src/praisonai-ts/src/agent/simple.ts:753`)
|
||||
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:196` (ctor `src/praisonai-ts/src/agent/simple.ts:900`)
|
||||
- Counts: 42 python params: 29 exact, 8 camelCase, 3 alias, 2 flattened, 0 missing; 6 mismatches; 6 waived; 14 TS-only of 59
|
||||
|
||||
| Python param | Kind | Py default | Py type | Match | TS name | TS default | TS type | Status |
|
||||
@@ -53,7 +53,7 @@ This complements `PARITY.md`, which only tracks whether an export exists.
|
||||
| `backstory` | positional | null | `Optional[str]` | exact | `backstory` | `config.instructions` | `string` | default mismatch (waived) |
|
||||
| `instructions` | positional | null | `Optional[str]` | exact | `instructions` | undefined | `string` | ok |
|
||||
| `llm` | positional | null | `Optional[Union[str, Any]]` | exact | `llm` | undefined | `string \| LLMConfig` | ok |
|
||||
| `model` | positional | null | `Optional[Union[str, Any]]` | exact | `model` | `llmName \|\| resolveDefaultModel()` | `string` | default mismatch (waived) |
|
||||
| `model` | positional | null | `Optional[Union[str, Any]]` | exact | `model` | `llmName \|\| authModel \|\| resolveDefaultModel()` | `string` | default mismatch (waived) |
|
||||
| `base_url` | positional | null | `Optional[str]` | alias | `baseURL` | undefined | `string` | ok |
|
||||
| `api_key` | positional | null | `Optional[str]` | camelCase | `apiKey` | undefined | `string` | ok |
|
||||
| `auth` | positional | null | `Optional[str]` | exact | `auth` | undefined | `string` | ok |
|
||||
@@ -201,7 +201,7 @@ TS-only members: `dependencies`?
|
||||
### `Agent.start`
|
||||
|
||||
- Python: `src/praisonai-agents/praisonaiagents/agent/execution_mixin.py:832`
|
||||
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:1810`
|
||||
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:2305`
|
||||
- Counts: 1 python params: 1 exact, 0 camelCase, 0 alias, 0 flattened, 0 missing; 0 mismatches; 0 waived; 20 TS-only of 21
|
||||
|
||||
| Python param | Kind | Py default | Py type | Match | TS name | TS default | TS type | Status |
|
||||
@@ -213,7 +213,7 @@ TS-only members: `previousResult`?, `onToken`?, `signal`?, `onEvent`?, `options`
|
||||
### `Agent.chat`
|
||||
|
||||
- Python: `src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3101`
|
||||
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:2457`
|
||||
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:3338`
|
||||
- Counts: 17 python params: 7 exact, 9 camelCase, 1 alias, 0 flattened, 0 missing; 0 mismatches; 0 waived; 2 TS-only of 19
|
||||
|
||||
| Python param | Kind | Py default | Py type | Match | TS name | TS default | TS type | Status |
|
||||
|
||||
@@ -25,11 +25,21 @@
|
||||
"option": "reasoningEffort",
|
||||
"surface": "Agent"
|
||||
},
|
||||
{
|
||||
"file": "src/praisonai-ts/src/agent/simple.ts",
|
||||
"option": "toolConfig",
|
||||
"surface": "Agent"
|
||||
},
|
||||
{
|
||||
"file": "src/praisonai-ts/src/agent/simple.ts",
|
||||
"option": "web",
|
||||
"surface": "Agent"
|
||||
},
|
||||
{
|
||||
"file": "src/praisonai-ts/src/agent/simple.ts",
|
||||
"option": "attachments",
|
||||
"surface": "Agent.chat"
|
||||
},
|
||||
{
|
||||
"file": "src/praisonai-ts/src/agent/simple.ts",
|
||||
"option": "outputPydantic",
|
||||
@@ -52,31 +62,6 @@
|
||||
}
|
||||
],
|
||||
"surfaces": {
|
||||
"Agent.__init__": [
|
||||
"auth",
|
||||
"autonomy",
|
||||
"backend",
|
||||
"learn",
|
||||
"messageSteering",
|
||||
"reflection",
|
||||
"runOn",
|
||||
"runtime",
|
||||
"sandbox",
|
||||
"selfImprove",
|
||||
"templates",
|
||||
"toolConfig",
|
||||
"toolSearch",
|
||||
"toolsRunOn",
|
||||
"toolsets"
|
||||
],
|
||||
"Agent.chat": [
|
||||
"attachments",
|
||||
"config",
|
||||
"reasoningSteps",
|
||||
"taskDescription",
|
||||
"taskId",
|
||||
"taskName"
|
||||
],
|
||||
"AgentTeam.__init__": [
|
||||
"autonomy",
|
||||
"caching",
|
||||
@@ -139,5 +124,5 @@
|
||||
"when"
|
||||
]
|
||||
},
|
||||
"total": 76
|
||||
"total": 55
|
||||
}
|
||||
@@ -182,7 +182,7 @@
|
||||
"default": null,
|
||||
"default_kind": "literal",
|
||||
"gap": {
|
||||
"detail": "default differs: python=null typescript=`llmName || resolveDefaultModel()` (TS `model`)",
|
||||
"detail": "default differs: python=null typescript=`llmName || authModel || resolveDefaultModel()` (TS `model`)",
|
||||
"kind": "default"
|
||||
},
|
||||
"kind": "positional",
|
||||
@@ -191,7 +191,7 @@
|
||||
"required": false,
|
||||
"ts": {
|
||||
"canonical": "model",
|
||||
"default": "llmName || resolveDefaultModel()",
|
||||
"default": "llmName || authModel || resolveDefaultModel()",
|
||||
"default_kind": "expr",
|
||||
"kind": "property",
|
||||
"name": "model",
|
||||
@@ -1221,8 +1221,8 @@
|
||||
}
|
||||
],
|
||||
"typescript": {
|
||||
"ctor_location": "src/praisonai-ts/src/agent/simple.ts:753",
|
||||
"location": "src/praisonai-ts/src/agent/simple.ts:113"
|
||||
"ctor_location": "src/praisonai-ts/src/agent/simple.ts:900",
|
||||
"location": "src/praisonai-ts/src/agent/simple.ts:196"
|
||||
}
|
||||
},
|
||||
"Agent.chat": {
|
||||
@@ -1692,7 +1692,7 @@
|
||||
}
|
||||
],
|
||||
"typescript": {
|
||||
"location": "src/praisonai-ts/src/agent/simple.ts:2457",
|
||||
"location": "src/praisonai-ts/src/agent/simple.ts:3338",
|
||||
"options_interfaces": [
|
||||
"options: AgentChatOptions"
|
||||
],
|
||||
@@ -1946,7 +1946,7 @@
|
||||
}
|
||||
],
|
||||
"typescript": {
|
||||
"location": "src/praisonai-ts/src/agent/simple.ts:1810",
|
||||
"location": "src/praisonai-ts/src/agent/simple.ts:2305",
|
||||
"options_interfaces": [
|
||||
"options: AgentChatOptions"
|
||||
],
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
/**
|
||||
* Subscription auth on an Agent (Python parity: `Agent(auth="claude-code")`,
|
||||
* `agent/agent.py` lines 2087-2100 and `_AUTH_DEFAULT_MODELS`, over
|
||||
* `praisonaiagents/auth`).
|
||||
*
|
||||
* `auth` says "bill this run against my subscription seat, not my API key".
|
||||
* Two consequences follow, and both matter:
|
||||
*
|
||||
* 1. **It fails at construction, never at request time.** A user who asked to
|
||||
* be billed against a subscription must not be silently downgraded to
|
||||
* API-key billing, so an unknown or unregistered provider throws before the
|
||||
* agent exists — where the error is about the option, not about a request.
|
||||
* 2. **It pins the vendor.** A subscription seat belongs to one vendor, so the
|
||||
* OpenAI default model would ship the provider's OAuth token to the wrong
|
||||
* endpoint. When the caller gave no model, `auth` chooses one.
|
||||
*
|
||||
* The credential stores themselves (a Claude Code keychain entry, a Codex CLI
|
||||
* file) are host concerns and are not read here: a provider is registered with
|
||||
* `registerAuthProvider` from `src/llm`, which is also what Python's
|
||||
* `register_subscription_provider` is.
|
||||
*/
|
||||
|
||||
import { listAuthProviders, resolveAuth, type LLMAuth } from '../../llm';
|
||||
|
||||
/**
|
||||
* Default model per subscription provider, used only when the caller gave
|
||||
* `auth` without a model (Python `_AUTH_DEFAULT_MODELS`).
|
||||
*/
|
||||
export const AUTH_DEFAULT_MODELS: Readonly<Record<string, string>> = Object.freeze({
|
||||
'claude-code': 'anthropic/claude-sonnet-4-5',
|
||||
'qwen-cli': 'openai/qwen3-coder-plus',
|
||||
});
|
||||
|
||||
/**
|
||||
* Fail now if `name` names no registered provider (Python
|
||||
* `resolve_subscription_credentials`, which raises `AuthError` at
|
||||
* construction for exactly this reason).
|
||||
*/
|
||||
export function assertAuthProviderRegistered(name: string): void {
|
||||
const registered = listAuthProviders();
|
||||
if (registered.includes(name)) return;
|
||||
throw new Error(
|
||||
`Unknown subscription provider '${name}'. Registered: ${registered.length > 0 ? registered.join(', ') : '(none)'}.\n` +
|
||||
` A subscription store is a host concern: register one with registerAuthProvider('${name}', resolver).\n` +
|
||||
` Passing auth= and falling back to API-key billing would charge the wrong account, so this fails here rather than at request time.`
|
||||
);
|
||||
}
|
||||
|
||||
/** The model `auth` implies when the caller named none. */
|
||||
export function authDefaultModel(name: string): string | undefined {
|
||||
return AUTH_DEFAULT_MODELS[name];
|
||||
}
|
||||
|
||||
/** Resolve the provider's credentials (Python's `resolve_credentials()`). */
|
||||
export async function resolveAgentAuth(name: string): Promise<LLMAuth> {
|
||||
return resolveAuth(name);
|
||||
}
|
||||
|
||||
/**
|
||||
* Wrap `fetchImpl` so every outgoing request carries the provider's headers
|
||||
* (a user agent, beta flags). Existing headers on a request win, so a caller
|
||||
* can still override one deliberately.
|
||||
*/
|
||||
export function withAuthHeaders(
|
||||
fetchImpl: typeof fetch | undefined,
|
||||
headers: Record<string, string>
|
||||
): typeof fetch | undefined {
|
||||
if (Object.keys(headers).length === 0) return fetchImpl;
|
||||
const underlying: typeof fetch | undefined =
|
||||
fetchImpl ?? (typeof fetch === 'function' ? fetch : undefined);
|
||||
if (!underlying) return fetchImpl;
|
||||
return async (input: any, init?: any) => {
|
||||
const merged = new Headers(init?.headers ?? {});
|
||||
for (const [key, value] of Object.entries(headers)) {
|
||||
if (!merged.has(key)) merged.set(key, value);
|
||||
}
|
||||
return underlying(input, { ...(init ?? {}), headers: merged });
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
/**
|
||||
* Per-call options of `Agent.chat` (Python parity: `agent/chat_mixin.py`
|
||||
* `chat()` / `_chat_impl()`).
|
||||
*
|
||||
* Four behaviours live here, each a direct port of what Python does with the
|
||||
* keyword argument of the same name:
|
||||
*
|
||||
* - `attachments`: images for THIS turn only. Python's
|
||||
* `_build_multimodal_prompt` turns the text prompt plus a list of paths,
|
||||
* URLs or data URIs into an OpenAI multimodal `content` array. The text
|
||||
* still goes into history on its own; the attachment never does.
|
||||
* - `reasoningSteps`: Python forces a single NON-streaming completion for the
|
||||
* turn (`llm/llm.py`: "If reasoning_steps is True, do a single
|
||||
* non-streaming call"), because a reasoning response is only readable whole.
|
||||
* - `taskName` / `taskDescription` / `taskId`: metadata that Python threads
|
||||
* through the whole turn into `display_interaction` and the interaction
|
||||
* callbacks. Here the equivalent observer channel is the hooks manager, so
|
||||
* the three travel on the `agent_start` / `agent_complete` hook contexts and
|
||||
* are readable afterwards via `Agent.getTaskContext()`.
|
||||
* - `config`: Python only forwards it, unchanged, to a managed backend's
|
||||
* `execute()` (`execution_mixin.py::_delegate_to_backend`). So does this.
|
||||
*/
|
||||
|
||||
/** A text part of a multimodal prompt (OpenAI `content` array). */
|
||||
export interface PromptTextPart {
|
||||
type: 'text';
|
||||
text: string;
|
||||
}
|
||||
|
||||
/** An image part of a multimodal prompt (OpenAI `content` array). */
|
||||
export interface PromptImagePart {
|
||||
type: 'image_url';
|
||||
image_url: { url: string };
|
||||
}
|
||||
|
||||
export type PromptPart = PromptTextPart | PromptImagePart;
|
||||
|
||||
/**
|
||||
* Task metadata for one turn (Python's `task_name` / `task_description` /
|
||||
* `task_id` triple). `undefined` when the caller named none of the three.
|
||||
*/
|
||||
export interface TaskContext {
|
||||
name?: string;
|
||||
description?: string;
|
||||
id?: string;
|
||||
}
|
||||
|
||||
/** Python's `_build_multimodal_prompt` extension → media-type table. */
|
||||
const IMAGE_MEDIA_TYPES: Readonly<Record<string, string>> = Object.freeze({
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.png': 'image/png',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
});
|
||||
|
||||
/** Build the {@link TaskContext} for a turn, or `undefined` when unused. */
|
||||
export function resolveTaskContext(opts: {
|
||||
taskName?: string;
|
||||
taskDescription?: string;
|
||||
taskId?: string;
|
||||
}): TaskContext | undefined {
|
||||
const { taskName, taskDescription, taskId } = opts;
|
||||
if (taskName === undefined && taskDescription === undefined && taskId === undefined) return undefined;
|
||||
const context: TaskContext = {};
|
||||
if (taskName !== undefined) context.name = taskName;
|
||||
if (taskDescription !== undefined) context.description = taskDescription;
|
||||
if (taskId !== undefined) context.id = taskId;
|
||||
return context;
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge a {@link TaskContext} into a hook/observer payload. Absent fields are
|
||||
* not written, so a hook can tell "no task metadata" from "an empty name".
|
||||
*/
|
||||
export function withTaskContext<T extends Record<string, unknown>>(
|
||||
payload: T,
|
||||
task: TaskContext | undefined
|
||||
): T {
|
||||
if (!task) return payload;
|
||||
const merged = { ...payload } as Record<string, unknown>;
|
||||
if (task.name !== undefined) merged.taskName = task.name;
|
||||
if (task.description !== undefined) merged.taskDescription = task.description;
|
||||
if (task.id !== undefined) merged.taskId = task.id;
|
||||
return merged as T;
|
||||
}
|
||||
|
||||
/**
|
||||
* Python `_build_multimodal_prompt`: a string when there is nothing to
|
||||
* attach, otherwise the OpenAI multimodal `content` array with the text first.
|
||||
*
|
||||
* A local image file is read and inlined as a base64 data URI; an
|
||||
* `http(s)://` or `data:` attachment is passed through as a URL. Anything
|
||||
* else (a missing file, a non-image extension, an unreadable file) is
|
||||
* reported through `onWarning` and skipped -- exactly Python's behaviour,
|
||||
* which logs and continues rather than failing the turn.
|
||||
*
|
||||
* `readFile` is injectable so a test need not touch the filesystem.
|
||||
*/
|
||||
export async function buildMultimodalPrompt(
|
||||
prompt: string,
|
||||
attachments: readonly string[] | undefined,
|
||||
options: {
|
||||
onWarning?: (message: string) => void;
|
||||
readFile?: (file: string) => Buffer | null | Promise<Buffer | null>;
|
||||
} = {}
|
||||
): Promise<string | PromptPart[]> {
|
||||
if (!attachments || attachments.length === 0) return prompt;
|
||||
|
||||
const warn = options.onWarning ?? (() => {});
|
||||
const read = options.readFile ?? defaultReadFile;
|
||||
const content: PromptPart[] = [{ type: 'text', text: prompt }];
|
||||
|
||||
for (const attachment of attachments) {
|
||||
if (typeof attachment !== 'string' || attachment.length === 0) {
|
||||
warn(`Ignoring an attachment that is not a path, URL or data URI: ${String(attachment)}`);
|
||||
continue;
|
||||
}
|
||||
if (/^(https?:\/\/|data:)/.test(attachment)) {
|
||||
content.push({ type: 'image_url', image_url: { url: attachment } });
|
||||
continue;
|
||||
}
|
||||
const ext = extensionOf(attachment);
|
||||
const mediaType = IMAGE_MEDIA_TYPES[ext];
|
||||
if (!mediaType) {
|
||||
warn(`Ignoring attachment "${attachment}": only ${Object.keys(IMAGE_MEDIA_TYPES).join(', ')} images are supported.`);
|
||||
continue;
|
||||
}
|
||||
const bytes = await read(attachment);
|
||||
if (bytes === null) {
|
||||
warn(`Failed to load attachment "${attachment}"; it is not readable.`);
|
||||
continue;
|
||||
}
|
||||
content.push({
|
||||
type: 'image_url',
|
||||
image_url: { url: `data:${mediaType};base64,${bytes.toString('base64')}` },
|
||||
});
|
||||
}
|
||||
|
||||
return content;
|
||||
}
|
||||
|
||||
function extensionOf(file: string): string {
|
||||
const dot = file.lastIndexOf('.');
|
||||
const slash = Math.max(file.lastIndexOf('/'), file.lastIndexOf('\\'));
|
||||
return dot > slash ? file.slice(dot).toLowerCase() : '';
|
||||
}
|
||||
|
||||
async function defaultReadFile(file: string): Promise<Buffer | null> {
|
||||
try {
|
||||
// A computed specifier, not require(): the esm-shim gives any file using
|
||||
// require() a createRequire banner, which is a top-level await, and esbuild
|
||||
// refuses that at the chrome58 floor the mobile app ships against. This
|
||||
// keeps `fs` off the graph AND the file bannerless.
|
||||
const fs: typeof import('fs') = await import(/* @vite-ignore */ ['n', 'ode:fs'].join(''));
|
||||
if (!fs.existsSync(file) || !fs.statSync(file).isFile()) return null;
|
||||
return fs.readFileSync(file);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
/**
|
||||
* Real-time message steering (Python parity: `Agent(message_steering=...)`,
|
||||
* `agent/message_steering.py` `MessageSteering` / `SteeringMixin`,
|
||||
* `agent/protocols.py` `SteeringPriority` / `MessageSteeringProtocol`).
|
||||
*
|
||||
* Someone watching an agent work can send it guidance without cancelling the
|
||||
* run: `agent.steer("check the staging config first")`. The note is queued by
|
||||
* priority and folded into the next turn's prompt, with wording that tells the
|
||||
* model how urgently to act on it.
|
||||
*
|
||||
* Priority matters and is preserved end to end. An INTERRUPT that arrives as
|
||||
* ordinary "consider this feedback" guidance is worse than no steering at
|
||||
* all: the operator believes they stopped the agent, and it carried on.
|
||||
*/
|
||||
|
||||
/** Python `SteeringPriority`. */
|
||||
export enum SteeringPriority {
|
||||
LOW = 1,
|
||||
NORMAL = 5,
|
||||
HIGH = 10,
|
||||
URGENT = 20,
|
||||
INTERRUPT = 30,
|
||||
}
|
||||
|
||||
/** Python `SteeringMessage`. */
|
||||
export interface SteeringMessage {
|
||||
content: string;
|
||||
priority: SteeringPriority;
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** The interface an Agent needs from a steering implementation. */
|
||||
export interface MessageSteeringProtocol {
|
||||
readonly enabled: boolean;
|
||||
queueMessage(message: string, priority?: number): string;
|
||||
/** Every pending message, highest priority first, removing them from the queue. */
|
||||
drain(): SteeringMessage[];
|
||||
hasPendingMessages(): boolean;
|
||||
clearMessages(): number;
|
||||
enable(): void;
|
||||
disable(): void;
|
||||
}
|
||||
|
||||
/** Python's priority-aware wording, shared by every injection path. */
|
||||
export function formatSteeringNote(content: string, priority: SteeringPriority): string {
|
||||
if (priority >= SteeringPriority.INTERRUPT) {
|
||||
return `[INTERRUPT USER GUIDANCE]: ${content}\nPlease stop current work and follow this guidance immediately.`;
|
||||
}
|
||||
if (priority >= SteeringPriority.HIGH) {
|
||||
return `[URGENT USER GUIDANCE]: ${content}\nPlease acknowledge and adjust your approach accordingly.`;
|
||||
}
|
||||
return `[USER GUIDANCE]: ${content}\nPlease consider this feedback as you continue.`;
|
||||
}
|
||||
|
||||
function toPriority(value: number): SteeringPriority {
|
||||
const known = [
|
||||
SteeringPriority.LOW, SteeringPriority.NORMAL, SteeringPriority.HIGH,
|
||||
SteeringPriority.URGENT, SteeringPriority.INTERRUPT,
|
||||
];
|
||||
return known.includes(value as SteeringPriority) ? (value as SteeringPriority) : SteeringPriority.NORMAL;
|
||||
}
|
||||
|
||||
/**
|
||||
* The built-in implementation: a bounded priority queue (Python
|
||||
* `AgentMessageQueue`, default 50 messages).
|
||||
*/
|
||||
export class MessageSteering implements MessageSteeringProtocol {
|
||||
private queue: SteeringMessage[] = [];
|
||||
private _enabled = true;
|
||||
private counter = 0;
|
||||
|
||||
constructor(readonly maxMessages: number = 50) {}
|
||||
|
||||
get enabled(): boolean {
|
||||
return this._enabled;
|
||||
}
|
||||
|
||||
queueMessage(message: string, priority: number = SteeringPriority.NORMAL): string {
|
||||
if (!this._enabled) return '';
|
||||
if (this.queue.length >= this.maxMessages) return '';
|
||||
this.queue.push({
|
||||
content: message,
|
||||
priority: toPriority(priority),
|
||||
metadata: { timestamp: Date.now() },
|
||||
});
|
||||
this.counter += 1;
|
||||
return `steer_${Date.now()}_${this.counter}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Take every pending message, highest priority first. Python drains one per
|
||||
* check pre-turn and all of them mid-run; taking them all at the one
|
||||
* injection point this SDK has means a queued note is never left behind
|
||||
* while the agent answers without it.
|
||||
*/
|
||||
drain(): SteeringMessage[] {
|
||||
if (!this._enabled || this.queue.length === 0) return [];
|
||||
// Stable within a priority so two notes of equal urgency stay in the
|
||||
// order the operator wrote them.
|
||||
const ordered = this.queue
|
||||
.map((message, index) => ({ message, index }))
|
||||
.sort((a, b) => b.message.priority - a.message.priority || a.index - b.index)
|
||||
.map((entry) => entry.message);
|
||||
this.queue = [];
|
||||
return ordered;
|
||||
}
|
||||
|
||||
hasPendingMessages(): boolean {
|
||||
return this.queue.length > 0;
|
||||
}
|
||||
|
||||
clearMessages(): number {
|
||||
const count = this.queue.length;
|
||||
this.queue = [];
|
||||
return count;
|
||||
}
|
||||
|
||||
enable(): void {
|
||||
this._enabled = true;
|
||||
}
|
||||
|
||||
disable(): void {
|
||||
this._enabled = false;
|
||||
}
|
||||
}
|
||||
|
||||
function isSteeringProtocol(value: unknown): value is MessageSteeringProtocol {
|
||||
if (value === null || typeof value !== 'object') return false;
|
||||
const v = value as Record<string, unknown>;
|
||||
return typeof v.queueMessage === 'function' && typeof v.drain === 'function';
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the constructor option: `true` for the built-in implementation, a
|
||||
* custom object implementing the protocol, or `undefined` when off
|
||||
* (Python `SteeringMixin._init_message_steering`).
|
||||
*/
|
||||
export function resolveMessageSteering(
|
||||
input: boolean | Record<string, unknown> | undefined | null
|
||||
): MessageSteeringProtocol | undefined {
|
||||
if (input === undefined || input === null || input === false) return undefined;
|
||||
if (input === true) return new MessageSteering();
|
||||
if (isSteeringProtocol(input)) return input;
|
||||
throw new Error(
|
||||
'messageSteering must be true, false, or an object with queueMessage() and drain().'
|
||||
);
|
||||
}
|
||||
|
||||
/** The block appended to a prompt for the drained messages, or '' when there are none. */
|
||||
export function steeringNotesFor(messages: readonly SteeringMessage[]): string {
|
||||
if (messages.length === 0) return '';
|
||||
return messages.map((m) => formatSteeringNote(m.content, m.priority)).join('\n\n');
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
/**
|
||||
* "Where does this run?" — `backend=`, `runOn=` and `toolsRunOn=`
|
||||
* (Python parity: `agent/placement.py`, `agent/tools_placement.py`,
|
||||
* `agent/execution_mixin.py::_delegate_to_backend`).
|
||||
*
|
||||
* Two different questions, deliberately kept apart:
|
||||
*
|
||||
* - `runOn` / `backend` place the **whole agent** — model calls, loop and
|
||||
* tools — on a managed runtime. `runOn: "anthropic"` is shorthand for
|
||||
* `backend: <hosted agent for anthropic>`.
|
||||
* - `toolsRunOn` places **only the tools**. Thinking stays here; the shell,
|
||||
* the file writes and the code execution move.
|
||||
*
|
||||
* Because the two answer sets are disjoint, a wrong pairing is a typo worth
|
||||
* catching rather than a preference worth honouring: `runOn: "docker"` cannot
|
||||
* mean anything, and gets an error naming the parameter the caller wanted.
|
||||
* That validation is ported verbatim from `resolve_placement`.
|
||||
*
|
||||
* Neither a hosted runtime nor a remote sandbox ships in this package (they
|
||||
* are provider integrations), so both are resolved through registries a host
|
||||
* fills in — {@link registerManagedRuntime} and {@link registerToolPlace}.
|
||||
* An unregistered name fails loudly at construction with the same
|
||||
* "install the integration" shape Python uses, instead of running the agent
|
||||
* on the wrong machine.
|
||||
*/
|
||||
|
||||
/** A managed runtime that can host an entire agent loop (Python `ManagedBackendProtocol`). */
|
||||
export interface ManagedBackendLike {
|
||||
/** Run one prompt on managed infrastructure and return the whole response. */
|
||||
execute(prompt: unknown, options?: Record<string, unknown>): Promise<unknown> | unknown;
|
||||
/** Optional streaming form, used when the turn is streaming. */
|
||||
stream?(prompt: unknown, options?: Record<string, unknown>): AsyncIterable<string>;
|
||||
/** Optional id of the managed session, linked into the agent's session store. */
|
||||
managedSessionId?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* A place that can run a tool call (Python's `ToolPlace` / `SharedCompute`
|
||||
* adapters). `runTool` receives the tool name, its parsed arguments and the
|
||||
* local implementation, and returns the result — a local place just calls it.
|
||||
*/
|
||||
export interface ToolPlaceLike {
|
||||
readonly placeName: string;
|
||||
runTool(
|
||||
toolName: string,
|
||||
args: Record<string, unknown>,
|
||||
localImplementation: () => Promise<unknown>
|
||||
): Promise<unknown>;
|
||||
}
|
||||
|
||||
/** The `local` place: the tool runs in this process, unchanged (Python's default). */
|
||||
export const LOCAL_TOOL_PLACE: ToolPlaceLike = Object.freeze({
|
||||
placeName: 'local',
|
||||
async runTool(
|
||||
_toolName: string,
|
||||
_args: Record<string, unknown>,
|
||||
localImplementation: () => Promise<unknown>
|
||||
): Promise<unknown> {
|
||||
return localImplementation();
|
||||
},
|
||||
});
|
||||
|
||||
const managedRuntimes = new Map<string, () => ManagedBackendLike>();
|
||||
const toolPlaces = new Map<string, () => ToolPlaceLike>([['local', () => LOCAL_TOOL_PLACE]]);
|
||||
|
||||
/**
|
||||
* Register a runtime that `runOn: "<name>"` may name (Python's backend
|
||||
* entry-point registry, which the wrapper package fills in).
|
||||
*/
|
||||
export function registerManagedRuntime(name: string, factory: () => ManagedBackendLike): void {
|
||||
managedRuntimes.set(name.toLowerCase(), factory);
|
||||
}
|
||||
|
||||
/** Forget a managed runtime registration. Returns whether one was removed. */
|
||||
export function unregisterManagedRuntime(name: string): boolean {
|
||||
return managedRuntimes.delete(name.toLowerCase());
|
||||
}
|
||||
|
||||
/** Every runtime name `runOn=` accepts right now. */
|
||||
export function managedRuntimeNames(): string[] {
|
||||
return [...managedRuntimes.keys()].sort();
|
||||
}
|
||||
|
||||
/** Register a place that `toolsRunOn: "<name>"` may name (Python's compute bridge). */
|
||||
export function registerToolPlace(name: string, factory: () => ToolPlaceLike): void {
|
||||
toolPlaces.set(name.toLowerCase(), factory);
|
||||
}
|
||||
|
||||
/** Forget a tool place registration. `local` cannot be removed. */
|
||||
export function unregisterToolPlace(name: string): boolean {
|
||||
if (name.toLowerCase() === 'local') return false;
|
||||
return toolPlaces.delete(name.toLowerCase());
|
||||
}
|
||||
|
||||
/** Every place name `toolsRunOn=` accepts right now. */
|
||||
export function toolPlaceNames(): string[] {
|
||||
return [...toolPlaces.keys()].sort();
|
||||
}
|
||||
|
||||
/** Anything with an `execute()` is a backend (Python: `hasattr(backend, 'execute')`). */
|
||||
export function isManagedBackendLike(value: unknown): value is ManagedBackendLike {
|
||||
if (value === null || (typeof value !== 'object' && typeof value !== 'function')) return false;
|
||||
return typeof (value as Record<string, unknown>).execute === 'function';
|
||||
}
|
||||
|
||||
function isToolPlaceLike(value: unknown): value is ToolPlaceLike {
|
||||
if (value === null || typeof value !== 'object') return false;
|
||||
const v = value as Record<string, unknown>;
|
||||
return typeof v.runTool === 'function';
|
||||
}
|
||||
|
||||
/** Python `_name_of`: only bare strings are routed by name; objects carry their own config. */
|
||||
function nameOf(value: unknown): string | null {
|
||||
return typeof value === 'string' ? value.toLowerCase() : null;
|
||||
}
|
||||
|
||||
function quoteList(names: readonly string[]): string {
|
||||
return names.length === 0 ? '(none registered)' : names.map((n) => `'${n}'`).join(', ');
|
||||
}
|
||||
|
||||
/** The resolved answer to "where does this run?" (Python `Placement`). */
|
||||
export interface Placement {
|
||||
/** The managed backend running the whole agent, or `undefined`. */
|
||||
backend?: ManagedBackendLike;
|
||||
/** The place this agent's tools run, or `undefined` for in-process. */
|
||||
toolPlace?: ToolPlaceLike;
|
||||
}
|
||||
|
||||
export interface ResolvePlacementInput {
|
||||
/** Class name, used only so errors read naturally (Python's `owner`). */
|
||||
owner?: string;
|
||||
runOn?: unknown;
|
||||
toolsRunOn?: unknown;
|
||||
backend?: unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the three placement options against each other and resolve them
|
||||
* (Python `resolve_placement` plus `Agent._hosted_backend_for`).
|
||||
*
|
||||
* Throws when two options name two different places for the same thing, or
|
||||
* when a place is asked to do a job it cannot do.
|
||||
*/
|
||||
export function resolvePlacement(input: ResolvePlacementInput): Placement {
|
||||
const owner = input.owner ?? 'Agent';
|
||||
const { runOn, toolsRunOn, backend } = input;
|
||||
const managed = managedRuntimeNames();
|
||||
const places = toolPlaceNames();
|
||||
|
||||
// ── a place that cannot do the job asked of it ──────────────────────────
|
||||
const runOnName = nameOf(runOn);
|
||||
if (runOn !== undefined && runOn !== null && runOnName !== null && !managedRuntimes.has(runOnName)) {
|
||||
if (toolPlaces.has(runOnName)) {
|
||||
throw new TypeError(
|
||||
`${owner}(runOn: '${runOnName}') is not valid: runOn places the whole agent -- model calls, ` +
|
||||
`loop and tools -- on a managed runtime, and '${runOnName}' runs commands but cannot host an agent loop.\n` +
|
||||
` To run only the tools there: ${owner}({ toolsRunOn: '${runOnName}' })\n` +
|
||||
` Runtimes that host a whole agent: ${quoteList(managed)}`
|
||||
);
|
||||
}
|
||||
throw new TypeError(
|
||||
`${owner}(runOn: '${runOnName}') is not a known managed runtime.\n` +
|
||||
` Runtimes that host a whole agent: ${quoteList(managed)}\n` +
|
||||
` Places that can run tools: ${quoteList(places)} (pass those as toolsRunOn)\n` +
|
||||
` A managed runtime is a provider integration: register one with registerManagedRuntime('${runOnName}', ...).`
|
||||
);
|
||||
}
|
||||
|
||||
const toolsName = nameOf(toolsRunOn);
|
||||
if (toolsName !== null && managedRuntimes.has(toolsName) && !toolPlaces.has(toolsName)) {
|
||||
throw new TypeError(
|
||||
`${owner}(toolsRunOn: '${toolsName}') is not valid: '${toolsName}' hosts an entire agent, not individual tools.\n` +
|
||||
` Did you mean: ${owner}({ runOn: '${toolsName}' })`
|
||||
);
|
||||
}
|
||||
|
||||
// A typo here used to survive construction and only surface on the first
|
||||
// model turn -- after the prompt was built and, for a real run, after the
|
||||
// API spend. Validate at the call site instead.
|
||||
if (toolsName !== null && !toolPlaces.has(toolsName)) {
|
||||
throw new TypeError(
|
||||
`${owner}(toolsRunOn: '${toolsName}') is not a known place.\n` +
|
||||
` Places that can run tools: ${quoteList(places)}\n` +
|
||||
` Runtimes that host a whole agent: ${quoteList(managed)} (pass those as runOn)\n` +
|
||||
` A remote place is a provider integration: register one with registerToolPlace('${toolsName}', ...).`
|
||||
);
|
||||
}
|
||||
|
||||
// ── two names for the same thing ────────────────────────────────────────
|
||||
if (runOn !== undefined && runOn !== null && backend !== undefined && backend !== null) {
|
||||
throw new TypeError(
|
||||
`${owner}(runOn, backend) sets the agent's runtime twice. runOn is the short form of a hosted ` +
|
||||
`backend for that provider; pass one or the other.\n` +
|
||||
` Use backend only when you need to configure the runtime (model, system prompt, tools).`
|
||||
);
|
||||
}
|
||||
|
||||
if (runOn !== undefined && runOn !== null && toolsRunOn !== undefined && toolsRunOn !== null) {
|
||||
throw new TypeError(
|
||||
`${owner}(runOn, toolsRunOn) points the tools at two machines. runOn already runs everything ` +
|
||||
`-- including tools -- on the managed runtime.\n` +
|
||||
` Whole agent remote: ${owner}({ runOn })\n` +
|
||||
` Only tools remote: ${owner}({ toolsRunOn })`
|
||||
);
|
||||
}
|
||||
|
||||
if (backend !== undefined && backend !== null && toolsRunOn !== undefined && toolsRunOn !== null) {
|
||||
throw new TypeError(
|
||||
`${owner}(backend, toolsRunOn) points the tools at two machines: backend runs the whole agent ` +
|
||||
`(tools included) on its managed runtime, so the tools cannot also run elsewhere.\n` +
|
||||
` Drop toolsRunOn to keep the hosted runtime, or drop backend to keep local thinking with remote tools.`
|
||||
);
|
||||
}
|
||||
|
||||
// ── resolve ─────────────────────────────────────────────────────────────
|
||||
const placement: Placement = {};
|
||||
|
||||
if (backend !== undefined && backend !== null) {
|
||||
if (!isManagedBackendLike(backend)) {
|
||||
throw new TypeError(
|
||||
`${owner}(backend) does not support execute(): a managed backend must expose ` +
|
||||
`execute(prompt, options) returning the response.`
|
||||
);
|
||||
}
|
||||
placement.backend = backend;
|
||||
} else if (runOn !== undefined && runOn !== null) {
|
||||
if (isManagedBackendLike(runOn)) {
|
||||
placement.backend = runOn;
|
||||
} else {
|
||||
placement.backend = managedRuntimes.get(runOnName as string)!();
|
||||
}
|
||||
}
|
||||
|
||||
if (toolsRunOn !== undefined && toolsRunOn !== null) {
|
||||
if (isToolPlaceLike(toolsRunOn)) {
|
||||
placement.toolPlace = toolsRunOn;
|
||||
} else if (toolsName !== null) {
|
||||
placement.toolPlace = toolPlaces.get(toolsName)!();
|
||||
} else {
|
||||
throw new TypeError(
|
||||
`${owner}(toolsRunOn) must be a place name or an object with runTool(name, args, run).`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return placement;
|
||||
}
|
||||
|
||||
/** Everything Python forwards to a managed backend's `execute()`. */
|
||||
export interface BackendDelegationOptions {
|
||||
temperature?: number;
|
||||
tools?: unknown;
|
||||
outputJson?: unknown;
|
||||
outputPydantic?: unknown;
|
||||
reasoningSteps?: boolean;
|
||||
stream?: boolean;
|
||||
taskName?: string;
|
||||
taskDescription?: string;
|
||||
taskId?: string;
|
||||
/** Python's `config` keyword: forwarded verbatim, never interpreted here. */
|
||||
config?: Record<string, unknown>;
|
||||
forceRetrieval?: boolean;
|
||||
skipRetrieval?: boolean;
|
||||
attachments?: readonly string[];
|
||||
toolChoice?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Hand the whole turn to the managed backend (Python
|
||||
* `_delegate_to_backend`). Streams through `backend.stream()` when the turn is
|
||||
* streaming and the backend offers one, so tokens still reach `onToken`.
|
||||
*/
|
||||
export async function delegateToBackend(
|
||||
backend: ManagedBackendLike,
|
||||
prompt: string,
|
||||
options: BackendDelegationOptions,
|
||||
onToken?: (token: string) => void
|
||||
): Promise<string> {
|
||||
if (options.stream && typeof backend.stream === 'function') {
|
||||
let accumulated = '';
|
||||
for await (const chunk of backend.stream(prompt, { ...options })) {
|
||||
const text = typeof chunk === 'string' ? chunk : String(chunk);
|
||||
if (text.length === 0) continue;
|
||||
accumulated += text;
|
||||
onToken?.(text);
|
||||
}
|
||||
return accumulated;
|
||||
}
|
||||
const result = await backend.execute(prompt, { ...options });
|
||||
return result === null || result === undefined ? '' : String(result);
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
/**
|
||||
* Model-scoped runtime selection (Python parity: `Agent(runtime=...)`,
|
||||
* `runtime/config.py` `AgentRuntimeConfig`, `agent/agent.py`
|
||||
* `_resolve_runtime_config` / `_validate_runtime_capabilities`, and the
|
||||
* delegation in `agent/unified_execution_mixin.py`).
|
||||
*
|
||||
* `runtime` hands the whole turn to a registered {@link AgentRuntimeProtocol}
|
||||
* rather than running the local chat loop — the successor to the deprecated
|
||||
* `cli_backend`. Two things happen, both at construction:
|
||||
*
|
||||
* - The runtime id is resolved through the registry, so a typo fails at the
|
||||
* call site instead of on the first model turn.
|
||||
* - `requiredCapabilities` are checked against what the runtime reports, so a
|
||||
* run that needs streaming does not start on a runtime that cannot stream
|
||||
* and then discover it halfway through.
|
||||
*
|
||||
* `runtime` and `backend`/`runOn` answer the same question — where the loop
|
||||
* runs — so naming both is a contradiction, and is rejected as one.
|
||||
*/
|
||||
|
||||
import { isAgentRuntime, resolveRuntime, type AgentRuntimeProtocol } from '../../runtime';
|
||||
|
||||
/** Python `AgentRuntimeConfig`, plus the capability fields `RuntimeConfig` carries. */
|
||||
export interface AgentRuntimeConfig {
|
||||
/** Runtime identifier, e.g. `"praisonai"` (Python `runtime` / `preferred_runtime`). */
|
||||
runtime: string;
|
||||
/** Capabilities the runtime must report before the agent is built. */
|
||||
requiredCapabilities: string[];
|
||||
/** Runtime-specific options, forwarded on every turn (Python `config_overrides`). */
|
||||
configOverrides: Record<string, unknown>;
|
||||
metadata: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** The default runtime when `runtime: true` names none (Python's builtin). */
|
||||
export const DEFAULT_RUNTIME_ID = 'praisonai';
|
||||
|
||||
/** A resolved runtime plus the config that selected it. */
|
||||
export interface ResolvedAgentRuntime {
|
||||
runtime: AgentRuntimeProtocol;
|
||||
config: AgentRuntimeConfig;
|
||||
}
|
||||
|
||||
function stringList(value: unknown, field: string): string[] {
|
||||
if (value === undefined || value === null) return [];
|
||||
if (!Array.isArray(value) || value.some((v) => typeof v !== 'string')) {
|
||||
throw new TypeError(`runtime.${field} must be an array of capability names`);
|
||||
}
|
||||
return value as string[];
|
||||
}
|
||||
|
||||
function record(value: unknown, field: string): Record<string, unknown> {
|
||||
if (value === undefined || value === null) return {};
|
||||
if (typeof value !== 'object' || Array.isArray(value)) {
|
||||
throw new TypeError(`runtime.${field} must be an object`);
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check the runtime reports every required capability
|
||||
* (Python `_validate_runtime_capabilities`). A capability counts as present
|
||||
* when the matrix carries it with a truthy value.
|
||||
*/
|
||||
export function validateRuntimeCapabilities(
|
||||
runtime: AgentRuntimeProtocol,
|
||||
required: readonly string[]
|
||||
): void {
|
||||
if (required.length === 0) return;
|
||||
const matrix = runtime.capabilities() ?? {};
|
||||
const missing = required.filter((capability) => !matrix[capability]);
|
||||
if (missing.length === 0) return;
|
||||
const reported = Object.keys(matrix).filter((k) => matrix[k]);
|
||||
throw new Error(
|
||||
`Runtime '${runtime.runtimeName}' does not provide the required capabilit${missing.length === 1 ? 'y' : 'ies'}: ` +
|
||||
`${missing.join(', ')}.\n` +
|
||||
` It reports: ${reported.length > 0 ? reported.join(', ') : '(none)'}.`
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the constructor option. `true` selects the builtin runtime, a
|
||||
* string names one, an object supplies `runtime`/`requiredCapabilities`/
|
||||
* `configOverrides` (snake_case accepted), and an object that already
|
||||
* satisfies {@link AgentRuntimeProtocol} is used as-is.
|
||||
*/
|
||||
export function resolveAgentRuntime(
|
||||
input: boolean | string | Record<string, unknown> | undefined | null
|
||||
): ResolvedAgentRuntime | undefined {
|
||||
if (input === undefined || input === null || input === false) return undefined;
|
||||
|
||||
let config: AgentRuntimeConfig;
|
||||
let instance: AgentRuntimeProtocol | undefined;
|
||||
|
||||
if (input === true) {
|
||||
config = { runtime: DEFAULT_RUNTIME_ID, requiredCapabilities: [], configOverrides: {}, metadata: {} };
|
||||
} else if (typeof input === 'string') {
|
||||
config = { runtime: input, requiredCapabilities: [], configOverrides: {}, metadata: {} };
|
||||
} else if (isAgentRuntime(input)) {
|
||||
instance = input;
|
||||
config = { runtime: input.runtimeName, requiredCapabilities: [], configOverrides: {}, metadata: {} };
|
||||
} else if (typeof input === 'object') {
|
||||
const id = input.runtime ?? input.preferredRuntime ?? input.preferred_runtime;
|
||||
if (id !== undefined && typeof id !== 'string') {
|
||||
throw new TypeError('runtime.runtime must be a runtime id string');
|
||||
}
|
||||
config = {
|
||||
runtime: (id as string) ?? DEFAULT_RUNTIME_ID,
|
||||
requiredCapabilities: stringList(
|
||||
input.requiredCapabilities ?? input.required_capabilities,
|
||||
'requiredCapabilities'
|
||||
),
|
||||
configOverrides: record(input.configOverrides ?? input.config_overrides, 'configOverrides'),
|
||||
metadata: record(input.metadata, 'metadata'),
|
||||
};
|
||||
} else {
|
||||
throw new TypeError(
|
||||
`Invalid runtime type: ${typeof input}. Expected boolean, string, object, or an AgentRuntimeProtocol instance.`
|
||||
);
|
||||
}
|
||||
|
||||
const runtime = instance ?? resolveRuntime(config.runtime);
|
||||
validateRuntimeCapabilities(runtime, config.requiredCapabilities);
|
||||
return { runtime, config };
|
||||
}
|
||||
|
||||
/** Options a delegated turn carries to the runtime (Python's `**kwargs`). */
|
||||
export interface RuntimeTurnOptions {
|
||||
systemPrompt?: string | null;
|
||||
modelRef?: string | null;
|
||||
tools?: unknown;
|
||||
temperature?: number;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one turn on the resolved runtime (Python `_chat_via_runtime`). A
|
||||
* runtime reports failure in the result rather than by throwing, so the
|
||||
* error is re-raised here — silently returning its empty `content` would
|
||||
* read as "the model said nothing".
|
||||
*/
|
||||
export async function runTurnOnRuntime(
|
||||
resolved: ResolvedAgentRuntime,
|
||||
prompt: string,
|
||||
options: RuntimeTurnOptions
|
||||
): Promise<string> {
|
||||
const result = await resolved.runtime.runTurn(prompt, {
|
||||
...resolved.config.configOverrides,
|
||||
...options,
|
||||
});
|
||||
if (result.error) {
|
||||
throw new Error(`Runtime '${resolved.runtime.runtimeName}' failed: ${result.error}`);
|
||||
}
|
||||
return result.content ?? '';
|
||||
}
|
||||
@@ -0,0 +1,434 @@
|
||||
/**
|
||||
* Sandboxed code execution (Python parity: `Agent(sandbox=...)`,
|
||||
* `agent/sandbox_mixin.py` `SandboxMixin`, `sandbox/config.py`,
|
||||
* `sandbox/security.py`).
|
||||
*
|
||||
* `sandbox: true` gives the agent a subprocess sandbox; a config object names
|
||||
* a different one. Two things then become true of `agent.executeCode()`:
|
||||
*
|
||||
* 1. The code is analysed BEFORE it runs, and the warnings travel with the
|
||||
* result. Python's note applies here too -- the analysis is best-effort
|
||||
* static pattern matching for the operator's awareness; the sandbox is
|
||||
* what actually isolates.
|
||||
* 2. Without a sandbox configured (and no per-call `runIn`), the call fails
|
||||
* with an error that names both ways to fix it, rather than quietly
|
||||
* running the code in this process.
|
||||
*
|
||||
* The runner is an injectable interface. Only `subprocess` ships here;
|
||||
* `docker`, `e2b` and friends are provider integrations a host registers with
|
||||
* {@link registerSandboxRunner}, and naming one that is not registered fails
|
||||
* loudly instead of silently downgrading to local execution.
|
||||
*/
|
||||
|
||||
/** Python `SandboxConfig` (the subset the Agent needs). */
|
||||
export interface SandboxConfig {
|
||||
sandboxType: string;
|
||||
image: string;
|
||||
workingDir: string;
|
||||
env: Record<string, string>;
|
||||
autoCleanup: boolean;
|
||||
persistFiles: boolean;
|
||||
/** Seconds a single execution may take. */
|
||||
timeout: number;
|
||||
metadata: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** Python's `SandboxConfig` defaults (`sandbox/config.py`). */
|
||||
export function defaultSandboxConfig(sandboxType: string = 'subprocess'): SandboxConfig {
|
||||
return {
|
||||
sandboxType,
|
||||
image: 'python:3.12-slim',
|
||||
workingDir: '/workspace',
|
||||
env: {},
|
||||
autoCleanup: true,
|
||||
persistFiles: false,
|
||||
timeout: 30,
|
||||
metadata: {},
|
||||
};
|
||||
}
|
||||
|
||||
/** Python `SecurityWarning`. */
|
||||
export interface SecurityWarning {
|
||||
pattern: string;
|
||||
message: string;
|
||||
severity: 'low' | 'medium' | 'high' | 'critical';
|
||||
lineNumber?: number;
|
||||
context?: string;
|
||||
}
|
||||
|
||||
/** Python `SandboxResult` (the fields the Agent surfaces). */
|
||||
export interface SandboxResult {
|
||||
success: boolean;
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
exitCode: number;
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** Anything that can run code somewhere isolated. */
|
||||
export interface SandboxRunner {
|
||||
readonly sandboxType: string;
|
||||
execute(code: string, options: { language: string; config: SandboxConfig }): Promise<SandboxResult>;
|
||||
}
|
||||
|
||||
type PatternRule = [RegExp, string, SecurityWarning['severity']];
|
||||
|
||||
/** Python `DANGEROUS_PATTERNS`. */
|
||||
const PYTHON_PATTERNS: PatternRule[] = [
|
||||
[/import\s+os\s*;?\s*os\.system/, 'Direct system command execution via os.system', 'high'],
|
||||
[/subprocess\.call.*shell\s*=\s*True/, 'Shell injection risk with subprocess', 'high'],
|
||||
[/subprocess\.run.*shell\s*=\s*True/, 'Shell injection risk with subprocess', 'high'],
|
||||
[/subprocess\.Popen.*shell\s*=\s*True/, 'Shell injection risk with subprocess', 'high'],
|
||||
[/__import__\(['"]os['"]\)/, 'Dynamic import of os module', 'medium'],
|
||||
[/eval\s*\(/, 'Dynamic code execution with eval()', 'critical'],
|
||||
[/exec\s*\(/, 'Dynamic code execution with exec()', 'critical'],
|
||||
[/compile\s*\(.*exec/, 'Code compilation for execution', 'high'],
|
||||
[/open\s*\(.*['"]\/etc/, 'Access to system configuration files', 'medium'],
|
||||
[/open\s*\(.*['"]\/root/, 'Access to root directory', 'high'],
|
||||
[/open\s*\(.*['"]\/home/, 'Access to user directories', 'medium'],
|
||||
[/shutil\.rmtree/, 'Recursive directory deletion', 'high'],
|
||||
[/os\.remove\s*\(/, 'File deletion', 'medium'],
|
||||
[/os\.unlink\s*\(/, 'File deletion', 'medium'],
|
||||
[/socket\.socket/, 'Direct socket creation', 'medium'],
|
||||
[/urllib\.request/, 'HTTP requests via urllib', 'low'],
|
||||
[/requests\./, 'HTTP requests via requests library', 'low'],
|
||||
[/os\.fork\s*\(/, 'Process forking', 'high'],
|
||||
[/multiprocessing\./, 'Multiprocessing usage', 'medium'],
|
||||
[/sys\.modules/, 'Module manipulation', 'high'],
|
||||
[/__builtins__/, 'Access to builtin functions', 'high'],
|
||||
[/while\s+True:/, 'Infinite loop (potential DoS)', 'medium'],
|
||||
];
|
||||
|
||||
/** Python `_check_bash_patterns`. */
|
||||
const BASH_PATTERNS: PatternRule[] = [
|
||||
[/rm\s+-rf\s+\//i, 'Recursive deletion of root directory', 'critical'],
|
||||
[/rm\s+-rf\s+\*/i, 'Recursive deletion with wildcard', 'high'],
|
||||
[/>\s*\/dev\/sd[a-z]/i, 'Direct disk device access', 'critical'],
|
||||
[/dd\s+if=.*of=/i, 'Low-level disk operations', 'high'],
|
||||
[/mkfs\./i, 'Filesystem creation', 'critical'],
|
||||
[/fdisk/i, 'Disk partitioning', 'critical'],
|
||||
[/shutdown/i, 'System shutdown', 'high'],
|
||||
[/reboot/i, 'System reboot', 'high'],
|
||||
[/chmod\s+777/i, 'Overly permissive file permissions', 'medium'],
|
||||
[/curl.*\|\s*bash/i, 'Piped execution from network', 'high'],
|
||||
[/wget.*\|\s*bash/i, 'Piped execution from network', 'high'],
|
||||
];
|
||||
|
||||
/** Python `_check_generic_patterns`. */
|
||||
const GENERIC_PATTERNS: PatternRule[] = [
|
||||
[/eval\s*\(/i, 'Dynamic code execution', 'critical'],
|
||||
[/exec\s*\(/i, 'Dynamic code execution', 'high'],
|
||||
[/system\s*\(/i, 'System command execution', 'high'],
|
||||
[/shell\s*\(/i, 'Shell command execution', 'high'],
|
||||
];
|
||||
|
||||
function scan(code: string, rules: readonly PatternRule[]): SecurityWarning[] {
|
||||
const warnings: SecurityWarning[] = [];
|
||||
const lines = code.split('\n');
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
for (const [pattern, message, severity] of rules) {
|
||||
if (pattern.test(lines[i])) {
|
||||
warnings.push({
|
||||
pattern: pattern.source,
|
||||
message,
|
||||
severity,
|
||||
lineNumber: i + 1,
|
||||
context: lines[i].trim(),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return warnings;
|
||||
}
|
||||
|
||||
/**
|
||||
* Static analysis of code about to be sandboxed (Python `check_code_safety`).
|
||||
* Warnings only: the sandbox provides the real isolation.
|
||||
*/
|
||||
export function checkCodeSafety(code: string, language: string = 'python'): SecurityWarning[] {
|
||||
if (language === 'python') return scan(code, PYTHON_PATTERNS);
|
||||
if (language === 'bash' || language === 'sh' || language === 'shell') return scan(code, BASH_PATTERNS);
|
||||
return scan(code, GENERIC_PATTERNS);
|
||||
}
|
||||
|
||||
/** Python `format_warnings`. */
|
||||
export function formatWarnings(warnings: readonly SecurityWarning[]): string {
|
||||
if (warnings.length === 0) return 'No security issues detected.';
|
||||
const bySeverity: Record<string, SecurityWarning[]> = { critical: [], high: [], medium: [], low: [] };
|
||||
for (const warning of warnings) bySeverity[warning.severity].push(warning);
|
||||
|
||||
const lines: string[] = [`Security analysis found ${warnings.length} potential issue(s):`, ''];
|
||||
for (const severity of ['critical', 'high', 'medium', 'low'] as const) {
|
||||
if (bySeverity[severity].length === 0) continue;
|
||||
lines.push(`${severity.toUpperCase()} RISK:`);
|
||||
for (const warning of bySeverity[severity]) {
|
||||
const where = warning.lineNumber ? ` (line ${warning.lineNumber})` : '';
|
||||
const context = warning.context ? `\n Context: ${warning.context}` : '';
|
||||
lines.push(` - ${warning.message}${where}${context}`);
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
lines.push('Note: These are warnings only. The sandbox provides real isolation.');
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
const KNOWN_KEYS = new Set([
|
||||
'sandboxType', 'sandbox_type', 'image', 'workingDir', 'working_dir', 'env',
|
||||
'autoCleanup', 'auto_cleanup', 'persistFiles', 'persist_files', 'timeout', 'metadata',
|
||||
]);
|
||||
|
||||
/**
|
||||
* Resolve the constructor option. `true` selects the subprocess sandbox
|
||||
* (Python `SandboxConfig.subprocess()`), a string names the sandbox type, an
|
||||
* object supplies fields (snake_case accepted).
|
||||
*/
|
||||
export function resolveSandbox(
|
||||
input: boolean | string | Record<string, unknown> | undefined | null
|
||||
): SandboxConfig | undefined {
|
||||
if (input === undefined || input === null || input === false) return undefined;
|
||||
if (input === true) return defaultSandboxConfig('subprocess');
|
||||
if (typeof input === 'string') return defaultSandboxConfig(input);
|
||||
if (typeof input !== 'object') throw new Error('sandbox must be a boolean, a sandbox type, or a config object');
|
||||
|
||||
const unknown = Object.keys(input).filter((k) => !KNOWN_KEYS.has(k));
|
||||
if (unknown.length > 0) {
|
||||
throw new Error(
|
||||
`Unknown sandbox field(s): ${unknown.join(', ')}. Valid fields: sandboxType, image, workingDir, env, autoCleanup, persistFiles, timeout, metadata`
|
||||
);
|
||||
}
|
||||
const base = defaultSandboxConfig(
|
||||
(input.sandboxType ?? input.sandbox_type ?? 'subprocess') as string
|
||||
);
|
||||
return {
|
||||
...base,
|
||||
image: (input.image as string) ?? base.image,
|
||||
workingDir: (input.workingDir ?? input.working_dir ?? base.workingDir) as string,
|
||||
env: (input.env as Record<string, string>) ?? base.env,
|
||||
autoCleanup: (input.autoCleanup ?? input.auto_cleanup ?? base.autoCleanup) as boolean,
|
||||
persistFiles: (input.persistFiles ?? input.persist_files ?? base.persistFiles) as boolean,
|
||||
timeout: (input.timeout as number) ?? base.timeout,
|
||||
metadata: (input.metadata as Record<string, unknown>) ?? base.metadata,
|
||||
};
|
||||
}
|
||||
|
||||
const runners = new Map<string, (config: SandboxConfig) => SandboxRunner>();
|
||||
|
||||
/**
|
||||
* Register a sandbox implementation for a `sandboxType`. `docker`, `e2b` and
|
||||
* the rest are provider integrations, so a host supplies them; naming an
|
||||
* unregistered type errors rather than downgrading to local execution.
|
||||
*/
|
||||
export function registerSandboxRunner(
|
||||
sandboxType: string,
|
||||
factory: (config: SandboxConfig) => SandboxRunner
|
||||
): void {
|
||||
runners.set(sandboxType.toLowerCase(), factory);
|
||||
}
|
||||
|
||||
/** Forget a sandbox runner registration. `subprocess` cannot be removed. */
|
||||
export function unregisterSandboxRunner(sandboxType: string): boolean {
|
||||
if (sandboxType.toLowerCase() === 'subprocess') return false;
|
||||
return runners.delete(sandboxType.toLowerCase());
|
||||
}
|
||||
|
||||
/** Every registered sandbox type. */
|
||||
export function sandboxRunnerNames(): string[] {
|
||||
return ['subprocess', ...runners.keys()].filter((v, i, a) => a.indexOf(v) === i).sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Interpreter table. `command` is a factory rather than a value so the Node
|
||||
* executable (`process.execPath`) is read only when a call actually spawns a
|
||||
* subprocess -- never at module load. A literal `process.execPath` here would
|
||||
* evaluate `process` while the Agent graph initialises, which throws a
|
||||
* ReferenceError in a browser/mobile WebView (issue #4437). `process` is
|
||||
* reached through globalThis for the same reason.
|
||||
*/
|
||||
const INTERPRETERS: Readonly<Record<string, { command: () => string; args: (file: string) => string[]; extension: string }>> =
|
||||
Object.freeze({
|
||||
python: { command: () => 'python3', args: (f) => [f], extension: '.py' },
|
||||
bash: { command: () => 'bash', args: (f) => [f], extension: '.sh' },
|
||||
sh: { command: () => 'sh', args: (f) => [f], extension: '.sh' },
|
||||
javascript: { command: () => nodeExecPath(), args: (f) => [f], extension: '.js' },
|
||||
node: { command: () => nodeExecPath(), args: (f) => [f], extension: '.js' },
|
||||
});
|
||||
|
||||
/** The Node executable, resolved lazily so it never runs on a browser graph. */
|
||||
function nodeExecPath(): string {
|
||||
const proc = (globalThis as { process?: { execPath?: string } }).process;
|
||||
return proc?.execPath ?? 'node';
|
||||
}
|
||||
|
||||
/**
|
||||
* The built-in `subprocess` runner: the code goes to a temp file and runs in a
|
||||
* child process with the config's cwd, environment and timeout. Isolation is
|
||||
* process-level only -- that is exactly what Python's subprocess sandbox is,
|
||||
* and why the config carries `docker` as an option.
|
||||
*/
|
||||
export function createSubprocessRunner(config: SandboxConfig): SandboxRunner {
|
||||
return {
|
||||
sandboxType: 'subprocess',
|
||||
async execute(code, options): Promise<SandboxResult> {
|
||||
const interpreter = INTERPRETERS[options.language];
|
||||
if (!interpreter) {
|
||||
return {
|
||||
success: false,
|
||||
stdout: '',
|
||||
stderr: `The subprocess sandbox cannot run "${options.language}" (known: ${Object.keys(INTERPRETERS).join(', ')}).`,
|
||||
exitCode: 1,
|
||||
};
|
||||
}
|
||||
// Loaded through computed specifiers so these never reach a browser
|
||||
// bundle. `require()` is not enough: scripts/webview-gate.mjs counts a
|
||||
// require-call as a static import, and only a true dynamic import keeps
|
||||
// the builtin off the graph. Without this the Agent entry fails the gate
|
||||
// with "Node builtins imported STATICALLY" -- a blank screen at import
|
||||
// time in a webview, which is issue #4437 all over again.
|
||||
const [fs, os, path, childProcess] = await Promise.all([
|
||||
import(/* @vite-ignore */ ['n', 'ode:fs'].join('')) as Promise<typeof import('fs')>,
|
||||
import(/* @vite-ignore */ ['n', 'ode:os'].join('')) as Promise<typeof import('os')>,
|
||||
import(/* @vite-ignore */ ['n', 'ode:path'].join('')) as Promise<typeof import('path')>,
|
||||
import(/* @vite-ignore */ ['n', 'ode:child_process'].join('')) as Promise<typeof import('child_process')>,
|
||||
]);
|
||||
const { spawnSync } = childProcess;
|
||||
|
||||
// Where the code runs and where relative paths resolve. `persistFiles`
|
||||
// reuses a stable workspace so artifacts one call writes are there for
|
||||
// the next; otherwise each call gets a throwaway temp directory. `cwd`
|
||||
// is passed to the child so relative paths resolve against the workspace
|
||||
// rather than the host process directory.
|
||||
//
|
||||
// The default `workingDir` (`/workspace`) is a *container* path from
|
||||
// Python's SandboxConfig; the subprocess runner is host-local, so an
|
||||
// explicit non-default path is honoured verbatim while the container
|
||||
// default maps to a stable per-workspace temp dir instead of the host
|
||||
// filesystem root.
|
||||
const containerDefault = defaultSandboxConfig().workingDir;
|
||||
const explicitDir = config.workingDir && config.workingDir !== containerDefault;
|
||||
let dir: string;
|
||||
let ephemeral: boolean;
|
||||
if (explicitDir) {
|
||||
dir = path.resolve(config.workingDir);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
ephemeral = false;
|
||||
} else if (config.persistFiles) {
|
||||
dir = path.join(os.tmpdir(), 'praison-sandbox-workspace');
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
ephemeral = false;
|
||||
} else {
|
||||
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'praison-sandbox-'));
|
||||
ephemeral = true;
|
||||
}
|
||||
const file = path.join(dir, `code${interpreter.extension}`);
|
||||
try {
|
||||
fs.writeFileSync(file, code, 'utf8');
|
||||
const result = spawnSync(interpreter.command(), interpreter.args(file), {
|
||||
cwd: dir,
|
||||
encoding: 'utf8',
|
||||
timeout: options.config.timeout * 1000,
|
||||
env: { ...process.env, ...options.config.env },
|
||||
});
|
||||
const exitCode = result.status ?? 1;
|
||||
return {
|
||||
success: exitCode === 0 && !result.error,
|
||||
stdout: result.stdout ?? '',
|
||||
stderr: result.error ? String(result.error.message) : (result.stderr ?? ''),
|
||||
exitCode,
|
||||
};
|
||||
} finally {
|
||||
// Only ever remove a temp directory this call created. `autoCleanup`
|
||||
// must never delete a persisted or user-named workspace -- that would
|
||||
// defeat `persistFiles` and could remove host files.
|
||||
if (config.autoCleanup && ephemeral) {
|
||||
try {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
} catch {
|
||||
// A leftover temp directory is not worth failing the execution for.
|
||||
}
|
||||
} else if (config.autoCleanup && !ephemeral && !config.persistFiles) {
|
||||
// A user-named workingDir without persistFiles: clean up only the
|
||||
// code file we wrote, leaving the workspace itself intact.
|
||||
try {
|
||||
fs.rmSync(file, { force: true });
|
||||
} catch {
|
||||
// Non-fatal.
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Build the runner for a config, or throw naming what is registered. */
|
||||
export function createSandboxRunner(config: SandboxConfig): SandboxRunner {
|
||||
const type = config.sandboxType.toLowerCase();
|
||||
const factory = runners.get(type);
|
||||
if (factory) return factory(config);
|
||||
if (type === 'subprocess' || type === 'local') return createSubprocessRunner(config);
|
||||
throw new Error(
|
||||
`No sandbox runner is registered for "${config.sandboxType}" (available: ${sandboxRunnerNames().join(', ')}).\n` +
|
||||
` A remote sandbox is a provider integration: register one with registerSandboxRunner('${config.sandboxType}', ...).`
|
||||
);
|
||||
}
|
||||
|
||||
/** Per-call options of `Agent.executeCode` (Python `execute_code`). */
|
||||
export interface ExecuteCodeOptions {
|
||||
/** `python` (default), `bash`, `javascript`, ... */
|
||||
language?: string;
|
||||
/** Run the static pre-check (default true). */
|
||||
checkSecurity?: boolean;
|
||||
/**
|
||||
* Where to run THIS call: a sandbox type, a config object, or `true` for
|
||||
* the default subprocess sandbox. Overrides the agent's `sandbox`.
|
||||
*/
|
||||
runIn?: boolean | string | Record<string, unknown>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `code` in a sandbox, exactly as Python's `Agent.execute_code` does:
|
||||
* resolve where it runs, analyse it first, execute, and attach the warnings
|
||||
* to the result rather than only logging them.
|
||||
*
|
||||
* Runners are cached per sandbox type in `runners`: building a fresh one per
|
||||
* call would start and tear down a container each time, which for Docker is
|
||||
* seconds of latency and for any backend loses the previous call's files.
|
||||
*/
|
||||
export async function executeCodeInSandbox(params: {
|
||||
code: string;
|
||||
options?: ExecuteCodeOptions;
|
||||
/** The agent's own `sandbox` config, used when the call names no `runIn`. */
|
||||
defaultConfig?: SandboxConfig;
|
||||
runners: Map<string, SandboxRunner>;
|
||||
/** Used only to make the "no sandbox configured" error read naturally. */
|
||||
agentName: string;
|
||||
onWarning?: (message: string) => void;
|
||||
}): Promise<SandboxResult> {
|
||||
const { code, runners, agentName, defaultConfig } = params;
|
||||
const options = params.options ?? {};
|
||||
const language = options.language ?? 'python';
|
||||
const config = options.runIn !== undefined ? resolveSandbox(options.runIn) : defaultConfig;
|
||||
if (!config) {
|
||||
throw new Error(
|
||||
`Agent ${agentName}: no sandbox configured. Either name a place on the call --\n` +
|
||||
" agent.executeCode(code, { runIn: 'subprocess' })\n" +
|
||||
' or set a default for every call with new Agent({ sandbox: true }).'
|
||||
);
|
||||
}
|
||||
|
||||
let warnings: SecurityWarning[] = [];
|
||||
if (options.checkSecurity !== false) {
|
||||
warnings = checkCodeSafety(code, language);
|
||||
if (warnings.length > 0) params.onWarning?.(formatWarnings(warnings));
|
||||
}
|
||||
|
||||
let runner = runners.get(config.sandboxType);
|
||||
if (!runner) {
|
||||
runner = createSandboxRunner(config);
|
||||
runners.set(config.sandboxType, runner);
|
||||
}
|
||||
const result = await runner.execute(code, { language, config });
|
||||
if (warnings.length > 0) {
|
||||
result.metadata = { ...(result.metadata ?? {}), securityWarnings: warnings };
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -149,3 +149,59 @@ export function proposalFromToolArgs(args: Record<string, unknown>): SkillPropos
|
||||
instructions: typeof args.instructions === 'string' ? args.instructions : '',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A tool call as either transport reports it: the OpenAI-compatible shape
|
||||
* (`function.name` + a JSON `arguments` string) or the AI SDK's
|
||||
* (`toolName` + parsed `args`).
|
||||
*/
|
||||
export interface RawSkillToolCall {
|
||||
function?: { name?: string; arguments?: string };
|
||||
toolName?: string;
|
||||
args?: unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one review turn and return what it proposed (Python
|
||||
* `agent/skill_review.py` `SkillReviewMixin`).
|
||||
*
|
||||
* `askModel` performs the isolated call: it is given the review prompt and the
|
||||
* single tool the turn may use, and returns the tool calls the model made.
|
||||
* Keeping the transport outside means the Agent decides how to reach the
|
||||
* model and this stays testable without one.
|
||||
*
|
||||
* Nothing here throws: a review is bookkeeping, and it must never break the
|
||||
* task that produced it. Failures are reported through `onError`.
|
||||
*/
|
||||
export async function runSkillReviewTurn(
|
||||
config: SelfImproveConfig,
|
||||
trajectory: SkillReviewTrajectory,
|
||||
askModel: (prompt: string, tool: typeof SKILL_MANAGE_TOOL) => Promise<RawSkillToolCall[]>,
|
||||
onError: (error: unknown) => void = () => {}
|
||||
): Promise<SkillProposal[]> {
|
||||
if (!config.enabled || !config.policy.shouldReview(trajectory)) return [];
|
||||
try {
|
||||
const calls = await askModel(config.policy.reviewPrompt(trajectory), SKILL_MANAGE_TOOL);
|
||||
const proposals: SkillProposal[] = [];
|
||||
for (const call of calls ?? []) {
|
||||
const name = call.function?.name ?? call.toolName;
|
||||
if (name !== 'skill_manage') continue;
|
||||
const raw = call.function?.arguments ?? call.args;
|
||||
let args: Record<string, unknown>;
|
||||
try {
|
||||
args = typeof raw === 'string' ? JSON.parse(raw) : ((raw as Record<string, unknown>) ?? {});
|
||||
} catch {
|
||||
// A malformed argument blob is one bad proposal, not a failed review.
|
||||
continue;
|
||||
}
|
||||
const proposal = proposalFromToolArgs(args);
|
||||
if (!proposal) continue;
|
||||
proposals.push(proposal);
|
||||
await config.policy.onProposal?.(proposal);
|
||||
}
|
||||
return proposals;
|
||||
} catch (error) {
|
||||
onError(error);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,381 @@
|
||||
/**
|
||||
* Progressive tool disclosure (Python parity: `tools/tool_search.py`,
|
||||
* `Agent(tool_search=...)`).
|
||||
*
|
||||
* A large tool list costs context on every single request, whether or not the
|
||||
* model uses any of it. With tool search on, deferrable tools are replaced in
|
||||
* the request by three bridge tools -- `tool_search`, `tool_describe`,
|
||||
* `tool_call` -- and the model discovers what it needs, when it needs it.
|
||||
*
|
||||
* The architecture invariants are Python's, and each one is load-bearing:
|
||||
*
|
||||
* 1. Core tools never defer ({@link PRAISONAI_CORE_TOOLS}).
|
||||
* 2. Unknown tools stay visible -- a tool is never silently dropped.
|
||||
* 3. The catalog is stateless: rebuilt from the current tool defs every turn.
|
||||
* 4. Bridge dispatch reads the PRE-assembly deferrable set.
|
||||
* 5. The catalog is scoped to this agent's own tools.
|
||||
* 6. `tool_call` is unwrapped BEFORE hooks, approval and events, so they see
|
||||
* the real tool name rather than the bridge.
|
||||
* 7. In `auto` mode tools defer only once their schemas cross
|
||||
* `thresholdPct` of the context window.
|
||||
* 8. BM25 is inlined -- no new dependency -- with a substring fallback.
|
||||
*/
|
||||
|
||||
import { ToolSearchConfig } from '../../config';
|
||||
|
||||
/** An OpenAI function schema, as the Agent stores them. */
|
||||
export type ToolDef = Record<string, any>;
|
||||
|
||||
/** Tools that never defer (Python `PRAISONAI_CORE_TOOLS`). */
|
||||
export const PRAISONAI_CORE_TOOLS: ReadonlySet<string> = new Set([
|
||||
// File operations
|
||||
'read_file', 'write_file', 'list_files', 'get_file_info',
|
||||
'copy_file', 'move_file', 'delete_file',
|
||||
// Shell operations
|
||||
'execute_command', 'list_processes', 'kill_process', 'get_system_info',
|
||||
// Web operations
|
||||
'search_web', 'web_search', 'internet_search', 'web_crawl', 'crawl_web',
|
||||
// Schedule operations
|
||||
'schedule_add', 'schedule_list', 'schedule_remove',
|
||||
// Memory operations
|
||||
'store_memory', 'search_memory',
|
||||
// Human-in-the-loop clarification
|
||||
'clarify',
|
||||
]);
|
||||
|
||||
/** The three bridge tool names. */
|
||||
export const BRIDGE_TOOL_NAMES: ReadonlySet<string> = new Set(['tool_search', 'tool_describe', 'tool_call']);
|
||||
|
||||
function functionOf(toolDef: ToolDef): Record<string, any> {
|
||||
const fn = toolDef?.function;
|
||||
return fn && typeof fn === 'object' ? fn : {};
|
||||
}
|
||||
|
||||
function nameOf(toolDef: ToolDef): string {
|
||||
const name = functionOf(toolDef).name;
|
||||
return typeof name === 'string' ? name : '';
|
||||
}
|
||||
|
||||
/** Python `_is_tool_deferrable`. */
|
||||
function isToolDeferrable(toolDef: ToolDef): boolean {
|
||||
if (toolDef?.__praisonai_deferrable__ === true) return true;
|
||||
const fn = functionOf(toolDef);
|
||||
if (fn.deferrable === true) return true;
|
||||
// Prefix only: a substring test would defer unrelated tools by accident.
|
||||
return nameOf(toolDef).startsWith('mcp_');
|
||||
}
|
||||
|
||||
/** Split tools into "never defer" and "may defer" (Python `classify_tools`). */
|
||||
export function classifyTools(
|
||||
toolDefs: readonly ToolDef[],
|
||||
config: ToolSearchConfig
|
||||
): { core: ToolDef[]; deferrable: ToolDef[] } {
|
||||
const coreNames = config.coreTools ?? PRAISONAI_CORE_TOOLS;
|
||||
const core: ToolDef[] = [];
|
||||
const deferrable: ToolDef[] = [];
|
||||
for (const toolDef of toolDefs) {
|
||||
const name = nameOf(toolDef);
|
||||
if (coreNames.has(name)) {
|
||||
core.push(toolDef);
|
||||
} else if (isToolDeferrable(toolDef)) {
|
||||
deferrable.push(toolDef);
|
||||
} else {
|
||||
// Invariant 2: an unknown tool stays visible rather than vanishing.
|
||||
core.push(toolDef);
|
||||
}
|
||||
}
|
||||
return { core, deferrable };
|
||||
}
|
||||
|
||||
/** Rough schema cost (Python `estimate_tool_schema_tokens`: ~3.5 chars/token). */
|
||||
export function estimateToolSchemaTokens(toolDefs: readonly ToolDef[]): number {
|
||||
if (toolDefs.length === 0) return 0;
|
||||
let totalChars = 0;
|
||||
for (const toolDef of toolDefs) {
|
||||
try {
|
||||
totalChars += JSON.stringify(toolDef).length;
|
||||
} catch {
|
||||
totalChars += String(toolDef).length;
|
||||
}
|
||||
}
|
||||
return Math.trunc(totalChars / 3.5);
|
||||
}
|
||||
|
||||
/** Python `should_defer_tools`: the `auto` threshold rule. */
|
||||
export function shouldDeferTools(
|
||||
deferrable: readonly ToolDef[],
|
||||
config: ToolSearchConfig,
|
||||
contextLength?: number
|
||||
): boolean {
|
||||
const enabled = config.enabled;
|
||||
if (enabled === 'off' || enabled === false) return false;
|
||||
if (enabled === 'on' || enabled === true) return true;
|
||||
if (enabled !== 'auto') return false;
|
||||
if (deferrable.length === 0) return false;
|
||||
const contextLimit = contextLength ?? 20000;
|
||||
const thresholdTokens = Math.trunc(contextLimit * (config.thresholdPct / 100));
|
||||
return estimateToolSchemaTokens(deferrable) >= thresholdTokens;
|
||||
}
|
||||
|
||||
/** One catalog row the model can search. */
|
||||
export interface CatalogEntry {
|
||||
name: string;
|
||||
description: string;
|
||||
}
|
||||
|
||||
function catalogOf(deferrable: readonly ToolDef[]): CatalogEntry[] {
|
||||
const catalog: CatalogEntry[] = [];
|
||||
for (const toolDef of deferrable) {
|
||||
const fn = functionOf(toolDef);
|
||||
const name = typeof fn.name === 'string' ? fn.name : '';
|
||||
if (!name) continue;
|
||||
catalog.push({ name, description: typeof fn.description === 'string' ? fn.description : '' });
|
||||
}
|
||||
return catalog;
|
||||
}
|
||||
|
||||
function tokenize(text: string): string[] {
|
||||
if (!text) return [];
|
||||
return text.toLowerCase().match(/\b[a-z0-9_]+\b/g) ?? [];
|
||||
}
|
||||
|
||||
/**
|
||||
* BM25 over the tool catalog, inlined so tool search adds no dependency
|
||||
* (Python `BM25ToolSearcher`, same k1/b constants).
|
||||
*/
|
||||
export class BM25ToolSearcher {
|
||||
private readonly termFrequencies: Array<Map<string, number>> = [];
|
||||
private readonly docFrequencies = new Map<string, number>();
|
||||
private readonly totalDocs: number;
|
||||
private readonly avgDocLength: number;
|
||||
|
||||
constructor(private readonly catalog: readonly CatalogEntry[]) {
|
||||
this.totalDocs = catalog.length;
|
||||
let totalLength = 0;
|
||||
for (const item of catalog) {
|
||||
const tokens = tokenize(`${item.name} ${item.description}`);
|
||||
const tf = new Map<string, number>();
|
||||
for (const token of tokens) tf.set(token, (tf.get(token) ?? 0) + 1);
|
||||
this.termFrequencies.push(tf);
|
||||
totalLength += tokens.length;
|
||||
for (const token of new Set(tokens)) {
|
||||
this.docFrequencies.set(token, (this.docFrequencies.get(token) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
this.avgDocLength = catalog.length > 0 ? totalLength / catalog.length : 0;
|
||||
}
|
||||
|
||||
search(query: string, limit: number = 5): CatalogEntry[] {
|
||||
const queryTokens = tokenize(query);
|
||||
if (queryTokens.length === 0 || this.avgDocLength === 0) return [];
|
||||
const k1 = 1.5;
|
||||
const b = 0.75;
|
||||
const scored: Array<{ score: number; item: CatalogEntry }> = [];
|
||||
for (let i = 0; i < this.catalog.length; i++) {
|
||||
const tf = this.termFrequencies[i];
|
||||
let docLength = 0;
|
||||
for (const count of tf.values()) docLength += count;
|
||||
let score = 0;
|
||||
for (const token of queryTokens) {
|
||||
const termFreq = tf.get(token);
|
||||
if (termFreq === undefined) continue;
|
||||
const df = this.docFrequencies.get(token) ?? 0;
|
||||
const idf = Math.log((this.totalDocs - df + 0.5) / (df + 0.5));
|
||||
score += idf * (termFreq * (k1 + 1)) / (termFreq + k1 * (1 - b + b * (docLength / this.avgDocLength)));
|
||||
}
|
||||
if (score > 0) scored.push({ score, item: this.catalog[i] });
|
||||
}
|
||||
scored.sort((a, b2) => b2.score - a.score);
|
||||
return scored.slice(0, limit).map((s) => s.item);
|
||||
}
|
||||
}
|
||||
|
||||
/** Python `search_catalog`: BM25 for a substantial query, substring otherwise. */
|
||||
export function searchCatalog(
|
||||
deferrable: readonly ToolDef[],
|
||||
query: string,
|
||||
limit: number = 5
|
||||
): CatalogEntry[] {
|
||||
if (deferrable.length === 0 || query.trim().length === 0) return [];
|
||||
const catalog = catalogOf(deferrable);
|
||||
if (catalog.length === 0) return [];
|
||||
|
||||
if (query.trim().length >= 3) {
|
||||
const results = new BM25ToolSearcher(catalog).search(query, limit);
|
||||
if (results.length > 0) return results;
|
||||
}
|
||||
|
||||
const queryLower = query.toLowerCase();
|
||||
const matches: CatalogEntry[] = [];
|
||||
for (const item of catalog) {
|
||||
if (item.name.toLowerCase().includes(queryLower) || item.description.toLowerCase().includes(queryLower)) {
|
||||
matches.push(item);
|
||||
if (matches.length >= limit) break;
|
||||
}
|
||||
}
|
||||
return matches;
|
||||
}
|
||||
|
||||
/** The three bridge tool schemas (Python `bridge_tool_schemas`). */
|
||||
export function bridgeToolSchemas(): ToolDef[] {
|
||||
return [
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'tool_search',
|
||||
description: 'Search for available tools by name or functionality. Use this to discover what tools are available before using them.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
query: { type: 'string', description: 'Search query describing the tool functionality you need' },
|
||||
limit: { type: 'integer', description: 'Maximum number of results to return', default: 5, minimum: 1, maximum: 20 },
|
||||
},
|
||||
required: ['query'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'tool_describe',
|
||||
description: 'Get the full schema and documentation for a specific tool. Use this after tool_search to understand how to use a tool.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
tool_name: { type: 'string', description: 'Exact name of the tool to describe' },
|
||||
},
|
||||
required: ['tool_name'],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'tool_call',
|
||||
description: 'Execute a tool with the given arguments. Use this after tool_describe to actually run the tool.',
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
tool_name: { type: 'string', description: 'Exact name of the tool to execute' },
|
||||
tool_args: { type: 'object', description: 'Arguments to pass to the tool', additionalProperties: true },
|
||||
},
|
||||
required: ['tool_name', 'tool_args'],
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
/** What the turn needs to know once tools have been assembled. */
|
||||
export interface ToolSearchAssembly {
|
||||
/** The tool list actually sent to the model. */
|
||||
tools: ToolDef[];
|
||||
/** Whether deferrable tools were replaced by the bridge. */
|
||||
bridgeMode: boolean;
|
||||
/** Tools kept out of the request; the bridge resolves calls against these. */
|
||||
deferrableTools: ToolDef[];
|
||||
/** Name/description rows the model can search. */
|
||||
catalog: CatalogEntry[];
|
||||
config: ToolSearchConfig;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide between bridge mode and pass-through, and build the tool list for
|
||||
* the request (Python `assemble_tool_defs`).
|
||||
*/
|
||||
export function assembleToolDefs(
|
||||
toolDefs: readonly ToolDef[] | undefined,
|
||||
config: ToolSearchConfig,
|
||||
contextLength?: number
|
||||
): ToolSearchAssembly {
|
||||
const tools = toolDefs ? [...toolDefs] : [];
|
||||
const passThrough: ToolSearchAssembly = {
|
||||
tools, bridgeMode: false, deferrableTools: [], catalog: [], config,
|
||||
};
|
||||
if (config.enabled === 'off' || config.enabled === false) return passThrough;
|
||||
|
||||
const { core, deferrable } = classifyTools(tools, config);
|
||||
if (deferrable.length === 0 || !shouldDeferTools(deferrable, config, contextLength)) return passThrough;
|
||||
|
||||
return {
|
||||
tools: [...core, ...bridgeToolSchemas()],
|
||||
bridgeMode: true,
|
||||
deferrableTools: deferrable,
|
||||
catalog: catalogOf(deferrable),
|
||||
config,
|
||||
};
|
||||
}
|
||||
|
||||
/** Python `dispatch_tool_search`. */
|
||||
export function dispatchToolSearch(
|
||||
query: string,
|
||||
limit: number | undefined,
|
||||
deferrable: readonly ToolDef[],
|
||||
config: ToolSearchConfig
|
||||
): Record<string, unknown> {
|
||||
let searchLimit = limit ?? config.searchDefaultLimit;
|
||||
searchLimit = Math.min(searchLimit, config.maxSearchLimit);
|
||||
searchLimit = Math.max(searchLimit, 1);
|
||||
return {
|
||||
query,
|
||||
results: searchCatalog(deferrable, query, searchLimit),
|
||||
total_available: deferrable.length,
|
||||
};
|
||||
}
|
||||
|
||||
/** Python `dispatch_tool_describe`. */
|
||||
export function dispatchToolDescribe(
|
||||
toolName: string,
|
||||
deferrable: readonly ToolDef[]
|
||||
): Record<string, unknown> {
|
||||
for (const toolDef of deferrable) {
|
||||
if (nameOf(toolDef) === toolName) {
|
||||
return { tool_name: toolName, schema: toolDef, found: true };
|
||||
}
|
||||
}
|
||||
return {
|
||||
tool_name: toolName,
|
||||
error: `Tool '${toolName}' not found in available tools`,
|
||||
found: false,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Unwrap a `tool_call` bridge invocation into the real call
|
||||
* (Python `resolve_underlying_call`). Anything else is returned unchanged.
|
||||
*/
|
||||
export function resolveUnderlyingCall(
|
||||
toolName: string,
|
||||
toolArgs: Record<string, unknown>
|
||||
): { name: string; args: Record<string, unknown> } {
|
||||
if (toolName !== 'tool_call') return { name: toolName, args: toolArgs };
|
||||
if (toolArgs === null || typeof toolArgs !== 'object' || Array.isArray(toolArgs)) {
|
||||
throw new TypeError(
|
||||
'tool_call expects an object for tool_args. Ensure the model output is properly formatted.'
|
||||
);
|
||||
}
|
||||
const realName = toolArgs.tool_name;
|
||||
if (typeof realName !== 'string' || realName.length === 0) {
|
||||
throw new Error("tool_call requires a 'tool_name' parameter");
|
||||
}
|
||||
const realArgs = toolArgs.tool_args;
|
||||
return {
|
||||
name: realName,
|
||||
args: realArgs !== null && typeof realArgs === 'object' && !Array.isArray(realArgs)
|
||||
? (realArgs as Record<string, unknown>)
|
||||
: {},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the constructor option into a config, or `undefined` when tool
|
||||
* search is off. Mirrors `ToolSearchConfig.fromRaw` plus the `false` fast path.
|
||||
*/
|
||||
export function resolveToolSearch(
|
||||
input: boolean | string | Record<string, unknown> | undefined | null
|
||||
): ToolSearchConfig | undefined {
|
||||
if (input === undefined || input === null || input === false) return undefined;
|
||||
const config = ToolSearchConfig.fromRaw(input);
|
||||
if (config.enabled === 'off' || config.enabled === false) return undefined;
|
||||
return config;
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* Named toolset groups on an Agent (Python parity: `Agent(toolsets=[...])`,
|
||||
* `agent/agent.py` lines 2358-2392, over `toolsets.py`).
|
||||
*
|
||||
* `toolsets: ['web']` resolves the named group to its tool names, drops the
|
||||
* ones the agent already has, and attaches the rest. Resolution and
|
||||
* attachment are deliberately split:
|
||||
*
|
||||
* - {@link resolveToolsetToolNames} runs in the CONSTRUCTOR and throws on an
|
||||
* unknown toolset name, exactly as Python does — a typo must not survive
|
||||
* until the first model turn.
|
||||
* - {@link loadToolsetTools} runs on first use, because each builtin lives
|
||||
* behind a dynamic import and pulling four provider SDKs into the module
|
||||
* graph for an agent that never calls one is a cost nobody asked for.
|
||||
*
|
||||
* A Python tool name with no TypeScript builtin is reported and skipped:
|
||||
* `resolveToolsetBuiltinIds` never invents a tool, and silently pretending a
|
||||
* toolset was attached would be worse than saying it was not.
|
||||
*/
|
||||
|
||||
import { resolveToolsetsForModel, toolsetToolId } from '../../toolsets';
|
||||
|
||||
/**
|
||||
* Python tool name → the TypeScript builtin module and factory that provides
|
||||
* it. Keyed by the registry id `toolsetToolId()` returns, so the two mappings
|
||||
* cannot drift apart.
|
||||
*/
|
||||
const BUILTIN_TOOL_FACTORIES: Readonly<Record<string, { module: string; entry: string }>> = Object.freeze({
|
||||
'tavily-search': { module: 'tavily', entry: 'tavilySearch' },
|
||||
exa: { module: 'exa', entry: 'exaSearch' },
|
||||
'code-execution': { module: 'code-execution', entry: 'codeExecution' },
|
||||
'firecrawl-scrape': { module: 'firecrawl', entry: 'firecrawlScrape' },
|
||||
'firecrawl-crawl': { module: 'firecrawl', entry: 'firecrawlCrawl' },
|
||||
});
|
||||
|
||||
/**
|
||||
* The tool names a list of toolsets resolves to, ordered for `model`'s family
|
||||
* (Python `resolve_toolsets_for_model`). Throws on an unknown toolset name.
|
||||
*/
|
||||
export function resolveToolsetToolNames(names: readonly string[], model?: string): string[] {
|
||||
if (names.length === 0) return [];
|
||||
return resolveToolsetsForModel([...names], model ?? null);
|
||||
}
|
||||
|
||||
/** A tool object the Agent can accept (`name` + `execute`). */
|
||||
export interface ToolsetTool {
|
||||
name: string;
|
||||
description?: string;
|
||||
parameters?: Record<string, unknown>;
|
||||
execute: (...args: unknown[]) => unknown;
|
||||
}
|
||||
|
||||
async function loadFactory(module: string, entry: string): Promise<((config?: unknown) => unknown) | null> {
|
||||
for (const suffix of ['/index.js', '', '.js']) {
|
||||
const specifier = ['..', '..', 'tools', 'builtins', module].join('/') + (suffix === '/index.js' ? '' : suffix);
|
||||
try {
|
||||
const loaded: Record<string, unknown> = await import(specifier);
|
||||
if (typeof loaded[entry] === 'function') return loaded[entry] as (config?: unknown) => unknown;
|
||||
} catch {
|
||||
// Try the next specifier shape; the caller is told if none worked.
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Instantiate the TypeScript builtins behind `toolNames`, skipping those the
|
||||
* agent already has. `onWarning` is called once per name that has no
|
||||
* TypeScript builtin, so a partially-supported toolset is audible.
|
||||
*/
|
||||
export async function loadToolsetTools(
|
||||
toolNames: readonly string[],
|
||||
existingToolNames: ReadonlySet<string>,
|
||||
onWarning: (message: string) => void = () => {}
|
||||
): Promise<ToolsetTool[]> {
|
||||
const tools: ToolsetTool[] = [];
|
||||
const seenIds = new Set<string>();
|
||||
const missing: string[] = [];
|
||||
|
||||
for (const toolName of toolNames) {
|
||||
if (existingToolNames.has(toolName)) continue;
|
||||
const id = toolsetToolId(toolName);
|
||||
if (id === null) {
|
||||
missing.push(toolName);
|
||||
continue;
|
||||
}
|
||||
if (seenIds.has(id)) continue;
|
||||
seenIds.add(id);
|
||||
const spec = BUILTIN_TOOL_FACTORIES[id];
|
||||
if (!spec) {
|
||||
missing.push(toolName);
|
||||
continue;
|
||||
}
|
||||
const factory = await loadFactory(spec.module, spec.entry);
|
||||
if (!factory) {
|
||||
onWarning(`The toolset tool "${toolName}" could not be loaded from the "${spec.module}" builtin.`);
|
||||
continue;
|
||||
}
|
||||
const tool = factory() as ToolsetTool;
|
||||
if (!tool || typeof tool.execute !== 'function' || existingToolNames.has(tool.name)) continue;
|
||||
tools.push(tool);
|
||||
}
|
||||
|
||||
if (missing.length > 0) {
|
||||
onWarning(
|
||||
`These toolset tools have no TypeScript builtin and were not attached: ${missing.join(', ')}.`
|
||||
);
|
||||
}
|
||||
return tools;
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -151,15 +151,18 @@ export async function resolveBackend(
|
||||
try {
|
||||
// The AI-SDK backend keys credentials by provider id
|
||||
// (config.providers[providerId]), while ProviderConfig is flat
|
||||
// (apiKey/baseUrl). Map the flat per-agent key onto the resolved
|
||||
// provider so a programmatic apiKey authenticates here too -- not
|
||||
// only via a provider env var.
|
||||
// (apiKey/baseUrl/fetch). Map the flat per-agent values onto the
|
||||
// resolved provider so a programmatic apiKey authenticates here too --
|
||||
// not only via a provider env var. `fetch` carries any subscription
|
||||
// `auth` headers wrapped onto it, so it must reach the AI SDK provider
|
||||
// or a subscription route would drop the headers the provider requires.
|
||||
const providers =
|
||||
options.config?.apiKey || options.config?.baseUrl
|
||||
options.config?.apiKey || options.config?.baseUrl || options.config?.fetch
|
||||
? {
|
||||
[providerId]: {
|
||||
...(options.config.apiKey ? { apiKey: options.config.apiKey } : {}),
|
||||
...(options.config.baseUrl ? { baseURL: options.config.baseUrl } : {}),
|
||||
...(options.config.fetch ? { fetch: options.config.fetch } : {}),
|
||||
},
|
||||
}
|
||||
: undefined;
|
||||
|
||||
@@ -33,13 +33,6 @@
|
||||
* The list is the work queue, and it is meant to shrink to nothing.
|
||||
*/
|
||||
export const UNHONOURED_OPTIONS: Readonly<Record<string, readonly string[]>> = {
|
||||
'Agent.__init__': [
|
||||
'auth', 'toolsets', 'reflection', 'autonomy', 'templates', 'selfImprove', 'toolConfig',
|
||||
'learn', 'backend', 'runOn', 'toolsRunOn', 'runtime', 'toolSearch', 'messageSteering', 'sandbox',
|
||||
],
|
||||
'Agent.chat': [
|
||||
'reasoningSteps', 'taskName', 'taskDescription', 'taskId', 'config', 'attachments',
|
||||
],
|
||||
'AgentTeam.__init__': [
|
||||
'managerLlm', 'memory', 'planning', 'context', 'execution', 'hooks', 'autonomy', 'knowledge',
|
||||
'guardrails', 'web', 'reflection', 'caching', 'learn', 'toolsRunOn', 'runOn',
|
||||
|
||||
@@ -0,0 +1,613 @@
|
||||
/**
|
||||
* Behaviour parity for the `Agent.__init__` options that decide WHERE and HOW
|
||||
* a turn runs: `backend`, `runOn`, `toolsRunOn`, `runtime`, `sandbox`,
|
||||
* `autonomy`, `toolConfig`, `messageSteering`, `selfImprove`, `auth`.
|
||||
*
|
||||
* Every option is proved to change what the code does, each next to a control.
|
||||
*/
|
||||
process.env.PRAISONAI_PARITY_SILENT = '1';
|
||||
|
||||
import { Agent } from '../../../../src/agent/simple';
|
||||
import { ApprovalManager } from '../../../../src/ai/tool-approval';
|
||||
import {
|
||||
registerManagedRuntime,
|
||||
registerToolPlace,
|
||||
unregisterManagedRuntime,
|
||||
unregisterToolPlace,
|
||||
type ToolPlaceLike,
|
||||
} from '../../../../src/agent/features/placement';
|
||||
import { checkCodeSafety } from '../../../../src/agent/features/sandbox';
|
||||
import { SteeringPriority } from '../../../../src/agent/features/message-steering';
|
||||
import { registerRuntime, unregisterRuntime } from '../../../../src/runtime';
|
||||
import { registerAuthProvider, resetAuthProviders } from '../../../../src/llm';
|
||||
|
||||
const mockLlm = {
|
||||
calls: [] as Array<{ method: string; args: any[] }>,
|
||||
chatQueue: [] as any[],
|
||||
textQueue: [] as any[],
|
||||
};
|
||||
|
||||
jest.mock('../../../../src/llm/openai', () => ({
|
||||
OpenAIService: jest.fn().mockImplementation((model: string, opts: any) => {
|
||||
mockLlm.calls.push({ method: 'construct', args: [model, opts] });
|
||||
const next = (queue: any[], fallback: any) => {
|
||||
const item = queue.length > 0 ? queue.shift() : fallback;
|
||||
if (item instanceof Error) throw item;
|
||||
return item;
|
||||
};
|
||||
return {
|
||||
generateText: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'generateText', args });
|
||||
return next(mockLlm.textQueue, 'text-response');
|
||||
}),
|
||||
generateChat: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'generateChat', args });
|
||||
return next(mockLlm.chatQueue, { content: 'chat-response', role: 'assistant' });
|
||||
}),
|
||||
streamChat: jest.fn(async (messages: any, temperature: number, onToken: (t: string) => void) => {
|
||||
mockLlm.calls.push({ method: 'streamChat', args: [messages, temperature] });
|
||||
onToken('streamed');
|
||||
return 'streamed';
|
||||
}),
|
||||
streamChatWithTools: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'streamChatWithTools', args });
|
||||
return next(mockLlm.chatQueue, { content: 'stream-tools-response', role: 'assistant' });
|
||||
}),
|
||||
};
|
||||
}),
|
||||
}));
|
||||
|
||||
const quiet = { verbose: false, stream: false } as const;
|
||||
const lastCall = () => mockLlm.calls[mockLlm.calls.length - 1];
|
||||
const callsOf = (method: string) => mockLlm.calls.filter((c) => c.method === method);
|
||||
const toolCallTurn = (name: string, args: Record<string, unknown>, id = 'call_1') => ({
|
||||
content: '',
|
||||
role: 'assistant',
|
||||
tool_calls: [{ id, type: 'function', function: { name, arguments: JSON.stringify(args) } }],
|
||||
});
|
||||
/** Every tool-result message the recorded requests carried. */
|
||||
const toolResults = () =>
|
||||
mockLlm.calls
|
||||
.flatMap((c) => (Array.isArray(c.args[0]) ? c.args[0] : []))
|
||||
.filter((m: any) => m?.role === 'tool');
|
||||
|
||||
beforeEach(() => {
|
||||
mockLlm.calls = [];
|
||||
mockLlm.chatQueue = [];
|
||||
mockLlm.textQueue = [];
|
||||
});
|
||||
|
||||
describe('Agent: backend and runOn', () => {
|
||||
it('backend takes the whole turn: the local transport is never used', async () => {
|
||||
const backend = { execute: jest.fn(async () => 'from-backend') };
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, backend });
|
||||
await expect(agent.chat('hi')).resolves.toBe('from-backend');
|
||||
expect(backend.execute).toHaveBeenCalledTimes(1);
|
||||
expect(callsOf('generateText')).toHaveLength(0);
|
||||
expect(agent.getManagedBackend()).toBe(backend);
|
||||
});
|
||||
|
||||
it('control: without backend the turn runs on the local transport', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await expect(agent.chat('hi')).resolves.toBe('text-response');
|
||||
expect(callsOf('generateText')).toHaveLength(1);
|
||||
expect(agent.getManagedBackend()).toBeUndefined();
|
||||
});
|
||||
|
||||
it('a streaming backend feeds tokens to the caller', async () => {
|
||||
const backend = {
|
||||
execute: jest.fn(async () => 'unused'),
|
||||
// eslint-disable-next-line require-yield
|
||||
async *stream() { yield 'he'; yield 'llo'; },
|
||||
};
|
||||
const agent = new Agent({ instructions: 'x', verbose: false, stream: true, backend });
|
||||
const tokens: string[] = [];
|
||||
await expect(agent.start('hi', undefined, (t) => tokens.push(t))).resolves.toBe('hello');
|
||||
expect(tokens).toEqual(['he', 'llo']);
|
||||
expect(backend.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('runOn resolves a registered managed runtime and delegates to it', async () => {
|
||||
const hosted = { execute: jest.fn(async () => 'hosted-answer') };
|
||||
registerManagedRuntime('test-cloud', () => hosted);
|
||||
try {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, runOn: 'test-cloud' });
|
||||
await expect(agent.chat('hi')).resolves.toBe('hosted-answer');
|
||||
expect(agent.getManagedBackend()).toBe(hosted);
|
||||
} finally {
|
||||
unregisterManagedRuntime('test-cloud');
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects runOn naming a place that only runs tools, pointing at toolsRunOn', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, runOn: 'local' }))
|
||||
.toThrow(/cannot host an agent loop[\s\S]*toolsRunOn/);
|
||||
});
|
||||
|
||||
it('rejects an unregistered runOn instead of running locally and billing the wrong place', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, runOn: 'anthropic' }))
|
||||
.toThrow(/not a known managed runtime/);
|
||||
});
|
||||
|
||||
it('rejects the combinations that name two places for one thing', () => {
|
||||
const backend = { execute: async () => 'x' };
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, backend, toolsRunOn: 'local' }))
|
||||
.toThrow(/points the tools at two machines/);
|
||||
registerManagedRuntime('test-cloud', () => backend);
|
||||
try {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, runOn: 'test-cloud', backend }))
|
||||
.toThrow(/sets the agent's runtime twice/);
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, runOn: 'test-cloud', toolsRunOn: 'local' }))
|
||||
.toThrow(/points the tools at two machines/);
|
||||
} finally {
|
||||
unregisterManagedRuntime('test-cloud');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: toolsRunOn', () => {
|
||||
const recordingPlace = (log: string[]): ToolPlaceLike => ({
|
||||
placeName: 'test-box',
|
||||
async runTool(name, args, run) {
|
||||
log.push(`${name}(${JSON.stringify(args)})`);
|
||||
return `remote:${await run()}`;
|
||||
},
|
||||
});
|
||||
|
||||
it('routes every tool call through the named place', async () => {
|
||||
const log: string[] = [];
|
||||
registerToolPlace('test-box', () => recordingPlace(log));
|
||||
try {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, toolsRunOn: 'test-box',
|
||||
toolFunctions: { lookup: (args: any) => `found ${args.term}` },
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('lookup', { term: 'x' }), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('find x');
|
||||
expect(log).toEqual(['lookup({"term":"x"})']);
|
||||
expect(toolResults()[0].content).toBe('remote:found x');
|
||||
} finally {
|
||||
unregisterToolPlace('test-box');
|
||||
}
|
||||
});
|
||||
|
||||
it('control: without toolsRunOn the tool runs in process, unwrapped', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
toolFunctions: { lookup: (args: any) => `found ${args.term}` },
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('lookup', { term: 'x' }), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('find x');
|
||||
expect(toolResults()[0].content).toBe('found x');
|
||||
expect(agent.getToolPlace()).toBeUndefined();
|
||||
});
|
||||
|
||||
it('rejects an unregistered place at construction rather than running tools locally', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, toolsRunOn: 'docker' }))
|
||||
.toThrow(/not a known place/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: runtime', () => {
|
||||
const stubRuntime = (overrides: Record<string, unknown> = {}) => ({
|
||||
runtimeName: 'test-runtime',
|
||||
runtimeVersion: '1.0',
|
||||
capabilities: () => ({ streaming: true }),
|
||||
supports: () => true,
|
||||
runTurn: jest.fn(async () => ({ content: 'runtime-answer', metadata: {}, error: null })),
|
||||
streamTurn: async function* () { /* not used */ },
|
||||
executeAgent: async () => ({}),
|
||||
streamAgent: async function* () { /* not used */ },
|
||||
validateConfig: async () => [],
|
||||
healthCheck: async () => ({ status: 'ok' }),
|
||||
...overrides,
|
||||
});
|
||||
|
||||
it('delegates the whole turn to the named runtime', async () => {
|
||||
const runtime = stubRuntime();
|
||||
registerRuntime('test-runtime', () => runtime as any);
|
||||
try {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, runtime: 'test-runtime' });
|
||||
await expect(agent.chat('hi')).resolves.toBe('runtime-answer');
|
||||
expect(runtime.runTurn).toHaveBeenCalledTimes(1);
|
||||
expect(callsOf('generateText')).toHaveLength(0);
|
||||
const options = (runtime.runTurn as jest.Mock).mock.calls[0][1];
|
||||
expect(options.modelRef).toBe(agent.getModel());
|
||||
expect(options.systemPrompt).toContain('x');
|
||||
} finally {
|
||||
unregisterRuntime('test-runtime');
|
||||
}
|
||||
});
|
||||
|
||||
it('control: without runtime the local loop runs', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('hi');
|
||||
expect(agent.getRuntime()).toBeUndefined();
|
||||
expect(callsOf('generateText')).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('forwards configOverrides on every turn', async () => {
|
||||
const runtime = stubRuntime();
|
||||
registerRuntime('test-runtime', () => runtime as any);
|
||||
try {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
runtime: { runtime: 'test-runtime', config_overrides: { workspace: '/tmp/x' } },
|
||||
});
|
||||
await agent.chat('hi');
|
||||
expect((runtime.runTurn as jest.Mock).mock.calls[0][1].workspace).toBe('/tmp/x');
|
||||
} finally {
|
||||
unregisterRuntime('test-runtime');
|
||||
}
|
||||
});
|
||||
|
||||
it('fails fast when the runtime lacks a required capability', () => {
|
||||
const runtime = stubRuntime({ capabilities: () => ({ streaming: false }) });
|
||||
registerRuntime('test-runtime', () => runtime as any);
|
||||
try {
|
||||
expect(() => new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
runtime: { runtime: 'test-runtime', requiredCapabilities: ['streaming'] },
|
||||
})).toThrow(/does not provide the required capability: streaming/);
|
||||
} finally {
|
||||
unregisterRuntime('test-runtime');
|
||||
}
|
||||
});
|
||||
|
||||
it('surfaces a runtime error instead of returning its empty content as an answer', async () => {
|
||||
const runtime = stubRuntime({
|
||||
runTurn: async () => ({ content: '', metadata: {}, error: 'the sandbox is offline' }),
|
||||
});
|
||||
registerRuntime('test-runtime', () => runtime as any);
|
||||
try {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, runtime: 'test-runtime' });
|
||||
await expect(agent.chat('hi')).rejects.toThrow(/the sandbox is offline/);
|
||||
expect(agent.lastStopReason).toBe('error');
|
||||
} finally {
|
||||
unregisterRuntime('test-runtime');
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects an unknown runtime id at construction', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, runtime: 'no-such-runtime' }))
|
||||
.toThrow(/Unknown runtime/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: autonomy', () => {
|
||||
it('full_auto approves every tool call without a prompt', async () => {
|
||||
let ran = 0;
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, autonomy: 'full_auto',
|
||||
toolFunctions: { deploy: () => { ran += 1; return 'deployed'; } },
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('deploy', {}), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('ship it');
|
||||
expect(ran).toBe(1);
|
||||
expect(agent.getAutonomy()!.level).toBe('full_auto');
|
||||
// full_auto is the iterative mode in Python, and it tracks changes.
|
||||
expect(agent.getAutonomy()!.mode).toBe('iterative');
|
||||
expect(agent.getAutonomy()!.trackChanges).toBe(true);
|
||||
});
|
||||
|
||||
it('auto_edit approves reads and edits but gates anything else', async () => {
|
||||
const decisions: string[] = [];
|
||||
// The agent's own manager, so the test answers the prompts rather than a
|
||||
// terminal: the point is WHICH calls reach it at all.
|
||||
const manager = new ApprovalManager();
|
||||
manager.onApprovalRequest(async (request: any) => {
|
||||
decisions.push(request.toolName);
|
||||
return false;
|
||||
});
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, autonomy: 'auto_edit', approval: manager,
|
||||
toolFunctions: { read_file: () => 'contents', deploy: () => 'deployed' },
|
||||
});
|
||||
mockLlm.chatQueue = [
|
||||
{ content: '', role: 'assistant', tool_calls: [
|
||||
{ id: 'a', type: 'function', function: { name: 'read_file', arguments: '{}' } },
|
||||
{ id: 'b', type: 'function', function: { name: 'deploy', arguments: '{}' } },
|
||||
] },
|
||||
{ content: 'done', role: 'assistant' },
|
||||
];
|
||||
await agent.chat('go');
|
||||
expect(decisions).toEqual(['deploy']);
|
||||
const results = toolResults();
|
||||
expect(results.find((r: any) => r.name === 'read_file').content).toBe('contents');
|
||||
expect(results.find((r: any) => r.name === 'deploy').content).toMatch(/denied by the approval gate/);
|
||||
});
|
||||
|
||||
it('control: no autonomy means no approval gate at all', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
expect((agent as any).approvalManager).toBeUndefined();
|
||||
expect(agent.getAutonomy()).toBeUndefined();
|
||||
});
|
||||
|
||||
it('breaks a doom loop instead of burning the iteration budget', async () => {
|
||||
let ran = 0;
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, autonomy: { level: 'full_auto', doomLoopThreshold: 3 },
|
||||
toolFunctions: { probe: () => { ran += 1; return 'nothing'; } },
|
||||
});
|
||||
// The model asks for the identical call over and over.
|
||||
mockLlm.chatQueue = [
|
||||
toolCallTurn('probe', { id: 1 }, 'c1'),
|
||||
toolCallTurn('probe', { id: 1 }, 'c2'),
|
||||
toolCallTurn('probe', { id: 1 }, 'c3'),
|
||||
toolCallTurn('probe', { id: 1 }, 'c4'),
|
||||
{ content: 'giving up', role: 'assistant' },
|
||||
];
|
||||
await agent.chat('probe it');
|
||||
// Two identical calls ran; the third was refused with a recovery note.
|
||||
expect(ran).toBe(2);
|
||||
const blocked = toolResults().filter((r: any) => r.content.includes('Doom loop detected'));
|
||||
expect(blocked.length).toBeGreaterThan(0);
|
||||
expect(blocked[0].content).toContain('Try a different approach');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: toolConfig', () => {
|
||||
it('fails a tool that exceeds the timeout instead of hanging the turn', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, toolConfig: { timeout: 0.02 },
|
||||
toolFunctions: {
|
||||
// unref'd: the tool is abandoned when the timeout fires, and a live
|
||||
// timer would otherwise keep the jest worker alive past the suite.
|
||||
slow: () => new Promise((resolve) => {
|
||||
const timer = setTimeout(() => resolve('late'), 200);
|
||||
timer.unref?.();
|
||||
}),
|
||||
},
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('slow', {}), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('go');
|
||||
expect(toolResults()[0].content).toMatch(/timed out after 0\.02s/);
|
||||
});
|
||||
|
||||
it('retries a failing tool up to maxAttempts', async () => {
|
||||
let attempts = 0;
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
toolConfig: { retryPolicy: { maxAttempts: 3, initialDelay: 0, backoffFactor: 1, maxDelay: 0 } },
|
||||
toolFunctions: {
|
||||
flaky: () => {
|
||||
attempts += 1;
|
||||
if (attempts < 3) throw new Error('transient');
|
||||
return 'ok';
|
||||
},
|
||||
},
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('flaky', {}), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('go');
|
||||
expect(attempts).toBe(3);
|
||||
expect(toolResults()[0].content).toBe('ok');
|
||||
});
|
||||
|
||||
it('truncates tool output past the configured budget', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, toolConfig: { outputLimit: 40, outputDirection: 'head' },
|
||||
toolFunctions: { dump: () => 'y'.repeat(500) },
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('dump', {}), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('go');
|
||||
const content = toolResults()[0].content;
|
||||
expect(content.length).toBeLessThan(500);
|
||||
expect(content).toContain('bytes truncated');
|
||||
});
|
||||
|
||||
it('forwards `parallel` to the model as parallel_tool_calls', async () => {
|
||||
// Python's ToolConfig.parallel is the deprecated alias for
|
||||
// ExecutionConfig.parallel_tool_calls, a REQUEST field: it tells the model
|
||||
// whether it may batch tool calls.
|
||||
const noopFetch = (async () => new Response('{}')) as unknown as typeof fetch;
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, fetch: noopFetch, toolConfig: { parallel: false },
|
||||
});
|
||||
expect((agent as any).requestExtras()).toEqual({ parallel_tool_calls: false });
|
||||
});
|
||||
|
||||
it('control: without toolConfig the output is passed through whole and a failure is not retried', async () => {
|
||||
let attempts = 0;
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
toolFunctions: {
|
||||
dump: () => { attempts += 1; return 'y'.repeat(500); },
|
||||
},
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('dump', {}), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('go');
|
||||
expect(attempts).toBe(1);
|
||||
expect(toolResults()[0].content).toHaveLength(500);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: messageSteering', () => {
|
||||
it('folds queued guidance into the next prompt, priority-aware', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, messageSteering: true });
|
||||
expect(agent.steer('check staging first')).not.toBe('');
|
||||
agent.steer('stop and re-read the ticket', SteeringPriority.INTERRUPT);
|
||||
|
||||
await agent.chat('deploy it');
|
||||
const prompt = callsOf('generateText')[0].args[0];
|
||||
// The interrupt is delivered first, and as an interrupt.
|
||||
expect(prompt.indexOf('[INTERRUPT USER GUIDANCE]')).toBeLessThan(prompt.indexOf('[USER GUIDANCE]'));
|
||||
expect(prompt).toContain('stop and re-read the ticket');
|
||||
expect(prompt).toContain('check staging first');
|
||||
expect(prompt).toContain('Please stop current work and follow this guidance immediately.');
|
||||
|
||||
// Drained: the next turn is not steered again by the same messages.
|
||||
mockLlm.calls = [];
|
||||
await agent.chat('carry on');
|
||||
expect(JSON.stringify(mockLlm.calls)).not.toContain('USER GUIDANCE');
|
||||
});
|
||||
|
||||
it('steers a turn that runs on a managed backend too', async () => {
|
||||
const seen: string[] = [];
|
||||
const backend = { execute: async (prompt: string) => { seen.push(prompt); return 'ok'; } };
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, messageSteering: true, backend });
|
||||
agent.steer('use the staging cluster');
|
||||
await agent.chat('deploy it');
|
||||
expect(seen[0]).toContain('use the staging cluster');
|
||||
});
|
||||
|
||||
it('control: without messageSteering the prompt is untouched and steer() is a no-op', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
expect(agent.steer('please stop')).toBe('');
|
||||
expect(agent.getMessageSteering()).toBeUndefined();
|
||||
await agent.chat('deploy it');
|
||||
expect(callsOf('generateText')[0].args[0]).toBe('deploy it');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: sandbox', () => {
|
||||
it('runs code in the configured sandbox and attaches the security pre-check', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, sandbox: true });
|
||||
expect(agent.hasSandbox).toBe(true);
|
||||
expect(agent.getSandboxConfig()!.sandboxType).toBe('subprocess');
|
||||
|
||||
const result = await agent.executeCode('console.log("hello from the box")', { language: 'javascript' });
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.stdout.trim()).toBe('hello from the box');
|
||||
});
|
||||
|
||||
it('control: without sandbox, executeCode refuses rather than running code in this process', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
expect(agent.hasSandbox).toBe(false);
|
||||
await expect(agent.executeCode('console.log(1)', { language: 'javascript' }))
|
||||
.rejects.toThrow(/no sandbox configured/);
|
||||
});
|
||||
|
||||
it('carries the security warnings on the result', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, sandbox: true });
|
||||
const result = await agent.executeCode('eval("1+1")\n', { language: 'javascript' });
|
||||
const warnings = (result.metadata as any).securityWarnings;
|
||||
expect(warnings.map((w: any) => w.severity)).toContain('critical');
|
||||
expect(warnings[0].lineNumber).toBe(1);
|
||||
});
|
||||
|
||||
it('the pre-check is language-aware', () => {
|
||||
expect(checkCodeSafety('rm -rf /', 'bash').map((w) => w.severity)).toContain('critical');
|
||||
expect(checkCodeSafety('rm -rf /', 'python')).toEqual([]);
|
||||
});
|
||||
|
||||
it('rejects an unregistered sandbox type instead of running locally', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, sandbox: { sandboxType: 'docker' } });
|
||||
await expect(agent.executeCode('print(1)')).rejects.toThrow(/No sandbox runner is registered for "docker"/);
|
||||
});
|
||||
|
||||
it('runs the child in the configured workingDir and persists files between calls', async () => {
|
||||
const os = require('os');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'praison-sandbox-test-'));
|
||||
try {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
sandbox: { workingDir: workspace, persistFiles: true },
|
||||
});
|
||||
// Relative paths resolve against the workspace (cwd), not the host dir.
|
||||
const write = await agent.executeCode(
|
||||
'const fs = require("fs"); fs.writeFileSync("artifact.txt", "kept");',
|
||||
{ language: 'javascript' }
|
||||
);
|
||||
expect(write.success).toBe(true);
|
||||
expect(fs.existsSync(path.join(workspace, 'artifact.txt'))).toBe(true);
|
||||
|
||||
// A later call sees the earlier call's artifact -- persistFiles honoured.
|
||||
const read = await agent.executeCode(
|
||||
'const fs = require("fs"); process.stdout.write(fs.readFileSync("artifact.txt", "utf8"));',
|
||||
{ language: 'javascript' }
|
||||
);
|
||||
expect(read.stdout.trim()).toBe('kept');
|
||||
} finally {
|
||||
fs.rmSync(workspace, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: selfImprove', () => {
|
||||
it('runs a guarded review turn restricted to skill_manage and reports the proposal', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet, selfImprove: true,
|
||||
toolFunctions: { lookup: () => 'found' },
|
||||
});
|
||||
mockLlm.chatQueue = [
|
||||
toolCallTurn('lookup', {}),
|
||||
{ content: 'done', role: 'assistant' },
|
||||
toolCallTurn('skill_manage', {
|
||||
action: 'create', name: 'lookup-flow', description: 'how to look things up', instructions: '1. call lookup',
|
||||
}, 'review_1'),
|
||||
];
|
||||
|
||||
await agent.chat('find it');
|
||||
expect(agent.lastSkillProposals).toEqual([{
|
||||
action: 'create', name: 'lookup-flow', description: 'how to look things up', instructions: '1. call lookup',
|
||||
}]);
|
||||
|
||||
// The review turn saw skill_manage and nothing else.
|
||||
const reviewCall = callsOf('generateChat')[callsOf('generateChat').length - 1];
|
||||
expect(reviewCall.args[2].map((t: any) => t.function.name)).toEqual(['skill_manage']);
|
||||
});
|
||||
|
||||
it('control: no review turn runs when selfImprove is off', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
toolFunctions: { lookup: () => 'found' },
|
||||
});
|
||||
mockLlm.chatQueue = [toolCallTurn('lookup', {}), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('find it');
|
||||
expect(agent.lastSkillProposals).toEqual([]);
|
||||
expect(callsOf('generateChat')).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('skips the review when the turn used no tools (the default policy)', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, selfImprove: true });
|
||||
await agent.chat('just chat');
|
||||
expect(callsOf('generateChat')).toHaveLength(0);
|
||||
expect(agent.lastSkillProposals).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: auth', () => {
|
||||
afterEach(() => {
|
||||
resetAuthProviders();
|
||||
});
|
||||
|
||||
it('pins the vendor when no model was named', () => {
|
||||
registerAuthProvider('claude-code', () => ({ apiKey: 'oauth-token' }));
|
||||
// A subscription seat belongs to one vendor, so the plain OpenAI default
|
||||
// would ship the OAuth token to the wrong endpoint.
|
||||
expect(new Agent({ instructions: 'x', ...quiet, auth: 'claude-code' }).getModel())
|
||||
.toBe('anthropic/claude-sonnet-4-5');
|
||||
// An explicit model still wins.
|
||||
expect(new Agent({ instructions: 'x', ...quiet, auth: 'claude-code', llm: 'gpt-4o-mini' }).getModel())
|
||||
.toBe('gpt-4o-mini');
|
||||
});
|
||||
|
||||
it('applies the resolved credentials to the transport before the first request', async () => {
|
||||
registerAuthProvider('qwen-cli', () => ({
|
||||
apiKey: 'oauth-token',
|
||||
baseURL: 'https://proxy.local/v1',
|
||||
headers: { 'user-agent': 'qwen-cli/1.0' },
|
||||
}));
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, auth: 'qwen-cli', llm: 'gpt-4o-mini' });
|
||||
await agent.chat('hi');
|
||||
const construction = callsOf('construct')[callsOf('construct').length - 1];
|
||||
expect(construction.args[1].apiKey).toBe('oauth-token');
|
||||
expect(construction.args[1].baseURL).toBe('https://proxy.local/v1');
|
||||
// The provider's headers ride on every request via the wrapped fetch.
|
||||
expect(typeof construction.args[1].fetch).toBe('function');
|
||||
});
|
||||
|
||||
it('control: without auth the transport keeps the caller\'s own credentials', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, llm: 'gpt-4o-mini', apiKey: 'sk-mine' });
|
||||
await agent.chat('hi');
|
||||
const construction = callsOf('construct')[callsOf('construct').length - 1];
|
||||
expect(construction.args[1].apiKey).toBe('sk-mine');
|
||||
});
|
||||
|
||||
it('fails at construction for an unregistered provider rather than billing the API key', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, auth: 'claude-code' }))
|
||||
.toThrow(/Unknown subscription provider 'claude-code'/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,360 @@
|
||||
/**
|
||||
* Behaviour parity for the `Agent.__init__` options that shape the prompt and
|
||||
* the tool list: `reflection`, `templates`, `toolSearch`, `toolsets`, `learn`.
|
||||
*
|
||||
* Each option is proved to change what the code does, next to a control that
|
||||
* shows the behaviour is absent without it.
|
||||
*/
|
||||
process.env.PRAISONAI_PARITY_SILENT = '1';
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { Agent } from '../../../../src/agent/simple';
|
||||
|
||||
const mockLlm = {
|
||||
calls: [] as Array<{ method: string; args: any[] }>,
|
||||
chatQueue: [] as any[],
|
||||
textQueue: [] as any[],
|
||||
};
|
||||
|
||||
jest.mock('../../../../src/llm/openai', () => ({
|
||||
OpenAIService: jest.fn().mockImplementation(() => {
|
||||
const next = (queue: any[], fallback: any) => {
|
||||
const item = queue.length > 0 ? queue.shift() : fallback;
|
||||
if (item instanceof Error) throw item;
|
||||
return item;
|
||||
};
|
||||
return {
|
||||
generateText: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'generateText', args });
|
||||
return next(mockLlm.textQueue, 'text-response');
|
||||
}),
|
||||
generateChat: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'generateChat', args });
|
||||
return next(mockLlm.chatQueue, { content: 'chat-response', role: 'assistant' });
|
||||
}),
|
||||
streamChat: jest.fn(async (messages: any, temperature: number, onToken: (t: string) => void) => {
|
||||
mockLlm.calls.push({ method: 'streamChat', args: [messages, temperature] });
|
||||
onToken('streamed');
|
||||
return 'streamed';
|
||||
}),
|
||||
streamChatWithTools: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'streamChatWithTools', args });
|
||||
return next(mockLlm.chatQueue, { content: 'stream-tools-response', role: 'assistant' });
|
||||
}),
|
||||
};
|
||||
}),
|
||||
}));
|
||||
|
||||
const quiet = { verbose: false, stream: false } as const;
|
||||
const lastCall = () => mockLlm.calls[mockLlm.calls.length - 1];
|
||||
const callsOf = (method: string) => mockLlm.calls.filter((c) => c.method === method);
|
||||
/** The system prompt a recorded call carried. */
|
||||
const systemPromptOf = (call: { method: string; args: any[] }): string =>
|
||||
call.method === 'generateText' ? call.args[1] : call.args[0][0]?.content ?? '';
|
||||
/**
|
||||
* The system prompt of the most recent TURN request. Feature passes
|
||||
* (reflection critiques, learning extraction) send system-less message lists,
|
||||
* so the last call is not necessarily the turn.
|
||||
*/
|
||||
const lastSystemPrompt = (): string => {
|
||||
for (let i = mockLlm.calls.length - 1; i >= 0; i--) {
|
||||
const call = mockLlm.calls[i];
|
||||
if (call.method === 'generateText') return call.args[1];
|
||||
const messages = call.args[0];
|
||||
if (Array.isArray(messages) && messages[0]?.role === 'system') return messages[0].content;
|
||||
}
|
||||
return '';
|
||||
};
|
||||
const toolCallTurn = (name: string, args: Record<string, unknown>) => ({
|
||||
content: '',
|
||||
role: 'assistant',
|
||||
tool_calls: [{ id: 'call_1', type: 'function', function: { name, arguments: JSON.stringify(args) } }],
|
||||
});
|
||||
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(() => {
|
||||
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'praison-features-'));
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
mockLlm.calls = [];
|
||||
mockLlm.chatQueue = [];
|
||||
mockLlm.textQueue = [];
|
||||
});
|
||||
|
||||
describe('Agent: reflection', () => {
|
||||
it('critiques its own answer and regenerates an unsatisfactory one', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, reflection: true });
|
||||
mockLlm.textQueue = ['first-answer'];
|
||||
mockLlm.chatQueue = [
|
||||
{ content: '{"reflection":"too vague","satisfactory":"no"}', role: 'assistant' },
|
||||
{ content: 'better-answer', role: 'assistant' },
|
||||
{ content: '{"reflection":"now specific","satisfactory":"yes"}', role: 'assistant' },
|
||||
];
|
||||
|
||||
await expect(agent.chat('hi')).resolves.toBe('better-answer');
|
||||
expect(agent.lastReflections.map((r) => r.satisfactory)).toEqual(['no', 'yes']);
|
||||
expect(agent.lastReflections[0].reflection).toBe('too vague');
|
||||
// The critique request quotes the answer it is about.
|
||||
expect(callsOf('generateChat')).toHaveLength(3);
|
||||
});
|
||||
|
||||
it('control: without reflection the first answer is returned and no critique is asked for', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
mockLlm.textQueue = ['first-answer'];
|
||||
await expect(agent.chat('hi')).resolves.toBe('first-answer');
|
||||
expect(agent.lastReflections).toEqual([]);
|
||||
expect(callsOf('generateChat')).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('a preset sets the round bounds: "minimal" accepts after one round', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, reflection: 'minimal' });
|
||||
mockLlm.textQueue = ['answer'];
|
||||
mockLlm.chatQueue = [{ content: '{"reflection":"could be better","satisfactory":"no"}', role: 'assistant' }];
|
||||
await expect(agent.chat('hi')).resolves.toBe('answer');
|
||||
// maxIterations 1: one critique, then stop -- no regeneration round.
|
||||
expect(callsOf('generateChat')).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('rejects an unknown preset at construction rather than silently not reflecting', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, reflection: 'exhaustive' }))
|
||||
.toThrow(/Invalid reflection preset/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: templates', () => {
|
||||
it('useSystemPrompt: false sends no system prompt at all', async () => {
|
||||
const agent = new Agent({ instructions: 'be terse', ...quiet, templates: { useSystemPrompt: false } });
|
||||
await agent.chat('hi');
|
||||
expect(systemPromptOf(lastCall())).toBe('');
|
||||
});
|
||||
|
||||
it('control: the instructions are the system prompt without the template', async () => {
|
||||
const agent = new Agent({ instructions: 'be terse', ...quiet });
|
||||
await agent.chat('hi');
|
||||
expect(systemPromptOf(lastCall())).toContain('be terse');
|
||||
});
|
||||
|
||||
it('the system template replaces the instructions, substituting role/goal', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'be terse', role: 'chef', goal: 'cook', ...quiet,
|
||||
templates: { system: '[{role}|{goal}] {instructions}' },
|
||||
});
|
||||
await agent.chat('hi');
|
||||
expect(systemPromptOf(lastCall())).toContain('[chef|cook] be terse');
|
||||
});
|
||||
|
||||
it('the prompt template wraps the user prompt', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, templates: { prompt: 'Question: {input}' } });
|
||||
await agent.chat('hi');
|
||||
expect(lastCall().args[0]).toBe('Question: hi');
|
||||
});
|
||||
|
||||
it('a response template without {response} becomes a formatting instruction on the prompt', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, templates: { response: '- bullet\\n- bullet' } });
|
||||
await agent.chat('hi');
|
||||
expect(lastCall().args[0]).toContain('IMPORTANT: Format your response according to this template:');
|
||||
expect(lastCall().args[0]).toContain('- bullet');
|
||||
});
|
||||
|
||||
it('a response template with {response} wraps the answer instead', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, templates: { response: '<answer>{response}</answer>' } });
|
||||
mockLlm.textQueue = ['42'];
|
||||
await expect(agent.chat('hi')).resolves.toBe('<answer>42</answer>');
|
||||
// The wrapper is applied to the answer, not sent as an instruction.
|
||||
expect(lastCall().args[0]).toBe('hi');
|
||||
});
|
||||
|
||||
it('rejects an unknown templates field rather than ignoring it', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, templates: { greeting: 'hi' } }))
|
||||
.toThrow(/Unknown templates field/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: toolSearch', () => {
|
||||
const mcpTool = (name: string, description: string) => ({
|
||||
type: 'function',
|
||||
function: { name, description, parameters: { type: 'object', properties: {} } },
|
||||
});
|
||||
|
||||
const agentWithMcpTools = (toolSearch: any) => new Agent({
|
||||
instructions: 'x',
|
||||
...quiet,
|
||||
toolSearch,
|
||||
tools: [mcpTool('mcp_weather', 'Look up the weather forecast for a city'), mcpTool('mcp_stocks', 'Fetch a stock quote')],
|
||||
toolFunctions: {
|
||||
mcp_weather: () => 'sunny',
|
||||
mcp_stocks: () => '42',
|
||||
},
|
||||
});
|
||||
|
||||
it('replaces deferrable tools with the three bridge tools', async () => {
|
||||
const agent = agentWithMcpTools('on');
|
||||
await agent.chat('what is the weather?');
|
||||
const sentTools = lastCall().args[2].map((t: any) => t.function.name);
|
||||
expect(sentTools).toEqual(['tool_search', 'tool_describe', 'tool_call']);
|
||||
expect(sentTools).not.toContain('mcp_weather');
|
||||
});
|
||||
|
||||
it('control: without toolSearch every tool schema is sent', async () => {
|
||||
const agent = agentWithMcpTools(false);
|
||||
await agent.chat('what is the weather?');
|
||||
const sentTools = lastCall().args[2].map((t: any) => t.function.name);
|
||||
expect(sentTools).toEqual(['mcp_weather', 'mcp_stocks']);
|
||||
});
|
||||
|
||||
it('answers tool_search from the deferred catalog without calling a user tool', async () => {
|
||||
const agent = agentWithMcpTools('on');
|
||||
let weatherRan = false;
|
||||
(agent as any).toolFunctions.mcp_weather = () => { weatherRan = true; return 'sunny'; };
|
||||
mockLlm.chatQueue = [
|
||||
toolCallTurn('tool_search', { query: 'weather forecast' }),
|
||||
{ content: 'done', role: 'assistant' },
|
||||
];
|
||||
|
||||
await agent.chat('what is the weather?');
|
||||
const toolResult = mockLlm.calls
|
||||
.flatMap((c) => (c.method === 'generateChat' ? c.args[0] : []))
|
||||
.find((m: any) => m.role === 'tool');
|
||||
const payload = JSON.parse(toolResult.content);
|
||||
expect(payload.query).toBe('weather forecast');
|
||||
expect(payload.results.map((r: any) => r.name)).toContain('mcp_weather');
|
||||
expect(payload.total_available).toBe(2);
|
||||
expect(weatherRan).toBe(false);
|
||||
});
|
||||
|
||||
it('unwraps tool_call so the real tool runs and the events name it', async () => {
|
||||
const agent = agentWithMcpTools('on');
|
||||
const seen: string[] = [];
|
||||
(agent as any).toolFunctions.mcp_weather = (args: any) => { seen.push(String(args.city)); return 'sunny'; };
|
||||
mockLlm.chatQueue = [
|
||||
toolCallTurn('tool_call', { tool_name: 'mcp_weather', tool_args: { city: 'Oslo' } }),
|
||||
{ content: 'it is sunny', role: 'assistant' },
|
||||
];
|
||||
|
||||
const events: any[] = [];
|
||||
await agent.start('weather?', undefined, () => {}, undefined, (e) => events.push(e));
|
||||
expect(seen).toEqual(['Oslo']);
|
||||
// Invariant 6: observers see the real tool, not the bridge.
|
||||
expect(events.find((e) => e.type === 'tool_call').name).toBe('mcp_weather');
|
||||
expect(events.find((e) => e.type === 'tool_result').name).toBe('mcp_weather');
|
||||
});
|
||||
|
||||
it('auto mode keeps small tool lists visible (they are under the threshold)', async () => {
|
||||
const agent = agentWithMcpTools('auto');
|
||||
await agent.chat('hi');
|
||||
const sentTools = lastCall().args[2].map((t: any) => t.function.name);
|
||||
expect(sentTools).toEqual(['mcp_weather', 'mcp_stocks']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: toolsets', () => {
|
||||
it('attaches the tools a named toolset resolves to', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, toolsets: ['web'] });
|
||||
await agent.chat('hi');
|
||||
// The 'web' toolset resolves to tavily_search and exa_search; those are
|
||||
// the two Python names with a TypeScript builtin behind them.
|
||||
const names = (agent as any).tools.map((t: any) => t.function.name);
|
||||
expect(names).toContain('tavilySearch');
|
||||
expect(names).toContain('webSearch');
|
||||
expect(typeof (agent as any).toolFunctions.tavilySearch).toBe('function');
|
||||
});
|
||||
|
||||
it('control: no toolsets means no tools are attached', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('hi');
|
||||
expect((agent as any).tools).toBeUndefined();
|
||||
});
|
||||
|
||||
it('rejects an unknown toolset name at construction, not on the first turn', () => {
|
||||
expect(() => new Agent({ instructions: 'x', ...quiet, toolsets: ['not-a-toolset'] }))
|
||||
.toThrow(/not-a-toolset/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent: learn', () => {
|
||||
const learnConfig = (extra: Record<string, unknown> = {}) => ({
|
||||
mode: 'agentic',
|
||||
storePath: fs.mkdtempSync(path.join(tmpDir, 'learn-')),
|
||||
persona: false,
|
||||
insights: true,
|
||||
thread: false,
|
||||
...extra,
|
||||
});
|
||||
|
||||
it('extracts learnings after a turn and injects them into the next system prompt', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, learn: learnConfig() });
|
||||
mockLlm.textQueue = ['answer'];
|
||||
mockLlm.chatQueue = [{ content: '{"insights":["the user works in metric units"]}', role: 'assistant' }];
|
||||
|
||||
await agent.chat('hi');
|
||||
expect(agent.getLearnManager()!.toSystemPromptContext()).toContain('the user works in metric units');
|
||||
|
||||
// The next turn's own request (not the extraction call that follows it)
|
||||
// carries what was learned.
|
||||
await agent.chat('again');
|
||||
const systemPrompt = lastSystemPrompt();
|
||||
expect(systemPrompt).toContain('Learned Insights');
|
||||
expect(systemPrompt).toContain('the user works in metric units');
|
||||
});
|
||||
|
||||
it('control: without learn nothing is extracted and no store exists', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('hi');
|
||||
expect(agent.getLearnManager()).toBeUndefined();
|
||||
// The only model call was the turn itself: no extraction pass ran.
|
||||
expect(mockLlm.calls).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('propose mode queues findings for approval instead of storing them', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, learn: learnConfig({ mode: 'propose' }) });
|
||||
mockLlm.textQueue = ['answer'];
|
||||
mockLlm.chatQueue = [{ content: '{"insights":["prefers short replies"]}', role: 'assistant' }];
|
||||
|
||||
await agent.chat('hi');
|
||||
const manager = agent.getLearnManager()!;
|
||||
expect(manager.toSystemPromptContext()).not.toContain('prefers short replies');
|
||||
expect(manager.pendingLearnings.map((e) => e.content)).toContain('prefers short replies');
|
||||
});
|
||||
|
||||
it('an unknown mode disables learning with a warning rather than throwing', () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, learn: 'occasionally' });
|
||||
expect(agent.getLearnManager()).toBeUndefined();
|
||||
});
|
||||
|
||||
it('nudges on the next turn once a turn did enough tool work', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
learn: learnConfig({ mode: 'propose', nudgeInterval: 1, nudgeMinToolIters: 1 }),
|
||||
toolFunctions: { lookup: () => 'found' },
|
||||
});
|
||||
// Turn 1 does one tool call, then answers. The nudge cannot appear on this
|
||||
// turn's own prompt (no work done yet at prompt-build time).
|
||||
mockLlm.chatQueue = [toolCallTurn('lookup', {}), { content: 'done', role: 'assistant' }];
|
||||
await agent.chat('find it');
|
||||
expect(lastSystemPrompt()).not.toContain('[System nudge]');
|
||||
|
||||
// Turn 2's prompt reflects the previous turn's tool work and carries the
|
||||
// nudge -- the regression was that the reset counter meant it never did.
|
||||
mockLlm.chatQueue = [{ content: 'ok', role: 'assistant' }];
|
||||
await agent.chat('again');
|
||||
expect(lastSystemPrompt()).toContain('[System nudge]');
|
||||
});
|
||||
|
||||
it('control: a turn with no tool work never nudges', async () => {
|
||||
const agent = new Agent({
|
||||
instructions: 'x', ...quiet,
|
||||
learn: learnConfig({ mode: 'propose', nudgeInterval: 1, nudgeMinToolIters: 1 }),
|
||||
});
|
||||
await agent.chat('hi');
|
||||
await agent.chat('again');
|
||||
expect(lastSystemPrompt()).not.toContain('[System nudge]');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,216 @@
|
||||
/**
|
||||
* Behaviour parity for the per-call options of `Agent.chat`.
|
||||
*
|
||||
* Every test here is paired: the option changes what the code does, and the
|
||||
* control shows the behaviour is absent when the option is not passed. That
|
||||
* pairing is the whole point — an option that is accepted and ignored passes
|
||||
* a "does the parameter exist?" test and fails the user.
|
||||
*/
|
||||
process.env.PRAISONAI_PARITY_SILENT = '1';
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { Agent } from '../../../../src/agent/simple';
|
||||
import { HooksManager } from '../../../../src/hooks/manager';
|
||||
import { buildMultimodalPrompt } from '../../../../src/agent/features/chat-options';
|
||||
|
||||
const mockLlm = {
|
||||
calls: [] as Array<{ method: string; args: any[] }>,
|
||||
chatQueue: [] as any[],
|
||||
textQueue: [] as any[],
|
||||
};
|
||||
|
||||
jest.mock('../../../../src/llm/openai', () => ({
|
||||
OpenAIService: jest.fn().mockImplementation(() => {
|
||||
const next = (queue: any[], fallback: any) => {
|
||||
const item = queue.length > 0 ? queue.shift() : fallback;
|
||||
if (item instanceof Error) throw item;
|
||||
return item;
|
||||
};
|
||||
return {
|
||||
generateText: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'generateText', args });
|
||||
return next(mockLlm.textQueue, 'text-response');
|
||||
}),
|
||||
generateChat: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'generateChat', args });
|
||||
return next(mockLlm.chatQueue, { content: 'chat-response', role: 'assistant' });
|
||||
}),
|
||||
streamChat: jest.fn(async (messages: any, temperature: number, onToken: (t: string) => void) => {
|
||||
mockLlm.calls.push({ method: 'streamChat', args: [messages, temperature] });
|
||||
onToken('streamed');
|
||||
return 'streamed';
|
||||
}),
|
||||
streamChatWithTools: jest.fn(async (...args: any[]) => {
|
||||
mockLlm.calls.push({ method: 'streamChatWithTools', args });
|
||||
return next(mockLlm.chatQueue, { content: 'stream-tools-response', role: 'assistant' });
|
||||
}),
|
||||
};
|
||||
}),
|
||||
}));
|
||||
|
||||
const quiet = { verbose: false, stream: false } as const;
|
||||
const lastCall = () => mockLlm.calls[mockLlm.calls.length - 1];
|
||||
/** The user message of the last generateChat call. */
|
||||
const lastUserMessage = () => {
|
||||
const messages = lastCall().args[0];
|
||||
return messages[messages.length - 1];
|
||||
};
|
||||
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(() => {
|
||||
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'praison-chat-options-'));
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
mockLlm.calls = [];
|
||||
mockLlm.chatQueue = [];
|
||||
mockLlm.textQueue = [];
|
||||
});
|
||||
|
||||
describe('Agent.chat: attachments', () => {
|
||||
it('sends a local image as an inline data URI, and the plain text without it', async () => {
|
||||
// A 1x1 PNG is enough: the point is that the bytes are read and encoded.
|
||||
const png = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==',
|
||||
'base64',
|
||||
);
|
||||
const file = path.join(tmpDir, 'pixel.png');
|
||||
fs.writeFileSync(file, png);
|
||||
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('what is this?', undefined, undefined, { attachments: [file] });
|
||||
|
||||
const message = lastUserMessage();
|
||||
expect(Array.isArray(message.content)).toBe(true);
|
||||
expect(message.content[0]).toEqual({ type: 'text', text: 'what is this?' });
|
||||
expect(message.content[1].type).toBe('image_url');
|
||||
expect(message.content[1].image_url.url).toBe(`data:image/png;base64,${png.toString('base64')}`);
|
||||
|
||||
// Ephemeral: the attachment never reaches history, only the text does.
|
||||
expect(agent.getHistory().map((m) => m.content)).toContain('what is this?');
|
||||
});
|
||||
|
||||
it('control: without attachments the prompt stays a plain string', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('what is this?');
|
||||
expect(lastCall().method).toBe('generateText');
|
||||
expect(lastCall().args[0]).toBe('what is this?');
|
||||
});
|
||||
|
||||
it('passes an http(s) or data: attachment through as a URL', async () => {
|
||||
const content = await buildMultimodalPrompt('look', ['https://example.com/a.png', 'data:image/png;base64,AAAA']);
|
||||
expect(content).toEqual([
|
||||
{ type: 'text', text: 'look' },
|
||||
{ type: 'image_url', image_url: { url: 'https://example.com/a.png' } },
|
||||
{ type: 'image_url', image_url: { url: 'data:image/png;base64,AAAA' } },
|
||||
]);
|
||||
});
|
||||
|
||||
it('warns and skips an unreadable or non-image attachment rather than failing the turn', async () => {
|
||||
const warnings: string[] = [];
|
||||
const content = await buildMultimodalPrompt('look', ['missing.png', 'notes.txt'], {
|
||||
onWarning: (m) => warnings.push(m),
|
||||
});
|
||||
expect(content).toEqual([{ type: 'text', text: 'look' }]);
|
||||
expect(warnings).toHaveLength(2);
|
||||
expect(warnings[0]).toContain('missing.png');
|
||||
expect(warnings[1]).toContain('notes.txt');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent.chat: reasoningSteps', () => {
|
||||
it('forces a single non-streaming completion even when streaming is on', async () => {
|
||||
const agent = new Agent({ instructions: 'x', verbose: false, stream: true });
|
||||
await agent.chat('hi', undefined, undefined, { reasoningSteps: true });
|
||||
expect(lastCall().method).toBe('generateText');
|
||||
});
|
||||
|
||||
it('control: the same agent streams when reasoningSteps is not set', async () => {
|
||||
const agent = new Agent({ instructions: 'x', verbose: false, stream: true });
|
||||
await agent.chat('hi');
|
||||
expect(lastCall().method).toBe('streamChat');
|
||||
});
|
||||
|
||||
it('beats an explicit per-call stream: true', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('hi', undefined, undefined, { stream: true, reasoningSteps: true });
|
||||
expect(lastCall().method).toBe('generateText');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent.chat: taskName / taskDescription / taskId', () => {
|
||||
it('reaches the lifecycle hooks and is readable afterwards', async () => {
|
||||
const seen: any[] = [];
|
||||
const hooks = new HooksManager();
|
||||
hooks.register('agent_start', (ctx: any) => { seen.push(['start', ctx]); return ctx; });
|
||||
hooks.register('agent_complete', (ctx: any) => { seen.push(['complete', ctx]); return ctx; });
|
||||
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, hooks });
|
||||
await agent.chat('hi', undefined, undefined, {
|
||||
taskName: 'summarise', taskDescription: 'condense the notes', taskId: 'T-7',
|
||||
});
|
||||
|
||||
expect(seen.map((s) => s[0])).toEqual(['start', 'complete']);
|
||||
for (const [, ctx] of seen) {
|
||||
expect(ctx.taskName).toBe('summarise');
|
||||
expect(ctx.taskDescription).toBe('condense the notes');
|
||||
expect(ctx.taskId).toBe('T-7');
|
||||
}
|
||||
expect(agent.getTaskContext()).toEqual({
|
||||
name: 'summarise', description: 'condense the notes', id: 'T-7',
|
||||
});
|
||||
});
|
||||
|
||||
it('control: no task metadata is invented when the caller names none', async () => {
|
||||
const seen: any[] = [];
|
||||
const hooks = new HooksManager();
|
||||
hooks.register('agent_start', (ctx: any) => { seen.push(ctx); return ctx; });
|
||||
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, hooks });
|
||||
await agent.chat('hi');
|
||||
|
||||
expect(seen[0].taskName).toBeUndefined();
|
||||
expect(seen[0].taskId).toBeUndefined();
|
||||
expect(agent.getTaskContext()).toBeUndefined();
|
||||
});
|
||||
|
||||
it('carries only the fields that were given', async () => {
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('hi', undefined, undefined, { taskId: 'T-1' });
|
||||
expect(agent.getTaskContext()).toEqual({ id: 'T-1' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('Agent.chat: config', () => {
|
||||
it('is forwarded verbatim to the managed backend that runs the turn', async () => {
|
||||
const seen: any[] = [];
|
||||
const backend = {
|
||||
execute: jest.fn(async (prompt: string, options: any) => {
|
||||
seen.push({ prompt, options });
|
||||
return 'from-backend';
|
||||
}),
|
||||
};
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, backend });
|
||||
|
||||
await expect(agent.chat('hi', undefined, undefined, { config: { thinkingBudget: 4096 } }))
|
||||
.resolves.toBe('from-backend');
|
||||
expect(seen[0].options.config).toEqual({ thinkingBudget: 4096 });
|
||||
// The local transport was never used: the runtime owns the turn.
|
||||
expect(mockLlm.calls).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('control: the same backend sees no config when the caller passes none', async () => {
|
||||
const seen: any[] = [];
|
||||
const backend = { execute: jest.fn(async (prompt: string, options: any) => { seen.push(options); return 'ok'; }) };
|
||||
const agent = new Agent({ instructions: 'x', ...quiet, backend });
|
||||
await agent.chat('hi');
|
||||
expect(seen[0].config).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -20,7 +20,7 @@ import { Guardrail } from '../../../src/guardrails';
|
||||
import { HooksManager } from '../../../src/hooks/manager';
|
||||
import { ContextManager } from '../../../src/context/manager';
|
||||
import { RulesManager } from '../../../src/memory/rules-manager';
|
||||
import { resetParityNotices, unhonouredOptions } from '../../../src/utils/parity-notice';
|
||||
import { resetParityNotices, unhonouredFor, unhonouredOptions } from '../../../src/utils/parity-notice';
|
||||
|
||||
// Recording double for the OpenAI-compatible service: every call is captured
|
||||
// with its positional arguments, and queued responses/errors drive tool loops
|
||||
@@ -111,46 +111,67 @@ beforeEach(() => {
|
||||
});
|
||||
|
||||
describe('Agent.__init__ parity: acceptance and notices', () => {
|
||||
it('accepts every accepted-with-notice option, stores it, and reports it', () => {
|
||||
it('stores every Python-parity option so `to_dict()`-style reads match', () => {
|
||||
// Placement options answer the same question in different words, so a
|
||||
// valid agent names at most one of them; each is covered on its own in
|
||||
// tests/unit/agent/features/agent-execution-features.test.ts.
|
||||
const backend = { execute: async () => 'x' };
|
||||
const agent = new Agent({
|
||||
instructions: 'x',
|
||||
...quiet,
|
||||
auth: 'claude-code',
|
||||
toolsets: ['web'],
|
||||
reflection: true,
|
||||
autonomy: 'full',
|
||||
templates: { greeting: 'hi' },
|
||||
autonomy: 'full_auto',
|
||||
templates: { system: 'S: {instructions}' },
|
||||
selfImprove: true,
|
||||
toolConfig: { timeout: 5 },
|
||||
learn: true,
|
||||
backend: { kind: 'managed' },
|
||||
runOn: 'anthropic',
|
||||
toolsRunOn: 'docker',
|
||||
runtime: { model: 'x' },
|
||||
backend,
|
||||
toolSearch: true,
|
||||
messageSteering: true,
|
||||
sandbox: true,
|
||||
});
|
||||
expect(agent.auth).toBe('claude-code');
|
||||
expect(agent.toolsets).toEqual(['web']);
|
||||
expect(agent.reflection).toBe(true);
|
||||
expect(agent.autonomy).toBe('full');
|
||||
expect(agent.templates).toEqual({ greeting: 'hi' });
|
||||
expect(agent.autonomy).toBe('full_auto');
|
||||
expect(agent.templates).toEqual({ system: 'S: {instructions}' });
|
||||
expect(agent.selfImprove).toBe(true);
|
||||
expect(agent.toolConfig).toEqual({ timeout: 5 });
|
||||
expect(agent.learn).toBe(true);
|
||||
expect(agent.backend).toEqual({ kind: 'managed' });
|
||||
expect(agent.runOn).toBe('anthropic');
|
||||
expect(agent.toolsRunOn).toBe('docker');
|
||||
expect(agent.runtime).toEqual({ model: 'x' });
|
||||
expect(agent.backend).toBe(backend);
|
||||
expect(agent.toolSearch).toBe(true);
|
||||
expect(agent.messageSteering).toBe(true);
|
||||
expect(agent.sandbox).toBe(true);
|
||||
expect(unhonouredOptions()).toEqual([
|
||||
'Agent.auth', 'Agent.autonomy', 'Agent.backend', 'Agent.learn', 'Agent.messageSteering',
|
||||
'Agent.reflection', 'Agent.runOn', 'Agent.runtime', 'Agent.sandbox', 'Agent.selfImprove',
|
||||
'Agent.toolConfig', 'Agent.toolSearch', 'Agent.toolsRunOn', 'Agent.toolsets', 'Agent.templates',
|
||||
].sort());
|
||||
expect(new Agent({ instructions: 'x', ...quiet, toolsRunOn: 'local' }).toolsRunOn).toBe('local');
|
||||
});
|
||||
|
||||
it('never reports a notice for an option outside the ledger', () => {
|
||||
// The ledger in utils/parity-notice.ts is the single list of options that
|
||||
// are accepted and not yet acted on. A notice for anything else would mean
|
||||
// a surface warning about an option nothing is tracking.
|
||||
new Agent({
|
||||
instructions: 'x',
|
||||
...quiet,
|
||||
toolsets: ['web'],
|
||||
reflection: true,
|
||||
autonomy: 'full_auto',
|
||||
templates: { system: 'S: {instructions}' },
|
||||
selfImprove: true,
|
||||
toolConfig: { timeout: 5 },
|
||||
learn: true,
|
||||
toolSearch: true,
|
||||
messageSteering: true,
|
||||
sandbox: true,
|
||||
});
|
||||
const ledger = new Set([
|
||||
...unhonouredFor('Agent.__init__'),
|
||||
// Options that are honoured for some inputs and announce themselves for
|
||||
// the rest (the "Partial" table of BEHAVIOUR_PARITY.md).
|
||||
'context', 'guardrails', 'knowledge', 'memory', 'reasoningEffort', 'web', 'toolConfig',
|
||||
]);
|
||||
for (const reported of unhonouredOptions()) {
|
||||
expect(ledger).toContain(reported.replace(/^Agent\./, ''));
|
||||
}
|
||||
});
|
||||
|
||||
it('applies the Python defaults and raises no notice when nothing is supplied', () => {
|
||||
@@ -477,7 +498,10 @@ describe('Agent.chat parity: per-call options', () => {
|
||||
expect(lastCall().args[1]).toBe(0.3);
|
||||
});
|
||||
|
||||
it('accepted-with-notice call options are reported, never dropped silently', async () => {
|
||||
it('call options that are honoured do not report themselves as ignored', async () => {
|
||||
// These six were accepted-and-ignored until they were implemented. The
|
||||
// expectation is derived from the ledger rather than written out, so it
|
||||
// cannot drift the next time an option is closed.
|
||||
const agent = new Agent({ instructions: 'x', ...quiet });
|
||||
await agent.chat('hi', undefined, undefined, {
|
||||
reasoningSteps: true,
|
||||
@@ -488,10 +512,13 @@ describe('Agent.chat parity: per-call options', () => {
|
||||
attachments: ['a.png'],
|
||||
forceRetrieval: true,
|
||||
});
|
||||
expect(unhonouredOptions()).toEqual([
|
||||
'Agent.chat.attachments', 'Agent.chat.config', 'Agent.chat.reasoningSteps',
|
||||
'Agent.chat.taskDescription', 'Agent.chat.taskId', 'Agent.chat.taskName',
|
||||
]);
|
||||
const ledgered = new Set(unhonouredFor('Agent.chat'));
|
||||
for (const reported of unhonouredOptions()) {
|
||||
const option = reported.replace(/^Agent\.chat\./, '');
|
||||
expect(ledgered.has(option)).toBe(true);
|
||||
}
|
||||
// Control: the ledger no longer lists any of them, so nothing is reported.
|
||||
expect(unhonouredOptions().filter((k) => k.startsWith('Agent.chat.'))).toEqual([]);
|
||||
});
|
||||
|
||||
it('seed is reported on the AI SDK path', async () => {
|
||||
|
||||
Reference in new issue
Block a user