Compare commits

..
Author SHA1 Message Date
MervinPraisonandClaude Opus 5 323b37bdfc fix(review): adopt every documented backend protocol, not just get_response
Review found the Agent(llm=<object>) fix was incomplete, and it was right.
Verified by running it:

    chat_completion-only backend -> self.llm=<object>  custom=False   BUG
    get_response backend         -> self.llm="custom/..." custom=True  ok

llm/protocols.py documents two more backend surfaces besides get_response:
LLMProviderProtocol via chat/achat, and UnifiedLLMProtocol via
chat_completion/achat_completion. A backend implementing either still fell
through to the plain OpenAI branch, where the object became the model
identifier and the turn went to the wrong backend -- the same defect the
branch was added to fix, for a different protocol.

Detection now duck-types across all of them. Five tests, including two
controls: a plain object with no backend method is still NOT adopted, and a
string llm is unaffected.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-08 15:11:15 +01:00
MervinPraison 00c56a493e Merge branch 'fix/comp-python-gaps' of https://github.com/MervinPraison/PraisonAI into fix/comp-python-gaps 2026-09-07 13:22:38 +01:00
MervinPraisonandClaude Opus 5 4b3449f423 chore(parity): waive the two new tool() guardrail parameters, with an expiry
CI's signatures gate failed on tool().input_guardrails and
tool().output_guardrails: both are new Python parameters with no
TypeScript counterpart.

These waivers are marked NOT an equivalence. Every other reason in
waivers.yaml documents two SDKs reaching the same result by different
routes, validated by running both. These record a real gap instead --
nothing in praisonai-ts resolves the same behaviour. They exist only to
unblock this PR, because porting is a praisonai-ts change and the two
packages do not share a branch.

Both carry expires: 2026-12-07. An expired waiver fails the check, so if
the TypeScript port is forgotten the gate goes red rather than the gap
quietly becoming permanent.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 13:22:14 +01:00
praisonai-triage-agent[bot]andMervin Praison 94da4aa6c6 fix: block model requests by scope depth, not a shared snapshot
no_model_requests() saved and restored the single _allowed boolean, so two
overlapping scopes (or cross-thread scopes) could re-enable requests while
another scope was still open, leaking a real provider call. Track active
scopes with a counter instead: requests stay blocked until every open scope
exits. Adds regression tests for overlapping and globally-allowed cases.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2026-09-07 11:52:04 +00:00
MervinPraison 93c90ddaf7 Merge branch 'fix/comp-toolguard' into fix/comp-python-gaps
# Conflicts:
#	src/praisonai-agents/praisonaiagents/guardrails/__init__.py
2026-09-07 12:46:50 +01:00
MervinPraisonandClaude Opus 5 0fe844a726 feat: per-tool guardrails
Guardrails were agent-wide: _tool_call_guardrails / _tool_result_guardrails
are fed only from Agent(guardrails=...) and fire for every tool. Approvals
gate calls interactively. Nothing sat between them, so validating one
tool's arguments -- rejecting a send_email whose recipient is off-domain --
meant hand-wrapping the function.

Adds input and output guardrails scoped to a single tool, declared
alongside the approval and trust options a tool already carries. An input
guardrail sees the arguments before the tool runs and may block or rewrite
them; an output guardrail may block or substitute the result. A blocked
call feeds a message back to the model rather than raising at the user.

Built on the existing GuardrailResult and GuardrailChain rather than a
second guardrail system, so there is one vocabulary for a validation
outcome. Ordering relative to the approval gate and the trust fence is
chosen for safety and documented at the call site.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 12:46:23 +01:00
MervinPraisonandClaude Opus 5 b6b0f19cff chore(parity): regenerate tracker for the new Python exports
Merging the three fixes made the tracker stale in a way none of the
individual branches showed: each was regenerated before the others
existed. Records 6 praisonai-ts gaps (GuardrailRetry plus the five
model_harness test-double exports), up from 0.

Recorded rather than avoided. Withholding the exports to keep the count
at zero would game the measurement instead of reporting it; Python is the
source of truth and TypeScript follows in a separate PR, which is exactly
the state this tracker exists to make visible.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 12:42:20 +01:00
MervinPraison 6f9fc29072 Merge branch 'fix/comp-output-validator' into fix/comp-python-gaps 2026-09-07 12:40:36 +01:00
MervinPraison 501c3664d4 Merge branch 'fix/comp-testdouble' into fix/comp-python-gaps 2026-09-07 12:40:36 +01:00
MervinPraisonandClaude Opus 5 59369664c4 fix: feed guardrail rejections back into the live conversation
_apply_guardrail_with_retry already retried on validation failure, but it
threw the conversation away: on rejection it built one fresh user message,
prompt + "Note: Previous response failed validation due to: ...", and sent
that alone. The system prompt, retrieved context, tool results and the
rejected answer never reached the retry. Proven against pristine code --
the second request was a single-element message list.

So the missing capability was never retry; it was feedback into the live
conversation. The retry now appends the rejected assistant turn plus the
reason as a user turn and re-calls with the full history, bounded by the
max_guardrail_retries that already existed. No competing setting is added.

Validators signal rejection the way they already could, by returning
(False, reason). GuardrailRetry is added for validators whose check fails
several frames down -- a Pydantic parse, a lookup helper -- whose message
was previously mangled into "Agent guardrail validation error: ...".

The whole-task layer had the same disease: AgentTeam re-ran tasks with the
identical prompt, logging the guardrail reason and dropping it. It now
fills the Task.validation_feedback slot that already existed, so
Process._build_task_context renders it into the re-run. Reuse of existing
wiring, not new wiring.

Two fixes fell out: the retry passed stream=True regardless of agent
config (the sync path logged an ERROR and fell back on every retry), and
GuardrailChain buried the rejection under "Guardrail error: ...".

Regenerates the parity tracker, which records GuardrailRetry as a new
praisonai-ts gap (count 0 -> 1) pending a TypeScript follow-up.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 12:40:29 +01:00
MervinPraisonandClaude Opus 5 f6788b2db7 feat: scriptable test model and a no-network guard
Nothing shipped let a user unit-test an agent offline. The repo's own
pattern is hand-rolled per test -- LLM.__new__(LLM) followed by assigning
twelve private methods -- repeated across three files.

ScriptedModel subclasses the real LLM and replaces only the
provider-facing methods, so system-prompt assembly, tool-schema
serialisation, the tool-execution loop, tool-result messages, streaming
and async all run production code. Replies are real litellm
ModelResponse objects, parsed by the same path as a live response, so the
double cannot drift from reality. It records requests, scripts tool calls,
and raises ScriptExhausted rather than hanging or inventing a reply.

allow_model_requests(False) blocks every real provider call at the 6
litellm request sites and both OpenAI client properties, and names the
offending call site -- which needed a frame walk, since requests are
dispatched through an asyncio loop on a worker thread.

Fixes a bug found while building this: Agent(llm=<model object>) fell
through every branch to the plain OpenAI path, leaving self.llm set to
the object itself and _using_custom_llm False, so every turn went to
OpenAI under a model name that was really a repr(). Passing your own
model did the opposite of what it means.

Both new error types derive from BaseException on purpose: the tool
loop's broad `except Exception` would otherwise turn "your script is one
reply short" into a silent None return.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 12:40:29 +01:00
MervinPraisonandClaude Opus 5 ac9006edc0 fix: never drop an attachment silently
_build_multimodal_prompt matched five image extensions and had no else
branch, so agent.chat(attachments=["report.pdf"]) appended nothing -- no
part, no warning, no error. The model answered from the text prompt alone
and the caller could not tell the file was ignored. A mistyped path
failed just as quietly, as did pathlib.Path attachments, directories,
non-image URLs, non-image data: URIs and video.

Routing moves to praisonaiagents/agent/attachments.py with one invariant:
an attachment either becomes a model-visible content part or produces a
loud signal. The split is on who can act on it, not on severity.

Addressing errors raise AttachmentError (a ValueError subclass, so
existing handlers still catch): the path does not exist, is a directory,
is unreadable, or is not a str/PathLike/dict. These are always caller
mistakes and there is no useful degraded answer -- continuing means the
model confidently discussing a document it never saw. The message names
the path and the cwd, since a relative path resolved from an unexpected
directory is the usual cause.

Capability limits warn and degrade visibly: the file is readable but the
target model cannot ingest that type. A logging.warning names the file,
model and remedy, AND a marker part is injected into the prompt so the
model knows too and can say it answered without the file. A log line
alone would still leave the answer looking complete.

PDF, audio and ~45 text/code extensions are now routed per model
capability via new supports_pdf_input / supports_audio_input /
supports_vision helpers. PDF degrades by local text extraction, labelled
as such. Images are unchanged, byte for byte.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-07 12:40:28 +01:00
47 changed files with 500 additions and 5712 deletions

No files matched your search

-59
View File
@@ -1,59 +0,0 @@
# Why this exists
#
# praisonai-agents has ~11,900 tests. Until this gate, CI executed 15 of them:
# one smoke step in test-optimized.yml naming two files. Everything else was
# collected by nobody.
#
# The cost of that showed up as two test modules that could not even be
# IMPORTED -- they referenced symbols moved or replaced months earlier, so 19
# tests silently contributed nothing while still appearing to exist. A failing
# test is loud; a test that never collects is not.
#
# This job does not run the suite. Running all 11,900 on every PR is a real
# decision about Actions compute, and it would go red on arrival because there
# are pre-existing failures nothing has been enforcing. A job that is red the
# day it lands teaches people to ignore it.
#
# Collection alone takes ~40s, costs almost nothing, and would have caught both
# dead modules the day they broke. It is the honest ratchet: it cannot pass
# while a test file is unimportable.
name: agents-collect-gate
on:
push:
branches: [ main ]
paths:
- 'src/praisonai-agents/**'
- '.github/workflows/agents-collect-gate.yml'
pull_request:
paths:
- 'src/praisonai-agents/**'
- '.github/workflows/agents-collect-gate.yml'
jobs:
collect:
name: every praisonai-agents test file imports
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install
run: |
set -euo pipefail
python -m pip install --upgrade pip
pip install -e src/praisonai-agents
pip install pytest pytest-asyncio pytest-timeout
- name: Collect the whole suite
working-directory: src/praisonai-agents
env:
# A key must be present or some modules bail at import; it is never
# used, because collection does not execute test bodies.
OPENAI_API_KEY: 'sk-collect-only-not-a-real-key'
run: |
set -euo pipefail
# No pipe here: piping pytest into tail discards its exit code, which
# is how a gate silently stops gating.
python -m pytest tests/ --collect-only -q --disable-warnings
+2 -5
View File
@@ -21,10 +21,7 @@ jobs:
- uses: actions/setup-python@v5
with:
# harbor requires >=3.12; on 3.11 `pip install harbor` resolved to no
# candidate version at all and every scheduled run died in ~15s, so
# this gate has never produced a Terminal-Bench score.
python-version: "3.12"
python-version: "3.11"
- name: Install Harbor + PraisonAI
run: |
@@ -37,7 +34,7 @@ jobs:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
PYTHONPATH: .
run: |
harbor run -c examples/terminal_bench/job_code_smoke.yaml
harbor run -c examples/terminal_bench/job_code_smoke.yaml || true
- name: Upload Harbor results
if: always()
-3
View File
@@ -26,9 +26,6 @@ output
!src/praisonai-code/praisonai_code/cli/output/
!src/praisonai-bot/praisonai_bot/cli/output/
!src/praisonai-ts/src/db/
# The bare `db` rule above also swallows the SQLite adapter's test
# directory, so the durability suite would silently never be committed.
!src/praisonai-ts/tests/unit/db/
!src/praisonai-ts/src/cli/output/
.files
-2
View File
@@ -53,11 +53,9 @@ Methods:
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">get_recommended_stage</a>(prompt: str) -> str</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">get_rules_context</a>(file_path: Optional[str] = None, include_manual: Optional[List[str]] = None) -> str</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">get_skills_prompt</a>() -> str</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">guardrail_retry_count</a>() -> int</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">handoff_to</a>(target_agent: 'Agent', prompt: str, context: Optional[Dict[str, Any]] = None, config: Optional['HandoffConfig'] = None) -> 'HandoffResult'</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">handoff_to_async</a>(target_agent: 'Agent', prompt: str, context: Optional[Dict[str, Any]] = None, config: Optional['HandoffConfig'] = None) -> 'HandoffResult'</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">is_closed</a>() -> bool</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">last_guardrail_error</a>()</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">last_stop_reason</a>() -> str</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">list_mcp_servers</a>() -> List[str]</code>
* <code title="class Agent">Agent.<a href="./src/praisonai-agents/praisonaiagents/agent/agent.py">llm_instance</a>()</code>
@@ -16,13 +16,9 @@ Architecture:
Harbor Container → `praisonai code -p --output json "TASK" --dangerously-skip-approval`
Notes:
- `--dangerously-skip-approval` registers an always-approve backend on the
approval registry (plus PRAISON_APPROVAL_MODE=auto /
PRAISONAI_TOOL_SAFETY=off) so the assistant runs fully autonomously in the
container. Before that registry wiring existed the flag left every
critical-tool prompt in place and the non-TTY container auto-denied all of
them, so the agent could not run a single shell command while still
exiting 0 -- one cause of the 0.000 scores in #3932.
- `--dangerously-skip-approval` sets PRAISON_APPROVAL_MODE=auto +
PRAISONAI_TOOL_SAFETY=off so the assistant runs fully autonomously in the
container (no approval hang in a non-TTY session).
- The JSON envelope provides token, cost, session, and status metadata while
the real exit status still surfaces install/auth/startup failures.
- The base `praisonai` package is sufficient; heavy `code` extras are not
+9 -9
View File
@@ -18,19 +18,19 @@ This report compares **Python SDK feature categories** against **Python document
| Category | Features | Docs | Lines |
|----------|----------|------|-------|
| ✅ AGUI | 1 | 2 | 511 |
| ✅ Agent | 22 | 57 | 18181 |
| ✅ Agent | 22 | 57 | 18177 |
| ✅ Agent-to-Agent (A2A) | 1 | 7 | 2996 |
| ✅ Agent-to-User (A2U) | 1 | 3 | 701 |
| ✅ Approval | 1 | 7 | 3644 |
| ✅ Approval | 1 | 7 | 3616 |
| ✅ Audio | 2 | 12 | 1000 |
| ✅ Auto Generation | 5 | 12 | 3879 |
| ✅ Autonomy | 3 | 7 | 2794 |
| ✅ Bots | 7 | 33 | 13928 |
| ✅ Budget | 1 | 1 | 287 |
| ✅ CLI | 5 | 123 | 38327 |
| ✅ CLI | 5 | 123 | 38201 |
| ✅ Chunking | 2 | 2 | 424 |
| ✅ Citations | 2 | 1 | 202 |
| ✅ Code Execution | 2 | 14 | 4974 |
| ✅ Code Execution | 2 | 14 | 4855 |
| ✅ Conditions | 1 | 3 | 1386 |
| ✅ Configuration | 3 | 8 | 3323 |
| ✅ Context Management | 16 | 36 | 14104 |
@@ -43,10 +43,10 @@ This report compares **Python SDK feature categories** against **Python document
| ✅ Failover | 2 | 1 | 480 |
| ✅ Files | 2 | 7 | 2376 |
| ✅ Flow | 1 | 3 | 788 |
| ✅ Gateway | 7 | 87 | 32733 |
| ✅ Gateway | 7 | 86 | 32228 |
| ✅ Guardrails | 4 | 4 | 2388 |
| ✅ Handoffs | 11 | 6 | 2718 |
| ✅ Hooks | 2 | 9 | 4696 |
| ✅ Hooks | 2 | 9 | 4690 |
| ✅ Image | 1 | 11 | 1255 |
| ✅ Knowledge | 4 | 18 | 5562 |
| ✅ LLM | 3 | 15 | 5917 |
@@ -59,7 +59,7 @@ This report compares **Python SDK feature categories** against **Python document
| ✅ Output | 3 | 5 | 1273 |
| ✅ Parallel Execution | 3 | 2 | 497 |
| ✅ Planning | 6 | 7 | 1825 |
| ✅ Plugins | 8 | 5 | 3139 |
| ✅ Plugins | 8 | 5 | 3124 |
| ✅ Prompts | 2 | 9 | 2018 |
| ✅ Providers | 1 | 57 | 8636 |
| ✅ Query | 1 | 2 | 736 |
@@ -70,12 +70,12 @@ This report compares **Python SDK feature categories** against **Python document
| ✅ Routing | 1 | 2 | 430 |
| ✅ Sandbox | 5 | 7 | 3670 |
| ✅ Security | 1 | 3 | 3558 |
| ✅ Sessions | 4 | 20 | 7930 |
| ✅ Sessions | 4 | 20 | 7898 |
| ✅ Skills | 6 | 16 | 5984 |
| ✅ Tasks | 2 | 8 | 3231 |
| ✅ Telemetry | 1 | 2 | 660 |
| ✅ Templates | 1 | 8 | 1667 |
| ✅ Tools | 12 | 145 | 44263 |
| ✅ Tools | 12 | 145 | 44258 |
| ✅ Tracing | 3 | 2 | 139 |
| ✅ Vector Store | 1 | 12 | 1219 |
| ✅ Video | 2 | 6 | 728 |
@@ -2233,25 +2233,6 @@ class Agent(GoalLoopMixin, SteeringMixin, SandboxMixin, SkillReviewMixin, Unifie
if _embed_model:
embedder_config = _local_embedder(
_target.engine, _target.base_url, _embed_model)
# Carry the real vector width. local/ is a dependency sink
# and cannot import the dimension table, so the caller
# supplies it: a store created at the wrong width either
# rejects every write or silently corrupts the index.
# A width the server *measured* (already in the config) is
# authoritative and must win over the static table. The
# table only fills a genuine gap, and even then never with
# the generic default: a guessed width is worse than letting
# the store infer one from the first vector.
_cfg = embedder_config.setdefault("config", {})
if not _cfg.get("embedding_dims"):
try:
from ..embedding.dimensions import (
DEFAULT_DIMENSION, get_dimensions)
_dims = get_dimensions(_embed_model)
if _dims and _dims != DEFAULT_DIMENSION:
_cfg["embedding_dims"] = _dims
except Exception: # noqa: BLE001 -- a missing width must not break setup
pass
if retrieval_config is not None:
retrieval_config.setdefault('embedder_config', embedder_config)
else:
@@ -2277,10 +2258,20 @@ class Agent(GoalLoopMixin, SteeringMixin, SandboxMixin, SkillReviewMixin, Unifie
# was sent to OpenAI under a model name that was really a repr -- the
# opposite of what passing your own model means. Duck-typed on
# ``get_response`` so any conforming backend works, not just ``LLM``.
# Duck-typed across every backend surface this package documents, not
# just `get_response`: llm/protocols.py defines LLMProviderProtocol via
# `chat`/`achat` and UnifiedLLMProtocol via
# `chat_completion`/`achat_completion`. Checking only `get_response`
# left those backends falling through to the plain OpenAI branch, where
# the object became the model identifier -- the same defect this branch
# exists to fix, just for a different protocol.
_MODEL_BACKEND_METHODS = (
"get_response", "chat", "achat", "chat_completion", "achat_completion",
)
_is_model_instance = (
llm is not None
and not isinstance(llm, (str, dict))
and callable(getattr(llm, "get_response", None))
and any(callable(getattr(llm, m, None)) for m in _MODEL_BACKEND_METHODS)
)
if _is_model_instance:
self._llm_instance = llm
@@ -101,18 +101,6 @@ def get_model_limit(model: str) -> int:
Returns:
Context window size in tokens
"""
# A local server told us the real window when it was probed. litellm knows no
# local tag, so without this a 40960-token model inherits the generic 128000
# default and compaction budgets it at three times its real room -- the
# server then truncates silently. No probing here: this is a cached read.
try:
from ..local.embed import context_length_for
probed = context_length_for(model)
if probed:
return probed
except Exception: # noqa: BLE001 -- budgeting must never fail on a lookup
pass
# Prefer data-driven lookup via litellm (context window = max_input_tokens)
info = _litellm_model_info(model)
if info:
@@ -34,19 +34,7 @@ MODEL_DIMENSIONS: Dict[str, int] = {
# cross-checked against `<family>.embedding_length` from /api/show. Without
# these every local embedder inherited DEFAULT_DIMENSION (1536, OpenAI's),
# so a vector index was built 2x-4x the size the model actually produces.
"nomic-embed-text": 768,
# Locally-served embedders. Each was silently taking DEFAULT_DIMENSION
# (1536) -- harmless while nothing read the value, wrong now that it sizes
# the vector store.
"bge-m3": 1024,
"bge-large": 1024,
"bge-base": 768,
"bge-small": 384,
"snowflake-arctic-embed": 1024,
"snowflake-arctic-embed2": 1024,
"granite-embedding": 384,
"paraphrase-multilingual": 768,
"qwen3-embedding": 1024, # measured
"nomic-embed-text": 768, # measured
"mxbai-embed-large": 1024, # measured
"all-minilm": 384, # measured; Ollama's short tag for all-MiniLM
# HuggingFace common models
@@ -13,48 +13,6 @@ import json
from typing import Dict, Any, List, Optional
def collapse_union_param_types(tools):
"""Rewrite tool schemas a local server cannot parse.
Ollama models `parameters.type` as a Go string, so a union like
`{"type": ["string", "null"]}` -- which is exactly what an Optional[str]
tool argument produces -- fails to unmarshal and 400s the WHOLE request,
not just that tool. Collapsing the nullable to its single concrete member
keeps the request valid; the field simply stops advertising that it is
nullable.
Only the nullable case (exactly one concrete type plus "null") is
collapsed. A genuine heterogeneous union like ``["string", "integer"]``
is left untouched: narrowing it to one arm would misrepresent the tool's
accepted inputs and could push the model toward invalid calls. Such unions
are rare in generated tool schemas and are the server's problem to reject,
not ours to silently rewrite.
Returns a new list; the caller's tool definitions are never mutated.
"""
def fix(node):
if isinstance(node, list):
return [fix(item) for item in node]
if not isinstance(node, dict):
return node
out = {}
for key, value in node.items():
if key == "type" and isinstance(value, list):
concrete = [t for t in value if t != "null"]
# Collapse only "<type> | null"; preserve real multi-type unions.
if "null" in value and len(concrete) == 1:
out[key] = concrete[0]
else:
out[key] = value
else:
out[key] = fix(value)
return out
if not tools:
return tools
return [fix(tool) for tool in tools]
def _recover_json_tool_calls(response_text: str, tools: List[Dict[str, Any]]) -> Optional[List[Dict[str, Any]]]:
"""Recover a tool call a local model emitted as JSON text.
@@ -96,10 +54,6 @@ def _recover_json_tool_calls(response_text: str, tools: List[Dict[str, Any]]) ->
class DefaultAdapter:
"""Default provider adapter with sensible fallbacks."""
def format_tools(self, tools):
"""Hosted providers get the schema untouched -- they support all of it."""
return tools
def supports_prompt_caching(self) -> bool:
return False
@@ -229,14 +183,6 @@ Now provide your final answer using this result. Summarize the information natur
def recover_tool_calls_from_text(self, response_text: str, tools: List[Dict[str, Any]]) -> Optional[List[Dict[str, Any]]]:
"""Ollama-specific tool call recovery from response text."""
return _recover_json_tool_calls(response_text, tools)
def format_tools(self, tools):
# Ollama 400s the whole request on a union-typed parameter.
return collapse_union_param_types(tools)
# A JSON grammar and tools cannot coexist here: the grammar makes the
# tool-call tag unemittable and the model fabricates instead.
format_and_tools_conflict = True
def get_default_settings(self) -> Dict[str, Any]:
@@ -274,10 +220,6 @@ class LocalOpenAIAdapter(DefaultAdapter):
# answer with the tool call as text, especially after a repair prompt.
return _recover_json_tool_calls(response_text, tools)
def format_tools(self, tools):
# llama.cpp and vLLM share Ollama's strict parameter-type handling.
return collapse_union_param_types(tools)
class AnthropicAdapter(DefaultAdapter):
"""Anthropic/Claude provider adapter."""
@@ -371,7 +313,6 @@ def get_provider_adapter(name: str) -> LLMProviderAdapterProtocol:
__all__ = [
"collapse_union_param_types",
'DefaultAdapter',
'OllamaAdapter',
'LocalOpenAIAdapter',
@@ -2526,18 +2526,6 @@ Respond with ONLY a valid JSON tool call in this format:
logging.error(f"Tools are not JSON serializable: {e}")
return None
# Let the provider adapter rewrite anything its server cannot parse.
# Ollama 400s the entire request on a union-typed parameter, which is
# what an Optional[str] tool argument produces; hosted providers get
# the schema untouched.
if formatted_tools:
try:
adapter = getattr(self, '_provider_adapter', None)
if adapter is not None:
formatted_tools = adapter.format_tools(formatted_tools)
except Exception as e: # noqa: BLE001 -- never break tool formatting
logging.debug(f"Adapter tool formatting skipped: {e}")
# Cache the formatted tools
result = formatted_tools if formatted_tools else None
if len(self._formatted_tools_cache) < self._max_cache_size:
@@ -6056,30 +6044,6 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
return model
return f"openai/{model}"
def _guard_format_with_tools(self, params: Dict[str, Any]) -> None:
"""Refuse a combination that makes the model fabricate.
On Ollama a JSON grammar makes the tool-call tag unemittable, so the
model cannot call the tool and instead invents a confident answer,
returned with HTTP 200 and no warning. Measured: the same request
without the grammar calls the tool correctly; with it the model
returned an invented temperature having never run the tool.
Raising is what this package's own quirk table prescribes -- a
fabricated answer that looks right is worse than an error.
"""
adapter = getattr(self, '_provider_adapter', None)
if adapter is None or not getattr(adapter, 'format_and_tools_conflict', False):
return
if params.get("tools") and params.get("response_format"):
raise ValueError(
"This local provider cannot honour a structured-output schema and "
"tools in the same request: the JSON grammar makes the tool call "
"unemittable, so the model silently fabricates an answer instead "
"of calling your tool. Send tools without output_json/output_pydantic, "
"or drop the tools for this call."
)
def _build_completion_params(self, **override_params) -> Dict[str, Any]:
"""Build parameters for litellm completion calls with all necessary config"""
params = {
@@ -6328,7 +6292,6 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
logging.debug(f"Claude memory tool enabled with beta header: {beta_header}")
self._guard_format_with_tools(params)
return params
# ── Responses API support ───────────────────────────────────────────
@@ -15,51 +15,7 @@ from typing import Optional, Sequence, Tuple
from .capabilities import LocalEngine
__all__ = ["ENV_EMBED_MODEL", "PREFERRED_EMBED_MODELS",
"select_embedding_model", "local_embedder_config",
"remember_model_facts", "context_length_for", "embedding_dimension_for"]
# Facts the discovery probe already learned, kept so callers that cannot probe
# (a token budgeter on a hot path, an embedder factory) can still use them.
# Populated by resolve(); process-local; never triggers I/O of its own.
_MODEL_FACTS: dict = {}
def remember_model_facts(model_id, context_length=None, embedding_dimension=None) -> None:
"""Record what /api/show already told us about a model."""
if not model_id:
return
facts = _MODEL_FACTS.setdefault(model_id, {})
if context_length:
facts["context_length"] = int(context_length)
if embedding_dimension:
facts["embedding_dimension"] = int(embedding_dimension)
def context_length_for(model_id):
"""The model's real context window, or None if we never probed it.
Without this a local model inherits the generic 128000 default, so context
compaction budgets a 40960-token model as if it had three times the room and
the server truncates silently.
"""
if not model_id:
return None
for key in (model_id, model_id.split("/", 1)[-1]):
got = _MODEL_FACTS.get(key, {}).get("context_length")
if got:
return got
return None
def embedding_dimension_for(model_id):
"""The embedder's real vector width, or None if unknown."""
if not model_id:
return None
for key in (model_id, model_id.split("/", 1)[-1]):
got = _MODEL_FACTS.get(key, {}).get("embedding_dimension")
if got:
return got
return None
"select_embedding_model", "local_embedder_config"]
ENV_EMBED_MODEL = "PRAISONAI_LOCAL_EMBED_MODEL"
@@ -127,20 +83,17 @@ def local_embedder_config(engine: LocalEngine, base_url: str,
Shaped for the knowledge/memory layer (mem0-style provider + config).
"""
dimension = embedding_dimension_for(model)
if engine is LocalEngine.OLLAMA:
config = {"model": model, "ollama_base_url": base_url}
if dimension:
# Without this the store is created at whatever the first vector
# happens to be, or at a 1536 default that no local embedder emits.
config["embedding_dims"] = dimension
return {"provider": "ollama", "config": config}
return {
"provider": "ollama",
"config": {"model": model, "ollama_base_url": base_url},
}
# Everything else speaks OpenAI over HTTP.
trimmed = base_url.rstrip("/")
config = {
"model": model,
"openai_base_url": trimmed if trimmed.endswith("/v1") else trimmed + "/v1",
return {
"provider": "openai",
"config": {
"model": model,
"openai_base_url": trimmed if trimmed.endswith("/v1") else trimmed + "/v1",
},
}
if dimension:
config["embedding_dims"] = dimension
return {"provider": "openai", "config": config}
@@ -216,17 +216,6 @@ def build_target(discovery: Discovery, model_id, *, caps=None,
"model_id": model_evidence if model_id else Evidence.UNKNOWN,
"engine_version": Evidence.SERVER if discovery.engine_version else Evidence.UNKNOWN,
}
# Keep the facts the probe already learned reachable by callers that must
# not probe: the token budgeter and the embedder factory.
if model_id:
from .embed import remember_model_facts
_facts = dict(extra)
remember_model_facts(
model_id,
context_length=_facts.get("context_length"),
embedding_dimension=_facts.get("embedding_length"),
)
return LocalTarget(
engine=engine,
base_url=discovery.base_url,
@@ -10,12 +10,9 @@ Provides enhanced retrieval with:
No heavy imports at module level.
"""
import logging
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Protocol, runtime_checkable
logger = logging.getLogger(__name__)
@runtime_checkable
class RetrieverProtocol(Protocol):
@@ -56,14 +53,6 @@ class RetrievalResult:
reranked: bool = False
filtered: bool = False
metadata: Dict[str, Any] = field(default_factory=dict)
#: True when the knowledge store raised instead of returning matches. An
#: empty ``chunks`` list alone cannot tell a caller which happened, and the
#: two mean opposite things: "nothing matched" is an answer, "the store is
#: unreachable" means the agent is about to answer without its knowledge
#: base and should say so.
retrieval_failed: bool = False
#: The store's error message when ``retrieval_failed`` is True.
error: Optional[str] = None
class SmartRetriever:
@@ -138,13 +127,6 @@ class SmartRetriever:
user_id=user_id,
agent_id=agent_id,
)
# Surface a store failure on the result. Without this the caller sees an
# empty chunk list and cannot tell whether the knowledge base had no
# match or was never reached.
if getattr(self, "_last_error", None):
result.retrieval_failed = True
result.error = self._last_error
result.metadata["retrieval_error"] = self._last_error
chunks = self._normalize_results(search_results)
result.total_found = len(chunks)
@@ -178,7 +160,6 @@ class SmartRetriever:
agent_id: Optional[str] = None,
) -> Any:
"""Perform search on knowledge store."""
self._last_error = None
try:
return self._knowledge.search(
query,
@@ -186,30 +167,7 @@ class SmartRetriever:
agent_id=agent_id,
limit=top_k,
)
except Exception as e:
# Record the failure rather than returning a bare [] that is
# indistinguishable from "nothing matched". Knowledge.search itself
# re-raises, so reaching here means the store really is broken --
# misconfigured, unreachable, or refusing auth.
#
# Scope note: Agent._get_knowledge_context does NOT go through this
# class -- it calls Knowledge.search directly and lets the exception
# propagate, so the agent path is already honest. SmartRetriever is
# a public export (praisonaiagents.rag) with no internal callers, so
# this is a latent defect for direct users of the class, not a live
# agent failure.
self._last_error = f"{type(e).__name__}: {e}"
# Do not log the raw query: it is caller-supplied text that can carry
# personal data, credentials or proprietary content, and this line
# lands at ERROR in application logs. This package redacts exactly
# that elsewhere (trace/redact.py). The exception identifies the
# store problem; the query LENGTH distinguishes an empty probe from
# a real search without reproducing the text.
logger.error(
"Knowledge retrieval failed (query length %d): %s. "
"Returning no chunks; the caller will see an empty result.",
len(query or ""), self._last_error,
)
except Exception:
return []
def _normalize_results(self, results: Any) -> List[Dict[str, Any]]:
@@ -53,45 +53,6 @@ MIN_BRANCHES_FOR_LLM_SUMMARY = 3
# and acquire a *different* lock object (which would defeat the run guard).
_RUN_LOCK_INIT_GUARD = threading.Lock()
class _WriteTrackingDict(dict):
"""A ``dict`` that records which keys were assigned after construction.
A parallel branch runs against its own deep copy of the shared variables and
the branch's writes must be merged back (see ``_merge_branch_variables``). The
only reliable way to know *which* keys a branch wrote - not equality (drops an
equal-valued write, mis-flags a value whose ``!=`` is not a bool) nor identity
(blind to a rebind to an interned immutable like ``x = "SAME"``) - is to record
the assignment as it happens. Keys set at construction time seed the branch's
starting scope and are deliberately *not* counted as writes.
"""
__slots__ = ("written_keys",)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# A set, not a copy of the values: only the identity of *which* keys were
# written matters, and the current value is read from the dict at merge.
self.written_keys = set()
def __setitem__(self, key, value):
self.written_keys.add(key)
super().__setitem__(key, value)
def update(self, *args, **kwargs):
# dict.update does NOT route through __setitem__, so a StepResult.variables
# merge (``all_variables.update(result.variables)``) would otherwise be an
# invisible write. Record its keys explicitly.
merged = dict(*args, **kwargs)
self.written_keys.update(merged)
super().update(merged)
def setdefault(self, key, default=None):
# setdefault also bypasses __setitem__ and creates the key when absent.
if key not in self:
self.written_keys.add(key)
return super().setdefault(key, default)
class WorkflowStepError(Exception):
"""Exception raised when workflow step execution fails."""
def __init__(self, message: str, cause: Exception = None, errors: List = None):
@@ -2894,89 +2855,6 @@ CONCISE SUMMARY:"""
return min(user_max, n)
return min(DEFAULT_MAX_PARALLEL_WORKERS, n)
@staticmethod
def _branch_variable_delta(
branch_variables: Optional[Dict[str, Any]]
) -> Dict[str, Any]:
"""Return only the variables a parallel branch actually wrote.
A branch runs against its *own* ``copy.deepcopy`` of the shared scope, so
merging the whole returned dict back would copy every untouched key as
well - replacing the parent's objects with per-branch clones and letting a
stale copy overwrite a value some other part of the run had moved on from.
Only the delta is merged: the keys the branch actually assigned.
Writes are read from ``_WriteTrackingDict.written_keys`` - the exact set of
keys the branch bound via ``variables[name] = ...`` (an ``output_variable``
write, a ``StepResult.variables`` merge, or a nested pattern's own merge).
Recording the assignment itself is the only detection that is always right:
* value **equality** dropped an equal-valued write (writing a variable back
to the value it already held), losing declaration-order semantics on a
reused ``output_variable``, and it misclassified an untouched value whose
``!=`` is not a bool (numpy arrays, some ORM/pydantic objects) as a write;
* object **identity** still could not see a write whose value is an interned
immutable (``x = "SAME"`` rebinds to the very object already there).
Tracking the assignment sidesteps both: an untouched key is never in
``written_keys`` (so the parent keeps its own object and no ``!=`` is ever
evaluated), and every real write is present regardless of its value.
"""
if not branch_variables:
return {}
written = getattr(branch_variables, "written_keys", None)
if written is None:
# Not a tracking dict (a branch replaced ``ctx.variables`` wholesale, or
# a nested pattern returned a plain dict). Fall back to merging every
# key so a real write is never dropped - the bug this method prevents.
return dict(branch_variables)
return {key: branch_variables[key] for key in written if key in branch_variables}
@staticmethod
def _merge_branch_variables(
all_variables: Dict[str, Any],
branch_deltas: List[Tuple[int, Dict[str, Any]]]
) -> None:
"""Merge parallel branches' variable writes back into the shared scope.
THE MERGE RULE (please do not "simplify" this back into a race):
1. Branches never share a variables dict - each gets a deep copy - because
concurrent writes to one dict is a data race. Isolation is the safe half
of the design; this function is the missing other half, the merge back.
2. Deltas are applied in **branch declaration order**, never completion
order, so the result does not depend on thread scheduling. Two runs of
the same workflow produce the same variables.
3. On a **key collision** - two branches writing the same variable with
different values - the later branch in declaration order wins, which is
exactly what would happen if the same steps ran sequentially (Parallel
is meant to be a faster way to run steps, not a different semantics).
The collision is logged as a warning naming the key and both branches,
because a silently discarded branch result is the bug this fixes; the
workflow author has almost certainly reused an ``output_variable`` or a
step name by accident.
4. Only real writes are merged (see ``_branch_variable_delta``), so an
untouched key keeps the parent's own object.
"""
winners: Dict[str, int] = {}
for idx, delta in branch_deltas:
for key, value in delta.items():
previous_idx = winners.get(key)
if previous_idx is not None:
try:
conflicting = bool(all_variables[key] != value)
except Exception:
conflicting = True
if conflicting:
logger.warning(
f"Parallel branches {previous_idx} and {idx} both wrote "
f"variable '{key}' with different values; branch {idx} wins "
f"(declaration order). Give the branches distinct "
f"output_variable names to avoid losing a result."
)
winners[key] = idx
all_variables[key] = value
def _execute_parallel(
self,
parallel_step: Parallel,
@@ -3028,8 +2906,6 @@ CONCISE SUMMARY:"""
# Use ThreadPoolExecutor for parallel execution
from ..trace.context_events import copy_context_to_callable, get_context_emitter
branch_deltas = [] # [(branch_idx, {var: value}), ...] in declaration order
# Determine effective workers based on user configuration
user_max = getattr(parallel_step, 'max_workers', None)
effective_workers = self._effective_workers(user_max, len(parallel_step.steps), label="Parallel")
@@ -3037,21 +2913,12 @@ CONCISE SUMMARY:"""
futures = []
for idx, step in enumerate(parallel_step.steps):
# Wrap execution to propagate context and set branch_id for parallel tracking
# Each branch writes into its OWN deep copy: concurrent writes to
# one shared dict would be a data race. The copy is a
# _WriteTrackingDict so the merge back (see _merge_branch_variables)
# knows exactly which keys the branch assigned - the branch's
# output_variable writes are not thrown away, and untouched keys are
# not merged as per-branch clones. Keys present at construction seed
# the scope and are not counted as writes.
branch_vars = _WriteTrackingDict(copy.deepcopy(all_variables))
def execute_with_branch(step=step, idx=idx, opt_prev=optimized_previous, branch_vars=branch_vars):
def execute_with_branch(step=step, idx=idx, opt_prev=optimized_previous):
emitter = get_context_emitter()
emitter.set_branch(f"parallel_{idx}")
try:
return self._execute_single_step_internal(
step, opt_prev, input, branch_vars, model, False, idx, stream, depth=depth+1
step, opt_prev, input, copy.deepcopy(all_variables), model, False, idx, stream, depth=depth+1
)
finally:
emitter.clear_branch()
@@ -3085,9 +2952,6 @@ CONCISE SUMMARY:"""
if branch_error is None:
results.append({"step": step_result["step"], "output": step_result["output"]})
outputs.append(step_result["output"])
branch_deltas.append(
(idx, self._branch_variable_delta(step_result.get("variables")))
)
continue
logger.error(f"Parallel branch {idx} failed: {branch_error}")
@@ -3109,15 +2973,6 @@ CONCISE SUMMARY:"""
# Record the failure but continue with other branches. Do not
# fold the exception text into outputs as if it were data.
results.append({"step": f"parallel_{idx}", "output": None, "error": str(branch_error)})
# "partial_ok" means partial results are kept, so the variables
# the branch's *completed* steps wrote are kept too. (The failed
# step itself never reaches the output_variable write, so nothing
# from it can leak in here.) fail_fast/fail_all raise instead, so
# nothing from a failed branch is merged under those modes.
if step_result is not None:
branch_deltas.append(
(idx, self._branch_variable_delta(step_result.get("variables")))
)
# Keep `outputs` index-aligned with parallel_step.steps so a
# downstream reader of parallel_outputs[idx] cannot silently
# receive a later branch's result once an earlier one fails.
@@ -3135,11 +2990,6 @@ CONCISE SUMMARY:"""
f"{len(errors)} parallel branches failed", errors=errors, cause=cause
) from cause
# Merge each branch's variable writes back into the shared scope. Without
# this every `output_variable` set inside a Parallel block was written to a
# deep copy that was discarded when the branch returned (silent data loss).
self._merge_branch_variables(all_variables, branch_deltas)
# Combine outputs
combined_output = "\n---\n".join(str(o) for o in outputs)
all_variables["parallel_outputs"] = outputs
@@ -2,18 +2,6 @@
import os
import pytest
from unittest.mock import patch, MagicMock
# This module lives in the praisonaiagents test tree but reaches into the
# praisonai wrapper for its CLI-handler cases. The wrapper is a separate,
# optional package that is not installed when praisonaiagents is tested in
# isolation (e.g. the agents-collect-gate). Skip the whole module rather than
# fail collection when it is absent -- the agents-only assertions below still
# run wherever the wrapper happens to be present.
pytest.importorskip(
"praisonai.cli.features.agents",
reason="praisonai wrapper not installed (praisonaiagents tested in isolation)",
)
from praisonai.cli.features.agents import MultiAgentHandler
@@ -15,14 +15,7 @@ from praisonaiagents.runtime.middleware import (
NormalizedToolResult,
MiddlewareContext
)
# This file used to import `get_default_registry` from runtime.registry. The
# middleware registry has since been split into its own module, so the name it
# wanted now lives in runtime.middleware_registry. Nothing in CI ran this file,
# so the import break went unnoticed.
from praisonaiagents.runtime.middleware_registry import MiddlewareRegistry
from praisonaiagents.runtime.middleware_registry import (
get_default_middleware_registry as get_default_registry,
)
from praisonaiagents.runtime.registry import RuntimeRegistry, get_default_registry
@dataclass
@@ -78,7 +71,7 @@ class StubHarnessMiddleware:
def test_middleware_registry():
"""Test basic middleware registry functionality."""
registry = MiddlewareRegistry()
registry = RuntimeRegistry()
middleware = StubHarnessMiddleware()
# Test registration
@@ -1,119 +0,0 @@
"""A knowledge-store failure must not look like an empty result set.
SmartRetriever._search used to do `except Exception: return []` with no log
line, so a store that was unreachable, misconfigured or refusing auth produced
byte-identical output to a healthy store with no matches.
Scope: SmartRetriever is exported from praisonaiagents.rag but has no callers
inside the package. Agent._get_knowledge_context takes a different route --
Knowledge.search directly, letting the exception propagate -- so the agent path
already distinguished the two cases. This is a latent defect for direct users
of the public class, not a live agent failure.
"""
import logging
import pytest
from praisonaiagents.rag.retriever import SmartRetriever
class BrokenStore:
"""A store that is down: search raises."""
def search(self, *a, **k):
raise RuntimeError("vector store unreachable: connection refused")
class EmptyStore:
"""A healthy store that genuinely has no matching documents."""
def search(self, *a, **k):
return []
class PopulatedStore:
def search(self, *a, **k):
return [{"memory": "Refunds are issued within 30 days.", "score": 0.9}]
def test_broken_store_is_flagged_as_a_failure():
result = SmartRetriever(BrokenStore()).retrieve("refund policy", top_k=3)
assert result.retrieval_failed is True
assert result.error is not None
assert "unreachable" in result.error
assert result.metadata["retrieval_error"] == result.error
def test_control_empty_store_is_not_a_failure():
"""The control: no matches is an answer, not an error."""
result = SmartRetriever(EmptyStore()).retrieve("refund policy", top_k=3)
assert result.retrieval_failed is False
assert result.error is None
assert result.chunks == []
def test_broken_and_empty_are_now_distinguishable():
"""The defect itself: these two were once identical."""
broken = SmartRetriever(BrokenStore()).retrieve("refund policy", top_k=3)
empty = SmartRetriever(EmptyStore()).retrieve("refund policy", top_k=3)
assert broken.chunks == empty.chunks == [] # still true
assert broken.retrieval_failed != empty.retrieval_failed
def test_broken_store_logs_an_error(caplog):
with caplog.at_level(logging.ERROR, logger="praisonaiagents.rag.retriever"):
SmartRetriever(BrokenStore()).retrieve("refund policy", top_k=3)
messages = [r.getMessage() for r in caplog.records]
assert any("Knowledge retrieval failed" in m for m in messages), caplog.text
assert any("unreachable" in m for m in messages), caplog.text
def test_control_healthy_store_logs_no_error(caplog):
with caplog.at_level(logging.ERROR, logger="praisonaiagents.rag.retriever"):
SmartRetriever(EmptyStore()).retrieve("refund policy", top_k=3)
assert caplog.records == []
def test_control_successful_retrieval_still_returns_chunks():
result = SmartRetriever(PopulatedStore()).retrieve("refund policy", top_k=3)
assert result.retrieval_failed is False
assert len(result.chunks) == 1
def test_a_later_success_clears_the_previous_error():
"""One broken call must not poison the next healthy one."""
r = SmartRetriever(BrokenStore())
assert r.retrieve("q", top_k=3).retrieval_failed is True
r._knowledge = PopulatedStore()
second = r.retrieve("q", top_k=3)
assert second.retrieval_failed is False
assert second.error is None
class TestAgentPathAlreadyDistinguishes:
"""The agent's own retrieval route is honest -- lock that in.
A review of the original fix correctly pointed out that SmartRetriever has
no callers inside the package, so fixing it did not change agent behaviour.
Agent._get_knowledge_context calls Knowledge.search directly and only
catches ImportError, so a store failure propagates. These tests exist so
that stays true: wrapping that call in a broad `except Exception` later
would reintroduce exactly the bug this file is about, on the path that
actually matters.
"""
@staticmethod
def _agent_with(knowledge):
from praisonaiagents import Agent
agent = Agent(instructions="answer", llm="gpt-4o")
agent.knowledge = knowledge
agent._knowledge_sources = None
return agent
def test_broken_store_propagates_rather_than_returning_empty(self):
agent = self._agent_with(BrokenStore())
with pytest.raises(RuntimeError, match="unreachable"):
agent._get_knowledge_context("refund policy", use_rag=False)
def test_control_empty_store_returns_empty_context(self):
agent = self._agent_with(EmptyStore())
context, citations = agent._get_knowledge_context("refund policy", use_rag=False)
assert context == ""
assert citations is None
@@ -11,6 +11,7 @@ from praisonaiagents.session.title import (
generate_title_async,
_create_fallback_title,
_resolve_small_model,
_DEFAULT_SMALL_MODEL,
)
@@ -150,11 +151,7 @@ class TestSmallModelResolution:
def test_falls_back_to_default_when_nothing_set(self):
"""With nothing configured, preserve the historical default."""
# _DEFAULT_SMALL_MODEL was replaced by default_auxiliary_model(), resolved
# at call time so PRAISONAI_AUXILIARY_MODEL / OPENAI_MODEL_NAME set after
# import still take effect.
from praisonaiagents.llm.model_providers import default_auxiliary_model
assert _resolve_small_model(None, None) == default_auxiliary_model()
assert _resolve_small_model(None, None) == _DEFAULT_SMALL_MODEL
def test_config_small_model_used(self, monkeypatch):
"""A configured defaults.small_model is used over the primary model."""
@@ -323,3 +323,56 @@ def test_a_callable_returning_garbage_is_not_swallowed(no_network):
with pytest.raises(ScriptedModelError, match="Script entry #1"):
agent.start("hi")
class TestEveryDocumentedBackendProtocolIsAdopted:
"""Agent(llm=<backend object>) must adopt the object, whichever protocol it implements.
The original fix duck-typed on `get_response` alone. Review pointed out that
llm/protocols.py also documents LLMProviderProtocol (`chat`/`achat`) and
UnifiedLLMProtocol (`chat_completion`/`achat_completion`); backends
implementing those still fell through to the plain OpenAI branch, where the
object became the model identifier and the turn went to the wrong backend.
"""
@staticmethod
def _agent(backend):
from praisonaiagents import Agent
return Agent(instructions="t", llm=backend)
def test_get_response_backend_is_adopted(self):
class B:
model = "custom/a"
def get_response(self, *a, **k): return "x"
agent = self._agent(B())
assert agent._using_custom_llm is True
assert agent.llm == "custom/a"
def test_chat_protocol_backend_is_adopted(self):
class B:
model = "custom/b"
def chat(self, *a, **k): return "x"
agent = self._agent(B())
assert agent._using_custom_llm is True
assert agent.llm == "custom/b"
def test_chat_completion_protocol_backend_is_adopted(self):
class B:
model = "custom/c"
def chat_completion(self, *a, **k): return "x"
async def achat_completion(self, *a, **k): return "x"
agent = self._agent(B())
assert agent._using_custom_llm is True
assert agent.llm == "custom/c"
def test_control_a_plain_object_is_not_adopted(self):
"""Control: adoption must be earned by implementing a backend surface."""
class NotABackend:
pass
agent = self._agent(NotABackend())
assert agent._using_custom_llm is False
def test_control_a_string_llm_is_unaffected(self):
from praisonaiagents import Agent
agent = Agent(instructions="t", llm="gpt-4o-mini")
assert agent.llm == "gpt-4o-mini"
@@ -1,332 +0,0 @@
"""
Regression tests: variables written inside a ``Parallel`` block must survive.
Every step inside a ``Parallel`` used to write its ``output_variable`` into a
``copy.deepcopy(all_variables)`` that was thrown away when the branch returned,
so the result was silently discarded:
branch_a_output present False
branch_b_output present False
after_output (control) True
The isolation itself is deliberate (concurrent branches writing one shared dict
is a data race); what was missing was the merge back. These tests pin the merge
rule: declaration-order application, last-declared branch wins a key collision
(with a warning), only real writes merged, and partial_ok keeping the writes a
failed branch had already made.
"""
import logging
import time
import pytest
from praisonaiagents import Workflow, WorkflowContext, StepResult
from praisonaiagents.task.task import Task
from praisonaiagents.workflows import parallel, loop
def _handler(output, record=None, sleep=0.0):
def run(ctx: WorkflowContext) -> StepResult:
if sleep:
time.sleep(sleep)
if record is not None:
record.append(dict(ctx.variables))
return StepResult(output=output)
return run
# ---------------------------------------------------------------------------
# The reproduction
# ---------------------------------------------------------------------------
def test_parallel_branch_output_variables_survive_the_block():
"""Both branches' output_variable writes must be visible after the block.
The ``after_output`` control assertion keeps this test from passing
vacuously: it fails on the buggy code too if variables stop being recorded
at all.
"""
seen = []
wf = Workflow(steps=[
parallel([
Task(name="branch_a", handler=_handler("A"),
output_variable="branch_a_output", max_retries=0),
Task(name="branch_b", handler=_handler("B"),
output_variable="branch_b_output", max_retries=0),
]),
Task(name="after", handler=_handler("AFTER", record=seen),
output_variable="after_output", max_retries=0),
])
result = wf.start("go")
variables = result["variables"]
assert result["status"] == "completed"
assert variables.get("branch_a_output") == "A", (
f"branch_a's output_variable was discarded: {sorted(variables)}"
)
assert variables.get("branch_b_output") == "B", (
f"branch_b's output_variable was discarded: {sorted(variables)}"
)
# Control: a step outside the block has always worked, so this asserts the
# test is measuring the parallel-specific loss and not a broken harness.
assert variables.get("after_output") == "AFTER"
# And the branch variables must be visible to the *next* step, not merely
# in the final result dict.
assert seen and seen[0].get("branch_a_output") == "A"
assert seen[0].get("branch_b_output") == "B"
def test_default_step_output_variable_name_also_survives():
"""A branch step with no explicit output_variable writes ``{name}_output``."""
wf = Workflow(steps=[
parallel([
Task(name="alpha", handler=_handler("A"), max_retries=0),
Task(name="beta", handler=_handler("B"), max_retries=0),
]),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
])
variables = wf.start("go")["variables"]
assert variables.get("alpha_output") == "A"
assert variables.get("beta_output") == "B"
assert variables.get("after_output") == "AFTER" # control
# ---------------------------------------------------------------------------
# The merge rule
# ---------------------------------------------------------------------------
def test_key_collision_last_declared_branch_wins_and_warns(caplog):
"""Two branches writing the same variable is ambiguous, not silent.
The winner is the last branch in *declaration* order - what sequential
execution of the same steps would produce - and the collision is logged.
"""
wf = Workflow(steps=[
parallel([
Task(name="first", handler=_handler("FROM-FIRST"),
output_variable="shared", max_retries=0),
Task(name="second", handler=_handler("FROM-SECOND"),
output_variable="shared", max_retries=0),
]),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
])
with caplog.at_level(logging.WARNING):
variables = wf.start("go")["variables"]
assert variables.get("shared") == "FROM-SECOND"
assert variables.get("after_output") == "AFTER" # control
assert any("shared" in rec.getMessage() and "Parallel branches" in rec.getMessage()
for rec in caplog.records), (
f"collision was not reported: {[r.getMessage() for r in caplog.records]}"
)
def test_collision_winner_does_not_depend_on_completion_order():
"""The first-declared branch finishes last here; declaration order still wins."""
wf = Workflow(steps=[
parallel([
Task(name="slow", handler=_handler("FROM-SLOW", sleep=0.25),
output_variable="shared", max_retries=0),
Task(name="fast", handler=_handler("FROM-FAST"),
output_variable="shared", max_retries=0),
], max_workers=2),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
])
variables = wf.start("go")["variables"]
assert variables.get("shared") == "FROM-FAST"
assert variables.get("after_output") == "AFTER" # control
def test_branches_do_not_write_back_variables_they_never_touched():
"""Only real writes merge; an untouched value keeps the parent's own object.
Each branch works on a deep copy, so merging a branch's whole dict back
would replace shared inputs with per-branch clones.
"""
original = {"a": [1, 2, 3]}
wf = Workflow(steps=[
parallel([
Task(name="one", handler=_handler("1"), output_variable="v1", max_retries=0),
Task(name="two", handler=_handler("2"), output_variable="v2", max_retries=0),
]),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
], variables={"payload": original})
variables = wf.start("go")["variables"]
assert variables["v1"] == "1" and variables["v2"] == "2"
assert variables["payload"] is original, (
"an untouched variable was replaced by a branch's deep copy"
)
def test_a_branch_may_deliberately_overwrite_an_existing_variable():
"""output_variable naming a pre-existing workflow variable must take effect."""
wf = Workflow(steps=[
parallel([
Task(name="rewriter", handler=_handler("NEW"),
output_variable="topic", max_retries=0),
]),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
], variables={"topic": "OLD"})
variables = wf.start("go")["variables"]
assert variables["topic"] == "NEW"
assert variables.get("after_output") == "AFTER" # control
def test_partial_ok_keeps_the_variables_a_failed_branch_did_write():
""""partial_ok" means partial results are kept - including partial variables.
The failing branch here writes a variable through ``StepResult.variables``
and is then rejected by its guardrail, so it is a branch that genuinely
failed yet genuinely wrote something. Its own ``output_variable`` is never
reached (the failure path returns before that write), so nothing the failure
invented can leak in.
"""
def writes_then_fails_guardrail(ctx: WorkflowContext) -> StepResult:
return StepResult(output="OK", variables={"partial_var": "OK"})
wf = Workflow(steps=[
parallel([
Task(name="half_done", handler=writes_then_fails_guardrail,
guardrails=lambda result: (False, "never valid"),
output_variable="half_done_var", max_retries=0,
on_error="continue"),
Task(name="healthy", handler=_handler("H"),
output_variable="healthy_var", max_retries=0),
], on_failure="partial_ok"),
])
result = wf.start("go")
variables = result["variables"]
assert result["status"] == "failed" # the branch really did fail
assert variables.get("healthy_var") == "H" # control: good branch merged
assert variables.get("partial_var") == "OK", (
"partial_ok discarded a variable the failed branch had already written"
)
assert "half_done_var" not in variables, (
"a failed step's output_variable must not be written"
)
@pytest.mark.parametrize("mode", ["fail_fast", "fail_all"])
def test_failing_modes_still_raise_and_merge_nothing(mode):
"""fail_fast/fail_all abort the block; the merge must not paper over that."""
from praisonaiagents.workflows.workflows import WorkflowStepError
def boom(ctx: WorkflowContext) -> StepResult:
raise RuntimeError("boom")
seen = []
wf = Workflow(steps=[
parallel([
Task(name="bad", handler=boom, output_variable="bad_var", max_retries=0),
], on_failure=mode),
Task(name="after", handler=_handler("AFTER", record=seen), max_retries=0),
])
with pytest.raises(WorkflowStepError):
wf.start("go")
# Nothing downstream ran, so no branch variable can have leaked forward.
assert seen == []
# ---------------------------------------------------------------------------
# Loop: deliberately iteration-scoped, and identical parallel vs sequential
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("is_parallel", [False, True])
def test_loop_iteration_variables_stay_iteration_scoped(is_parallel):
"""Loop is NOT the same case as Parallel and must not be "fixed" to match.
N iterations share one step name, so a per-iteration output_variable has N
conflicting values; the aggregate is exposed through the Loop's own
``output_variable`` / ``loop_outputs`` instead. Sequential and parallel
loops behave identically, and that parity is what this test pins.
"""
wf = Workflow(steps=[
loop(steps=[Task(name="inner",
handler=lambda ctx: StepResult(output=f"X-{ctx.variables.get('item')}"),
output_variable="inner_var", max_retries=0)],
over="items", parallel=is_parallel, output_variable="collected"),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
], variables={"items": ["a", "b"]})
variables = wf.start("go")["variables"]
assert variables["collected"] == ["X-a", "X-b"]
assert variables["loop_outputs"] == ["X-a", "X-b"]
assert "inner_var" not in variables
assert variables.get("after_output") == "AFTER" # control
# ---------------------------------------------------------------------------
# Write detection is by identity, not value equality
# ---------------------------------------------------------------------------
def test_equal_valued_write_still_wins_collision_in_declaration_order(caplog):
"""A later branch writing the same value the variable already held is a write.
Baseline ``shared="SAME"``; the first branch rebinds it to ``"OTHER"`` and the
second rebinds it back to ``"SAME"``. Sequential execution of these steps ends
with ``"SAME"`` (second is last), so the parallel merge must too. Value-equality
detection dropped the second branch's write because it equalled the baseline,
leaving ``"OTHER"`` and suppressing the collision warning.
"""
wf = Workflow(steps=[
parallel([
Task(name="first", handler=_handler("OTHER"),
output_variable="shared", max_retries=0),
Task(name="second", handler=_handler("SAME"),
output_variable="shared", max_retries=0),
]),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
], variables={"shared": "SAME"})
with caplog.at_level(logging.WARNING):
variables = wf.start("go")["variables"]
assert variables.get("shared") == "SAME", (
"an equal-valued write was dropped, breaking declaration-order semantics"
)
assert variables.get("after_output") == "AFTER" # control
assert any("shared" in rec.getMessage() and "Parallel branches" in rec.getMessage()
for rec in caplog.records), (
"an equal-valued collision must still be reported"
)
def test_untouched_value_with_non_bool_inequality_is_not_a_write(caplog):
"""A carried-through object whose ``!=`` is not a bool must not become a write.
``_NonBoolEq.__ne__`` returns a non-bool (like a numpy array), so value-based
detection ``bool(value != original)`` raised and conservatively classified the
untouched value as written - replacing the parent's object with a branch-local
clone and emitting a spurious collision warning across branches. Identity
detection never evaluates ``!=`` for an untouched key.
"""
class _NonBoolEq:
def __ne__(self, other):
return [True] # not a bool; bool([True]) is True but the array case raises
__hash__ = None
sentinel = _NonBoolEq()
wf = Workflow(steps=[
parallel([
Task(name="one", handler=_handler("1"), output_variable="v1", max_retries=0),
Task(name="two", handler=_handler("2"), output_variable="v2", max_retries=0),
]),
Task(name="after", handler=_handler("AFTER"), max_retries=0),
], variables={"carried": sentinel})
with caplog.at_level(logging.WARNING):
variables = wf.start("go")["variables"]
assert variables["v1"] == "1" and variables["v2"] == "2"
assert variables["carried"] is sentinel, (
"an untouched non-comparable value was replaced by a branch's deep copy"
)
assert not any("carried" in rec.getMessage() for rec in caplog.records), (
"an untouched value must not produce a collision warning"
)
@@ -16,47 +16,20 @@ def call_main(
ctx: typer.Context,
model: Optional[str] = typer.Option(None, "--model", "-m", help="LLM model to use"),
verbose: bool = typer.Option(False, "--verbose", "-v", help="Verbose output"),
public: bool = typer.Option(
False,
"--public",
help="Expose the call server publicly via ngrok (required for inbound PSTN webhooks)",
),
port: Optional[int] = typer.Option(
None, "--port", help="Port to run the call server on (default: $PORT or 8090)"
),
host: Optional[str] = typer.Option(
None, "--host", help="Host to bind the call server to (default: 127.0.0.1)"
),
):
"""
Start voice/call interaction mode.
--public/--port/--host are forwarded to the call server. They were
previously reachable only through the PUBLIC/PORT environment variables,
which made inbound PSTN (the one setup that needs a public webhook)
unreachable from the CLI.
Examples:
praisonai call
praisonai call --model gpt-4o
praisonai call --public
praisonai call --host 0.0.0.0 --port 9000
"""
from praisonai_code._wrapper_bridge import run_wrapper_command
argv = ['call']
if model:
argv.extend(['--model', model])
if verbose:
argv.append('--verbose')
if public:
argv.append('--public')
# Only forward when the user actually asked: the call server's own
# defaults differ from the legacy argparse defaults (PORT/8090 vs 8005),
# so forwarding unconditionally would silently retarget the port.
if port is not None:
argv.extend(['--port', str(port)])
if host is not None:
argv.extend(['--host', host])
run_wrapper_command(argv, feature="call")
@@ -14,125 +14,6 @@ from praisonai_code.cli.utils.env_utils import scopes_no_plugins
app = typer.Typer(help="Terminal-native code assistant mode")
# Default multi-step tool budget for a *coding* session.
#
# The core Agent defaults (ExecutionConfig.max_iter=20, max_tool_calls_per_turn
# =10) are general-purpose: they suit a question-answering agent with two or
# three tools, and they are deliberately left alone here so no existing library
# user's behaviour changes. They are far too small for a coding agent, whose
# unit of work is read -> edit -> run tests -> read the failure: roughly four
# tool calls per fix attempt, before any orientation (ls/glob/grep) at all. Ten
# calls is "list files, read two files, one grep" — it cannot reach a test run,
# so every non-trivial task truncated mid-way.
#
# 200 is chosen to cover the *tail* rather than the median: agentic coding
# trajectories cluster around 20-40 steps but the hard tasks — the ones a
# benchmark actually measures — run well past 100, and a budget set at the
# median silently truncates exactly those. 200 steps is ~50 edit/verify cycles,
# which comfortably covers a multi-file fix while still bounding a genuinely
# runaway loop. The orthogonal guards are unaffected and still stop a broken
# tool much earlier: the per-tool loop guard, ExecutionConfig.max_execution_time,
# max_budget, and the model's own context limit.
DEFAULT_CODE_MAX_STEPS = 200
# Env fallback so a sandbox/CI runner can set the budget without editing the
# command line (mirrors the --append-system-prompt / PRAISONAI_* convention).
_MAX_STEPS_ENV = "PRAISONAI_CODE_MAX_STEPS"
def resolve_code_max_steps(max_steps=None) -> int:
"""Resolve the coding session's multi-step tool budget.
Precedence: explicit ``--max-steps`` > ``PRAISONAI_CODE_MAX_STEPS`` >
:data:`DEFAULT_CODE_MAX_STEPS`. An unparseable or non-positive value falls
through to the next source rather than failing the run.
"""
import os as _os
for candidate in (max_steps, _os.environ.get(_MAX_STEPS_ENV)):
if candidate is None or candidate == "":
continue
try:
value = int(candidate)
except (TypeError, ValueError):
continue
if value > 0:
return value
return DEFAULT_CODE_MAX_STEPS
def build_code_execution_config(max_steps=None):
"""Build the ``ExecutionConfig`` a ``praisonai code`` session runs under.
Sets *both* budget knobs to the resolved value because the two tool loops
enforce different ones: the OpenAI-native loop is bounded by ``max_steps``
(via ``Agent._resolve_max_steps``), while the LiteLLM loop additionally
counts every tool call in the turn against ``max_tool_calls_per_turn`` —
which is why a Gemini/Anthropic-routed session stopped after exactly ten
calls while an OpenAI-routed one ran to twenty. Raising only one leaves the
other as the real (and invisible) ceiling.
Returns ``None`` if the core config type cannot be imported, so the agent
keeps its own defaults rather than failing to start.
"""
try:
from praisonaiagents.config import ExecutionConfig
except Exception: # noqa: BLE001 — budget is an enhancement, never a hard dep
try:
from praisonaiagents.config.feature_configs import ExecutionConfig
except Exception: # noqa: BLE001
return None
budget = resolve_code_max_steps(max_steps)
return ExecutionConfig(
max_steps=budget,
max_tool_calls_per_turn=budget,
)
def _install_bypass_approval_backend() -> None:
"""Register an always-approve backend for --no-safe/--dangerously-skip-approval.
The env vars those flags set are read by the CLI's own tool wiring, but the
core ``@require_approval`` decorator that gates critical tools resolves its
decision through the approval registry's backend. Registering
``AutoApproveBackend`` there is what actually makes the opt-out flags skip
approval — the same mechanism ``--plan`` uses to *tighten* the policy, used
here to loosen it.
Strictly opt-in: only ever called from the explicit --no-safe /
--dangerously-skip-approval branch, so the safe-by-default path is
untouched. Best-effort — a registry failure leaves approvals in place
(fails closed).
"""
try:
from praisonaiagents.approval import get_approval_registry
from praisonaiagents.approval.backends import AutoApproveBackend
get_approval_registry().set_backend(AutoApproveBackend())
except Exception: # noqa: BLE001 — never fail *open* on a wiring error
pass
def _clear_bypass_approval_backend() -> None:
"""Undo :func:`_install_bypass_approval_backend` for a safe-mode run.
Removes the global backend only when it is the ``AutoApproveBackend`` this
module installs, so a bypass left over from an earlier ``--no-safe`` call in
the same process (REPL/worker/test) cannot silently keep a later
safe-by-default session unguarded, while a backend registered by a caller or
by ``--plan`` is preserved.
"""
try:
from praisonaiagents.approval import get_approval_registry
from praisonaiagents.approval.backends import AutoApproveBackend
registry = get_approval_registry()
if isinstance(registry.get_backend(), AutoApproveBackend):
registry.remove_backend()
except Exception: # noqa: BLE001 — best-effort cleanup
pass
@app.callback(invoke_without_command=True)
@scopes_no_plugins
def code_main(
@@ -155,7 +36,6 @@ def code_main(
continue_session: bool = typer.Option(False, "--continue", "-c", help="Continue last session"),
agent: Optional[str] = typer.Option(None, "--agent", "-a", help="Use a named custom agent profile (applies its tools and permission/mode scope)"),
thinking: Optional[str] = typer.Option(None, "--thinking", help="Reasoning effort (off, minimal, low, medium, high)"),
max_steps: Optional[int] = typer.Option(None, "--max-steps", help=f"Maximum tool-calling steps for this coding session (default: {DEFAULT_CODE_MAX_STEPS}). Env fallback: PRAISONAI_CODE_MAX_STEPS"),
autonomy: bool = typer.Option(True, "--autonomy/--no-autonomy", help="Enable agent autonomy for complex tasks"),
append_system_prompt: Optional[str] = typer.Option(None, "--append-system-prompt", help="Append text (or @file) to the system prompt for this invocation only. Env fallback: PRAISONAI_APPEND_SYSTEM_PROMPT"),
profile: bool = typer.Option(False, "--profile", help="Enable CLI profiling (timing breakdown)"),
@@ -344,27 +224,12 @@ def code_main(
if dangerously_skip_approval or not safe_mode:
os.environ["PRAISON_APPROVAL_MODE"] = "auto"
os.environ["PRAISONAI_TOOL_SAFETY"] = "off"
# PRAISON_APPROVAL_MODE/PRAISONAI_TOOL_SAFETY are read by the CLI's own
# tool wiring, but the gate that actually prompts for a critical tool
# (e.g. acp_execute_command) is the core `@require_approval` decorator,
# which consults the approval *registry* backend. Without registering a
# bypass backend the flag left every prompt in place — and a scripted
# run with no tty answered "no" to each one, so the agent could not run
# a single command while still exiting 0. Install the same
# AutoApproveBackend the `--plan` branch already registers below (one
# mechanism, not a second env var).
_install_bypass_approval_backend()
else:
os.environ["PRAISON_APPROVAL_MODE"] = "prompt"
# Clear any stale bypass from a previous --no-safe run in the same
# process (REPL/worker/test). Leaving PRAISONAI_TOOL_SAFETY=off would
# silently keep later safe-default agents unguarded.
os.environ.pop("PRAISONAI_TOOL_SAFETY", None)
# Same reasoning for the registry: a bypass backend installed by an
# earlier --no-safe run in this process must not survive into a
# safe-by-default one. Only a backend *we* installed is removed, so a
# caller-supplied backend is never clobbered.
_clear_bypass_approval_backend()
# Headless one-shot: emit a clean, machine-readable envelope and exit with a
# status-reflecting code. Routes around the decorated interactive chat path
@@ -427,7 +292,6 @@ def code_main(
tools_arg=tools,
agent_profile=agent_profile,
approval=headless_approval,
max_steps=max_steps,
)
return
@@ -440,7 +304,6 @@ def code_main(
verbose=verbose,
profile_deep=profile_deep,
thinking_budget=thinking_budget,
max_steps=max_steps,
)
return
@@ -461,9 +324,6 @@ def code_main(
# profile (tools + permission/mode scope), consumed when the agent is built.
args.thinking_budget = thinking_budget
args.agent_profile = agent_profile
# Same coding-sized budget for the interactive session, threaded onto the
# resident TUI's existing `execution` capability option.
args.execution = build_code_execution_config(max_steps)
# Resolve the named profile's permission/mode scope into an approval config
# so the code session runs least-privilege (e.g. `--agent plan` is rejected
@@ -570,7 +430,6 @@ def _run_resident_code(prompt, args, *, plan=False, session_id=None):
plan_mode=plan,
enable_acp=not getattr(args, "no_acp", False),
enable_lsp=not getattr(args, "no_lsp", False),
execution=getattr(args, "execution", None),
)
tui = AsyncTUI(config=tui_config)
@@ -600,19 +459,6 @@ def _print_result_succeeded(result) -> bool:
return True
def _print_result_truncated(agent) -> bool:
"""Whether the headless run stopped by exhausting its step/tool budget.
Delegates to ``run._run_was_truncated`` so ``code -p`` and ``run`` share one
definition of "truncated" (the core records ``last_stop_reason ==
"max_steps"`` on the agent when either tool loop runs out of budget) instead
of growing a second, drifting guard.
"""
from praisonai_code.cli.commands.run import _run_was_truncated
return _run_was_truncated(agent)
def _collect_print_usage(model: Optional[str]) -> dict:
"""Return this run's ``{in, out, cost}`` usage from the token collector.
@@ -666,17 +512,14 @@ def _run_print_code(
tools_arg: Optional[str] = None,
agent_profile: Optional[dict] = None,
approval: Optional[Any] = None,
max_steps: Optional[int] = None,
):
"""Headless one-shot code run with clean stdout and a status exit code.
Builds the code agent directly (bypassing the decorated interactive path),
runs the prompt once, and emits a machine-readable envelope
``{result, session_id, usage:{in,out,cost}, status}`` to stdout. Exit code
is 0 on success, 2 when the run was *truncated* by the step/tool budget
(``status: "truncated"`` — a wrap-up summary, not a finished task), and 1 on
failure (empty result or raised error), giving scripts/CI/benchmarks a
reliable signal — parity with ``run --output json``.
is 0 on success and 1 on failure (empty result or raised error), giving
scripts/CI/benchmarks a reliable signal — parity with ``run --output json``.
``tools_arg`` is the raw ``--tools`` string, resolved *inside* this run's
try/except (via the same ``_resolve_tools_arg``/``ToolResolver`` as the
@@ -730,12 +573,6 @@ def _run_print_code(
# decorations so stdout carries only our envelope.
"output": "minimal",
}
# Give the coding agent a coding-sized step budget. Without this it inherits
# the general-purpose Agent defaults (20 steps / 10 tool calls per turn) and
# truncates before it can reach a test run — see DEFAULT_CODE_MAX_STEPS.
_execution = build_code_execution_config(max_steps)
if _execution is not None:
agent_config["execution"] = _execution
if model:
agent_config["llm"] = model
@@ -777,7 +614,6 @@ def _run_print_code(
status = "ok"
result = None
error_message = None
agent = None
try:
workspace = os.environ.get("PRAISONAI_WORKSPACE") or os.getcwd()
merged_tools = _get_headless_code_tools(
@@ -837,12 +673,6 @@ def _run_print_code(
if status != "error" and not _print_result_succeeded(result):
status = "failed"
elif status == "ok" and _print_result_truncated(agent):
# A step/tool-budget exhaustion still returns a non-empty wrap-up
# summary, so string-emptiness alone reports a *truncated* run as a
# completed one. Reuse `run`'s terminal-reason check so both headless
# surfaces agree, and expose it as a distinct status + exit code.
status = "truncated"
usage = _collect_print_usage(model)
@@ -859,13 +689,6 @@ def _run_print_code(
# exit code — nothing else — so it pipes cleanly.
if result:
print(result)
if status == "truncated":
typer.echo(
"Warning: run hit the step/tool-call budget; the output above "
"is a summary of partial progress, not a completed task. "
"Re-run with a higher --max-steps.",
err=True,
)
if error_message:
typer.echo(f"Error: {error_message}", err=True)
else:
@@ -879,10 +702,6 @@ def _run_print_code(
envelope["error"] = error_message
print(json.dumps(envelope))
if status == "truncated":
# Exit 2 = incomplete run (mirrors `run`'s truncation contract), distinct
# from a hard failure (1) and a clean completion (0).
raise typer.Exit(2)
if status != "ok":
raise typer.Exit(1)
@@ -900,7 +719,6 @@ def _run_profiled_code(
verbose: bool = False,
profile_deep: bool = False,
thinking_budget: Optional[int] = None,
max_steps: Optional[int] = None,
):
"""Run code assistant with profiling enabled."""
from praisonai_code.cli.features.cli_profiler import (
@@ -936,12 +754,6 @@ def _run_profiled_code(
# keeps profiler stdout clean.
"output": "verbose" if verbose else "minimal",
}
# Same coding-sized budget as the headless/interactive paths, so a profiled
# single-prompt run does not silently keep the general-purpose core defaults
# (20 steps / 10 tool calls per turn) and truncate a real coding task.
_execution = build_code_execution_config(max_steps)
if _execution is not None:
agent_config["execution"] = _execution
if model:
agent_config["llm"] = model
@@ -2,12 +2,6 @@
Hooks command group for PraisonAI CLI.
Provides hook management commands.
Only actions the legacy handler actually implements are registered here.
``add`` and ``remove`` used to be advertised but were never implemented: the
handler printed "Unknown hooks action" and the dispatcher still exited 0, so a
script that "added" a hook succeeded while doing nothing. Hooks are configured
by editing the file that ``praisonai hooks init`` creates.
"""
import typer
@@ -17,23 +11,34 @@ app = typer.Typer(help="Hook management")
@app.command("list")
def hooks_list():
"""List configured hooks."""
"""List available hooks."""
from praisonai_code._wrapper_bridge import run_wrapper_command
run_wrapper_command(['hooks', 'list'], feature="hooks")
argv = ['hooks', 'list']
run_wrapper_command(argv, feature="hooks")
@app.command("stats")
def hooks_stats():
"""Show hooks statistics."""
@app.command("add")
def hooks_add(
name: str = typer.Argument(..., help="Hook name"),
event: str = typer.Option(..., "--event", "-e", help="Event to hook into"),
):
"""Add a hook."""
from praisonai_code._wrapper_bridge import run_wrapper_command
run_wrapper_command(['hooks', 'stats'], feature="hooks")
argv = ['hooks', 'add', name, '--event', event]
run_wrapper_command(argv, feature="hooks")
@app.command("init")
def hooks_init():
"""Create the hooks.json template in this workspace."""
@app.command("remove")
def hooks_remove(
name: str = typer.Argument(..., help="Hook name to remove"),
):
"""Remove a hook."""
from praisonai_code._wrapper_bridge import run_wrapper_command
run_wrapper_command(['hooks', 'init'], feature="hooks")
argv = ['hooks', 'remove', name]
run_wrapper_command(argv, feature="hooks")
@@ -37,26 +37,6 @@ def _resolve_installer(global_env: bool = False):
return [sys.executable, "-m", "pip", "install"]
def _plugin_create_command_name():
"""Name of the registered command that creates a single-file plugin.
Derived from this Typer app's own command registry so a hint printed to the
user can never advertise a subcommand that is not registered (issue: the
``discover`` empty-state used to point at a non-existent ``plugins init``).
"""
registered = {
getattr(cmd, "name", None) or getattr(cmd, "callback", None).__name__
for cmd in app.registered_commands
}
for candidate in ("create", "init", "new"):
if candidate in registered:
return candidate
raise RuntimeError(
"plugins app registers no plugin-creation command; "
f"registered commands: {sorted(n for n in registered if n)}"
)
def _registered_entry_point_names():
"""Return the set of currently registered entry-point plugin names."""
try:
@@ -646,10 +626,7 @@ def plugins_discover(
if not plugins:
console.print("[yellow]No single-file plugins found.[/yellow]")
console.print(
"\nCreate one with: praisonai plugins "
f"{_plugin_create_command_name()} <name>"
)
console.print("\nCreate one with: praisonai plugins init <name>")
return
table = Table(title=f"Single-File Plugins ({len(plugins)} found)")
@@ -164,9 +164,8 @@ def _report_run_truncated(output: Any, result: Any) -> None:
if not getattr(output, "is_json_mode", False):
output.print_warning(
"Run hit the step/iteration limit; the answer above is a summary of "
"partial progress, not a completed task. Raise the budget with "
"ExecutionConfig(max_steps=...) on the agent (or `execution: "
"{max_steps: ...}` in the YAML) and re-run."
"partial progress, not a completed task. Re-run with a higher "
"--max-iter/max_steps to finish."
)
raise typer.Exit(2)
@@ -82,30 +82,6 @@ def _get_call_module():
call_module = getattr(_mod, 'call')
return call_module
def _build_call_args(args, argv=None):
"""Build the ``praisonai.api.call.main()`` argv from parsed CLI args.
``--port``/``--host`` are forwarded only when the user typed them. The
call server's own defaults (``$PORT``, else 8090 / 127.0.0.1) differ from
the shared argparse defaults (8005 / 127.0.0.1), so always forwarding
``args.port`` would silently move the server off its documented port.
"""
raw = sys.argv[1:] if argv is None else list(argv)
def _typed(flag):
return any(a == flag or a.startswith(flag + "=") for a in raw)
call_args = []
if getattr(args, "public", False):
call_args.append("--public")
if _typed("--port") and getattr(args, "port", None) is not None:
call_args.extend(["--port", str(args.port)])
if _typed("--host") and getattr(args, "host", None) is not None:
call_args.extend(["--host", str(args.host)])
return call_args
def _get_gradio():
"""Lazy import gradio only when gradio UI is used.
@@ -672,7 +648,10 @@ class PraisonAI:
print("[red]ERROR: Call feature is not installed. Install with:[/red]")
print("\npip install \"praisonai[call]\"\n")
return
_get_call_module().main(_build_call_args(args))
call_args = []
if args.public:
call_args.append('--public')
_get_call_module().main(call_args)
return
if args.command == 'train':
@@ -1079,7 +1058,9 @@ class PraisonAI:
print("\npip install \"praisonai[call]\"\n")
sys.exit(1)
call_args = _build_call_args(args)
call_args = []
if args.public:
call_args.append('--public')
_get_call_module().main(call_args)
sys.exit(0)
@@ -1208,9 +1189,8 @@ class PraisonAI:
# Get action from remaining args
action = unknown_args[0] if unknown_args else 'list'
# Propagate the handler's exit code: an unknown action used to
# print an error and still exit 0, so scripts saw success.
sys.exit(self.handle_hooks_command(action) or 0)
self.handle_hooks_command(action)
sys.exit(0)
elif args.command == 'knowledge':
self._require_agents()
@@ -1,344 +0,0 @@
"""Regression gates for four CLI surface defects.
Each test pins one defect and fails against the code as it stood before the
accompanying fix:
1. ``praisonai hooks add`` printed "Unknown hooks action: add" and still exited
``0``, so a script that added a hook succeeded while doing nothing. The
hooks Typer group also advertised ``add``/``remove`` (never implemented)
while omitting ``init``/``stats`` (which work).
2. ``praisonai hooks list`` told users to create ``.praison/hooks.json`` while
``HooksManager.CONFIG_FILE`` is ``.praisonai/hooks.json``; ``hooks init``
wrote that same wrong path, so init-then-list never saw the hooks.
3. ``praisonai plugins discover``'s empty state said "Create one with:
praisonai plugins init <name>". There is no ``init`` command; it is
``create``.
4. ``praisonai call`` accepted only ``--model``/``--verbose``, so ``--public``
(the ngrok exposure inbound PSTN requires) and ``--host`` were reachable
only through environment variables.
Expected values are derived from the real source of truth --
``HooksManager.CONFIG_FILE`` and the Typer command registries -- rather than
restating a corrected literal, so they cannot drift the way the bugs did.
Note on shape: the legacy argparse layer deliberately parses an *empty* argv
under pytest (``build_argument_parser(in_test_env)``), so a CliRunner
invocation cannot exercise the legacy leg in-process. The delegation chain is
therefore pinned seam by seam, plus one real subprocess that reads the process
exit code directly.
"""
import argparse
import os
import subprocess
import sys
import pytest
from typer.testing import CliRunner
from praisonaiagents.memory import HooksManager
import praisonai_code._wrapper_bridge as wrapper_bridge
import praisonai_code.cli.commands.call as call_command
import praisonai_code.cli.commands.hooks as hooks_command
import praisonai_code.cli.commands.plugins as plugins_command
from praisonai_code.cli.legacy import praison_ai as legacy
runner = CliRunner()
def _registered_names(app):
return [cmd.name or cmd.callback.__name__ for cmd in app.registered_commands]
def _hooks_handler():
from praisonai.cli.legacy.subcommand_handlers import handle_hooks_command
return handle_hooks_command
@pytest.fixture()
def wrapper_argv(monkeypatch):
"""Capture the legacy argv a Typer stub hands to the wrapper."""
calls = []
monkeypatch.setattr(
wrapper_bridge, "run_wrapper_command", lambda argv, *, feature: calls.append(list(argv))
)
return calls
# --------------------------------------------------------------------------
# Defect 1: an action that fails must not exit 0
# --------------------------------------------------------------------------
def test_unknown_hooks_action_returns_nonzero(tmp_path, monkeypatch):
"""The handler reports failure as an exit code, not just as printed text.
``None`` is not acceptable: the dispatcher does ``sys.exit(code or 0)``, so
an implicit ``None`` return is exactly the original "exits 0" defect. The
assertion therefore demands a real non-zero int.
"""
monkeypatch.chdir(tmp_path)
handler = _hooks_handler()
for action in ("add", "remove", "definitely-not-an-action"):
code = handler(None, action)
assert isinstance(code, int) and code != 0, (
f"hooks action {action!r} returned {code!r}; the dispatcher's "
f"sys.exit({code!r} or 0) would report success"
)
def test_known_hooks_actions_return_zero(tmp_path, monkeypatch):
"""The success path still reports success."""
monkeypatch.chdir(tmp_path)
handler = _hooks_handler()
for action in ("list", "stats", "init", "help"):
assert handler(None, action) == 0, action
def test_hooks_dispatch_propagates_handler_exit_code():
"""The legacy dispatcher must exit with the handler's code, not a fixed 0.
Pins the ``sys.exit(0)`` one level above the handler, so fixing only the
handler cannot leave the defect in place.
"""
import inspect
source = inspect.getsource(legacy)
marker = "elif args.command == 'hooks':"
assert marker in source, "hooks dispatch branch not found"
branch = source.split(marker, 1)[1].split("elif args.command ==", 1)[0]
assert "handle_hooks_command" in branch
assert "sys.exit(0)" not in branch, (
"hooks dispatch still exits 0 unconditionally; a failed action would "
f"report success. Branch:\n{branch}"
)
def test_hooks_group_advertises_only_actions_the_handler_accepts(tmp_path, monkeypatch, wrapper_argv):
"""Every registered hooks subcommand must map to an action that succeeds.
``add``/``remove`` were registered but unimplemented. This goes red if
either comes back, and equally if a registered command stops working.
"""
names = _registered_names(hooks_command.app)
assert names, "hooks group registers no commands"
for name in names:
wrapper_argv.clear()
result = runner.invoke(hooks_command.app, [name])
assert result.exit_code == 0, f"{name}: {result.output}"
assert wrapper_argv, f"'hooks {name}' dispatched nothing"
argv = wrapper_argv[0]
assert argv[0] == "hooks"
action = argv[1]
monkeypatch.chdir(tmp_path)
assert _hooks_handler()(None, action) == 0, (
f"'praisonai hooks {name}' is advertised but the handler rejects "
f"action {action!r}"
)
def test_hooks_add_exits_nonzero_end_to_end(tmp_path):
"""Read the real process exit code, not a message.
This is the user-visible contract: ``praisonai hooks add`` must not report
success while doing nothing.
"""
pytest.importorskip("praisonai")
proc = subprocess.run(
[sys.executable, "-m", "praisonai", "hooks", "add", "myhook", "--event", "BEFORE_TOOL"],
cwd=str(tmp_path),
capture_output=True,
text=True,
timeout=180,
)
assert proc.returncode != 0, (
"'praisonai hooks add' exited 0 while adding nothing.\n"
f"stdout: {proc.stdout}\nstderr: {proc.stderr}"
)
# --------------------------------------------------------------------------
# Defect 2: the advertised hooks path must be the one that is read
# --------------------------------------------------------------------------
def test_no_hooks_message_names_the_real_config_file(tmp_path, monkeypatch, capsys):
monkeypatch.chdir(tmp_path)
assert _hooks_handler()(None, "list") == 0
out = capsys.readouterr().out
assert HooksManager.CONFIG_FILE in out, (
f"'hooks list' advertises a path that is not "
f"{HooksManager.CONFIG_FILE!r}: {out!r}"
)
def test_hooks_init_writes_the_file_the_manager_loads(tmp_path, monkeypatch):
"""init -> list must round-trip: init wrote a path list never read."""
monkeypatch.chdir(tmp_path)
assert _hooks_handler()(None, "init") == 0
expected = tmp_path / HooksManager.CONFIG_FILE.replace("/", os.sep)
assert expected.is_file(), (
f"hooks init did not create {expected}; created instead: "
f"{sorted(str(p.relative_to(tmp_path)) for p in tmp_path.rglob('hooks.json'))}"
)
manager = HooksManager(workspace_path=str(tmp_path))
assert manager.get_stats().get("total_hooks", 0) > 0, (
"hooks written by 'hooks init' are invisible to HooksManager"
)
# --------------------------------------------------------------------------
# Defect 3: the plugin-creation hint must name a registered command
# --------------------------------------------------------------------------
def test_plugin_create_hint_names_a_registered_command():
advertised = plugins_command._plugin_create_command_name()
assert advertised in _registered_names(plugins_command.app), (
f"'praisonai plugins {advertised}' is advertised but not registered; "
f"registered: {_registered_names(plugins_command.app)}"
)
def test_plugins_discover_empty_state_hint_resolves(monkeypatch):
"""The empty-state hint must name a command the plugins app really has."""
from praisonaiagents.plugins import discovery
monkeypatch.setattr(discovery, "discover_plugins", lambda *a, **k: [])
result = runner.invoke(plugins_command.app, ["discover"])
assert result.exit_code == 0, result.output
hint_line = next(
(line for line in result.output.splitlines() if "Create one with:" in line),
None,
)
assert hint_line is not None, result.output
advertised = hint_line.split("praisonai plugins", 1)[1].split()[0]
assert advertised in _registered_names(plugins_command.app), (
f"empty-state hint advertises 'praisonai plugins {advertised}', which "
f"is not registered: {_registered_names(plugins_command.app)}"
)
# --------------------------------------------------------------------------
# Defect 4: --public/--port/--host must reach the call server
# --------------------------------------------------------------------------
def _run_server_capture(monkeypatch):
call_mod = pytest.importorskip("praisonai.api.call")
captured = {}
def fake_run_server(port, host="127.0.0.1", use_public=False):
captured.update(port=port, host=host, use_public=use_public)
monkeypatch.setattr(call_mod, "run_server", fake_run_server)
return call_mod, captured
@pytest.mark.parametrize(
"cli_args, expected_call_argv",
[
([], []),
(["--public"], ["--public"]),
(["--port", "9111"], ["--port", "9111"]),
(["--host", "0.0.0.0"], ["--host", "0.0.0.0"]),
(
["--public", "--host", "0.0.0.0", "--port", "9111"],
["--public", "--port", "9111", "--host", "0.0.0.0"],
),
],
)
def test_call_command_forwards_server_options(cli_args, expected_call_argv, monkeypatch, wrapper_argv):
"""Leg 1: the Typer command must accept and forward the server options."""
result = runner.invoke(call_command.app, cli_args)
assert result.exit_code == 0, result.output
assert wrapper_argv, "call command dispatched nothing"
legacy_argv = wrapper_argv[0]
assert legacy_argv[0] == "call"
# Leg 2: the legacy dispatcher maps its parsed args onto the call server's
# own argv. Reconstruct the args the legacy parser would produce.
parsed = argparse.Namespace(
public="--public" in legacy_argv,
port=int(legacy_argv[legacy_argv.index("--port") + 1]) if "--port" in legacy_argv else 8005,
host=legacy_argv[legacy_argv.index("--host") + 1] if "--host" in legacy_argv else "127.0.0.1",
)
assert legacy._build_call_args(parsed, argv=legacy_argv) == expected_call_argv
@pytest.mark.parametrize(
"cli_args, expected",
[
([], {"port": 8090, "host": "127.0.0.1", "use_public": False}),
(["--public"], {"port": 8090, "host": "127.0.0.1", "use_public": True}),
(["--port", "9111"], {"port": 9111, "host": "127.0.0.1", "use_public": False}),
(
["--host", "0.0.0.0", "--port", "9111"],
{"port": 9111, "host": "0.0.0.0", "use_public": False},
),
],
)
def test_call_options_reach_run_server(cli_args, expected, monkeypatch, wrapper_argv):
"""Full chain: Typer options -> legacy argv -> api.call.main -> run_server.
Plain ``praisonai call`` must keep the call server's own default port
(``$PORT``, else 8090); the shared argparse ``--port`` default is 8005, so
forwarding it unconditionally would silently retarget the port.
"""
monkeypatch.delenv("PORT", raising=False)
call_mod, captured = _run_server_capture(monkeypatch)
assert runner.invoke(call_command.app, cli_args).exit_code == 0
legacy_argv = wrapper_argv[0]
parsed = argparse.Namespace(
public="--public" in legacy_argv,
port=int(legacy_argv[legacy_argv.index("--port") + 1]) if "--port" in legacy_argv else 8005,
host=legacy_argv[legacy_argv.index("--host") + 1] if "--host" in legacy_argv else "127.0.0.1",
)
call_mod.main(legacy._build_call_args(parsed, argv=legacy_argv))
assert captured == expected
def test_public_bind_disables_the_auth_escape_hatch(monkeypatch):
"""--public must not open an unauthenticated surface.
``run_server`` forces ``0.0.0.0`` under ``--public`` and publishes it as
``PRAISONAI_CALL_BIND_HOST``; the ``PRAISONAI_CALL_AUTH=disabled`` escape
hatch is honoured only for a loopback bind, so exposing the server
publicly cannot also switch authentication off.
"""
import asyncio
fastapi = pytest.importorskip("fastapi")
from praisonai.api import agent_invoke
monkeypatch.setenv("PRAISONAI_CALL_AUTH", "disabled")
monkeypatch.setenv("PRAISONAI_CALL_BIND_HOST", "0.0.0.0")
with pytest.raises(fastapi.HTTPException) as exc:
asyncio.run(agent_invoke.verify_token(request=None, authorization=None))
assert exc.value.status_code == 503
def test_run_server_publishes_the_public_bind_host(monkeypatch):
"""The bind host the auth guard reads must reflect the --public override."""
call_mod = pytest.importorskip("praisonai.api.call")
monkeypatch.setattr(call_mod, "OPENAI_API_KEY", "sk-test")
monkeypatch.setattr(call_mod, "setup_public_url", lambda port: "https://example.ngrok.app")
monkeypatch.delenv("PRAISONAI_CALL_BIND_HOST", raising=False)
seen = {}
fake_uvicorn = type("_U", (), {"run": staticmethod(lambda app, **kw: seen.update(kw))})
monkeypatch.setitem(sys.modules, "uvicorn", fake_uvicorn)
monkeypatch.setattr(call_mod, "build_call_app", lambda **kw: object())
# A user asking for --public with a loopback --host must still end up bound
# (and reported) as 0.0.0.0, so PRAISONAI_CALL_AUTH=disabled stays refused.
call_mod.run_server(port=9111, host="127.0.0.1", use_public=True)
assert seen["host"] == "0.0.0.0"
assert os.environ["PRAISONAI_CALL_BIND_HOST"] == "0.0.0.0"
@@ -1,524 +0,0 @@
"""Regression tests for the four linked defects that stopped `praisonai code`
completing a coding task, and that stopped anyone noticing (#3932).
(a) The coding agent inherited the *general-purpose* Agent budget
(``ExecutionConfig.max_iter=20`` / ``max_tool_calls_per_turn=10``). Ten tool
calls is "list files, read two files, one grep" — it cannot reach a test
run, and `praisonai code` offered no flag, config key or env var to raise
it.
(b) A budget-truncated run still returns a non-empty wrap-up summary, and
``code -p`` classified any non-empty string as success: ``status: "ok"``,
exit 0. A truncated run was indistinguishable from a completed one to a
human or a CI adapter. These tests assert the **exit code**, not the message
text.
(c) ``--dangerously-skip-approval`` set two env vars that the gate which
actually prompts (the core ``@require_approval`` decorator, which resolves
through the approval *registry* backend) never reads. With stdin at
/dev/null every prompt was auto-denied, so the agent could not run a single
shell command — while still exiting 0.
(d) The Terminal-Bench smoke gate that would have caught all of this pinned
Python 3.11; ``harbor`` requires >=3.12, so every scheduled run died in
~15s at ``pip install harbor`` and no score was ever produced.
"""
import json
import os
import re
from pathlib import Path
import pytest
from typer.testing import CliRunner
from praisonai_code.cli.commands import code as code_module
from praisonai_code.cli.commands.code import (
DEFAULT_CODE_MAX_STEPS,
app,
build_code_execution_config,
resolve_code_max_steps,
)
@pytest.fixture(autouse=True)
def _satisfy_credential_gate(monkeypatch):
"""Seed a credential so the first-run onboarding gate lets the run reach the
Agent path under test (mirrors ``test_code_print_json.py``)."""
monkeypatch.setenv("OPENAI_API_KEY", "sk-test-dummy")
@pytest.fixture(autouse=True)
def _isolate_approval_registry():
"""Restore the process-global approval backend around every test.
The registry is a singleton; a test that installs a bypass backend must not
leak it into another test (that leak is precisely the hazard (c)'s cleanup
branch guards against).
"""
from praisonaiagents.approval import get_approval_registry
registry = get_approval_registry()
saved = registry._global_backend
try:
yield registry
finally:
registry._global_backend = saved
class _StubAgent:
"""Minimal Agent stand-in that records its construction kwargs.
``stop_reason`` drives the truncation contract: the core records
``last_stop_reason == "max_steps"`` on the agent when either tool loop runs
out of budget.
"""
last_kwargs: dict = {}
answer = "Here is the finished patch."
stop_reason = "completed"
def __init__(self, *_args, **kwargs):
type(self).last_kwargs = kwargs
self.last_stop_reason = type(self).stop_reason
def start(self, *_args, **_kwargs):
return type(self).answer
def _install_stub_agent(monkeypatch, *, answer=None, stop_reason="completed"):
import praisonaiagents
class _Agent(_StubAgent):
pass
_Agent.answer = answer if answer is not None else _StubAgent.answer
_Agent.stop_reason = stop_reason
monkeypatch.setattr(praisonaiagents, "Agent", _Agent, raising=False)
return _Agent
# ---------------------------------------------------------------------------
# (b) a truncated run must not report success
# ---------------------------------------------------------------------------
# The real wrap-up text a budget-exhausted LiteLLM turn returns. It is a
# perfectly non-empty string, which is exactly why emptiness alone could not
# tell truncation from completion.
TRUNCATION_SUMMARY = (
"Tool call limit reached (10 calls). Task may be too complex or there "
"may be a broken tool causing repeated calls."
)
def test_truncated_headless_run_exits_non_zero(monkeypatch):
"""A budget-truncated `code -p` run must NOT exit 0.
Asserts the exit code, not the message: a CI adapter branches on the status
code, and before the fix this run was exit 0 / ``status: "ok"``.
"""
_install_stub_agent(
monkeypatch, answer=TRUNCATION_SUMMARY, stop_reason="max_steps"
)
runner = CliRunner()
result = runner.invoke(
app, ["-p", "--output", "json", "Build the project and make the tests pass"]
)
assert result.exit_code != 0, result.output
# Exit 2 = incomplete run, distinct from a hard failure (1) and success (0),
# matching `praisonai run`'s existing truncation contract.
assert result.exit_code == 2, result.output
def test_truncated_headless_run_reports_a_distinguishable_status(monkeypatch):
"""The JSON envelope must let a consumer tell truncation from completion."""
_install_stub_agent(
monkeypatch, answer=TRUNCATION_SUMMARY, stop_reason="max_steps"
)
runner = CliRunner()
result = runner.invoke(
app, ["-p", "--output", "json", "Build the project and make the tests pass"]
)
payload = json.loads(result.stdout.strip().splitlines()[-1])
assert payload["status"] == "truncated"
# The partial wrap-up is still preserved for the caller.
assert payload["result"] == TRUNCATION_SUMMARY
def test_truncated_text_mode_warns_on_stderr_and_exits_non_zero(monkeypatch):
_install_stub_agent(
monkeypatch, answer=TRUNCATION_SUMMARY, stop_reason="max_steps"
)
runner = CliRunner()
result = runner.invoke(app, ["-p", "--output", "text", "Fix the build"])
assert result.exit_code == 2, result.output
assert "budget" in result.output.lower()
def test_completed_headless_run_still_exits_zero(monkeypatch):
"""Control: the completed/failed contract is unchanged."""
_install_stub_agent(monkeypatch, answer="All tests pass.", stop_reason="completed")
runner = CliRunner()
result = runner.invoke(app, ["-p", "--output", "json", "Fix the build"])
assert result.exit_code == 0, result.output
payload = json.loads(result.stdout.strip().splitlines()[-1])
assert payload["status"] == "ok"
def test_truncation_check_is_shared_with_run(monkeypatch):
"""`code -p` reuses `run`'s terminal-reason check rather than a second one."""
from praisonai_code.cli.commands.run import _run_was_truncated
calls = []
def _spy(agent):
calls.append(agent)
return False
monkeypatch.setattr(
"praisonai_code.cli.commands.run._run_was_truncated", _spy, raising=True
)
_install_stub_agent(monkeypatch, answer="done", stop_reason="completed")
runner = CliRunner()
runner.invoke(app, ["-p", "--output", "json", "Fix the build"])
assert calls, "code -p did not consult run._run_was_truncated"
# Sanity-check the shared helper's own contract while we are here.
assert _run_was_truncated(type("A", (), {"last_stop_reason": "max_steps"})()) is True
def test_run_truncation_hint_does_not_name_a_nonexistent_flag():
"""`run`'s truncation remediation told users to use `--max-iter`, which the
CLI has never had (0 occurrences as an option; control: `--model` = 56)."""
source = Path(code_module.__file__).with_name("run.py").read_text(encoding="utf-8")
assert "--max-iter" not in source
# ---------------------------------------------------------------------------
# (c) --dangerously-skip-approval must actually skip approval
# ---------------------------------------------------------------------------
def _global_backend_is_bypass(registry) -> bool:
from praisonaiagents.approval.backends import AutoApproveBackend
return isinstance(registry._global_backend, AutoApproveBackend)
def test_dangerously_skip_approval_installs_a_bypass_backend(
monkeypatch, _isolate_approval_registry
):
"""The flag must reach the gate that actually prompts.
``PRAISON_APPROVAL_MODE``/``PRAISONAI_TOOL_SAFETY`` are read by the CLI's
own tool wiring; the core ``@require_approval`` decorator resolves through
the approval registry's backend. Before the fix nothing registered a bypass
there, so every critical tool still prompted — and was auto-denied in a
non-TTY run.
"""
registry = _isolate_approval_registry
registry._global_backend = None
_install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
result = runner.invoke(
app,
["-p", "--output", "json", "--dangerously-skip-approval", "Fix the build"],
)
assert result.exit_code == 0, result.output
assert _global_backend_is_bypass(registry), (
"--dangerously-skip-approval left the approval gate in place; a "
"non-TTY run auto-denies every prompt"
)
def test_no_safe_also_installs_a_bypass_backend(monkeypatch, _isolate_approval_registry):
registry = _isolate_approval_registry
registry._global_backend = None
_install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
runner.invoke(app, ["-p", "--output", "json", "--no-safe", "Fix the build"])
assert _global_backend_is_bypass(registry)
def test_safe_by_default_never_installs_a_bypass_backend(
monkeypatch, _isolate_approval_registry
):
"""Safety guard: absent the opt-in flag, behaviour is unchanged."""
registry = _isolate_approval_registry
registry._global_backend = None
_install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
runner.invoke(app, ["-p", "--output", "json", "Fix the build"])
assert not _global_backend_is_bypass(registry)
assert registry._global_backend is None
def test_safe_mode_clears_a_bypass_left_by_an_earlier_run(
monkeypatch, _isolate_approval_registry
):
"""A bypass from an earlier --no-safe call in the same process (REPL/worker)
must not silently keep a later safe-by-default session unguarded."""
from praisonaiagents.approval.backends import AutoApproveBackend
registry = _isolate_approval_registry
registry._global_backend = AutoApproveBackend()
_install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
runner.invoke(app, ["-p", "--output", "json", "Fix the build"])
assert not _global_backend_is_bypass(registry)
def test_safe_mode_preserves_a_caller_supplied_backend(
monkeypatch, _isolate_approval_registry
):
"""Cleanup removes only the bypass this module installs."""
class _CustomBackend:
def request_approval_sync(self, request): # pragma: no cover - not called
raise AssertionError("not expected")
registry = _isolate_approval_registry
custom = _CustomBackend()
registry._global_backend = custom
_install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
runner.invoke(app, ["-p", "--output", "json", "Fix the build"])
assert registry._global_backend is custom
# ---------------------------------------------------------------------------
# (a) the coding agent needs a coding-sized, configurable budget
# ---------------------------------------------------------------------------
# The general-purpose Agent defaults the code agent used to inherit.
_CORE_DEFAULT_MAX_STEPS = 20
_CORE_DEFAULT_TOOL_CALLS = 10
def test_headless_code_agent_gets_a_coding_sized_budget(monkeypatch):
agent_cls = _install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
result = runner.invoke(app, ["-p", "--output", "json", "Fix the build"])
assert result.exit_code == 0, result.output
execution = agent_cls.last_kwargs.get("execution")
assert execution is not None, "code -p built the agent with no ExecutionConfig"
assert execution.max_steps == DEFAULT_CODE_MAX_STEPS
assert execution.max_tool_calls_per_turn == DEFAULT_CODE_MAX_STEPS
# Both loops must be raised: the OpenAI-native loop is bounded by max_steps,
# the LiteLLM loop additionally by max_tool_calls_per_turn.
assert execution.max_steps > _CORE_DEFAULT_MAX_STEPS
assert execution.max_tool_calls_per_turn > _CORE_DEFAULT_TOOL_CALLS
def test_max_steps_flag_overrides_the_default(monkeypatch):
agent_cls = _install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
result = runner.invoke(
app, ["-p", "--output", "json", "--max-steps", "42", "Fix the build"]
)
assert result.exit_code == 0, result.output
execution = agent_cls.last_kwargs["execution"]
assert execution.max_steps == 42
assert execution.max_tool_calls_per_turn == 42
def test_max_steps_env_override(monkeypatch):
monkeypatch.setenv("PRAISONAI_CODE_MAX_STEPS", "77")
agent_cls = _install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
runner.invoke(app, ["-p", "--output", "json", "Fix the build"])
assert agent_cls.last_kwargs["execution"].max_steps == 77
def test_max_steps_flag_beats_env(monkeypatch):
monkeypatch.setenv("PRAISONAI_CODE_MAX_STEPS", "77")
agent_cls = _install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
runner.invoke(app, ["-p", "--output", "json", "--max-steps", "5", "Fix the build"])
assert agent_cls.last_kwargs["execution"].max_steps == 5
def test_max_steps_resolution_rejects_junk(monkeypatch):
monkeypatch.delenv("PRAISONAI_CODE_MAX_STEPS", raising=False)
assert resolve_code_max_steps(None) == DEFAULT_CODE_MAX_STEPS
assert resolve_code_max_steps(0) == DEFAULT_CODE_MAX_STEPS
assert resolve_code_max_steps(-3) == DEFAULT_CODE_MAX_STEPS
monkeypatch.setenv("PRAISONAI_CODE_MAX_STEPS", "not-a-number")
assert resolve_code_max_steps(None) == DEFAULT_CODE_MAX_STEPS
def test_max_steps_flag_is_declared_on_the_cli():
import inspect
callback = app.registered_callback.callback
names = set()
for param in inspect.signature(callback).parameters.values():
names.update(getattr(param.default, "param_decls", None) or [])
assert "--max-steps" in names
def test_build_code_execution_config_raises_both_knobs():
cfg = build_code_execution_config()
assert cfg.resolved_max_steps() == DEFAULT_CODE_MAX_STEPS
assert cfg.resolved_max_tool_calls() == DEFAULT_CODE_MAX_STEPS
def test_profiled_run_gets_the_coding_sized_budget(monkeypatch):
"""A profiled single-prompt run must not silently keep the small core
defaults: `--profile` built its Agent with no ExecutionConfig, so a real
coding task would still truncate at 20 steps / 10 tool calls per turn."""
agent_cls = _install_stub_agent(monkeypatch, answer="done")
# The profiler prints a report and exits 0; we only need the Agent kwargs.
monkeypatch.setattr(
"praisonai_code.cli.features.cli_profiler.CLIProfiler",
lambda *a, **k: _NullProfiler(),
raising=True,
)
runner = CliRunner()
result = runner.invoke(
app, ["--profile", "--max-steps", "55", "Fix the build"]
)
assert result.exit_code == 0, result.output
execution = agent_cls.last_kwargs.get("execution")
assert execution is not None, "--profile built the agent with no ExecutionConfig"
assert execution.max_steps == 55
assert execution.max_tool_calls_per_turn == 55
class _NullProfiler:
"""No-op profiler so the --profile path exercises Agent construction only."""
def start(self): ...
def stop(self): ...
def mark_import_start(self): ...
def mark_import_end(self): ...
def mark_init_start(self): ...
def mark_init_end(self): ...
def mark_exec_start(self): ...
def mark_exec_end(self): ...
def print_report(self): ...
def test_interactive_dispatch_threads_the_budget_onto_args(monkeypatch):
"""The interactive/single-prompt dispatch must carry the coding budget on
``args.execution`` so the wrapper-legacy and resident TUI paths both apply
it — otherwise `--max-steps` is silently dropped for the common
`pip install praisonai` interactive session."""
captured = {}
def _fake_resident(prompt, args, *, plan=False, session_id=None):
captured["execution"] = getattr(args, "execution", None)
# Force the resident-TUI branch (wrapper absent) and capture the args.
monkeypatch.setattr(
"praisonai_code._wrapper_bridge.wrapper_available",
lambda: False,
raising=True,
)
monkeypatch.setattr(code_module, "_run_resident_code", _fake_resident, raising=True)
runner = CliRunner()
result = runner.invoke(app, ["--max-steps", "63", "Fix the build"])
assert result.exit_code == 0, result.output
execution = captured.get("execution")
assert execution is not None, "interactive dispatch dropped the execution budget"
assert execution.max_steps == 63
assert execution.max_tool_calls_per_turn == 63
# ---------------------------------------------------------------------------
# (d) the smoke gate must be able to install its harness
# ---------------------------------------------------------------------------
def _repo_root():
for parent in Path(__file__).resolve().parents:
if (parent / ".github" / "workflows").is_dir():
return parent
return None
def test_terminal_bench_smoke_gate_can_install_harbor():
"""`harbor` requires Python >=3.12. Pinned at 3.11 the gate died in ~15s at
`pip install harbor` on every scheduled run since it was added, so no
Terminal-Bench score has ever been produced (RESULTS.md: _pending first
run_)."""
root = _repo_root()
if root is None: # pragma: no cover - installed-package test run
pytest.skip("repository checkout not available")
workflow = root / ".github" / "workflows" / "terminal-bench-smoke.yml"
if not workflow.exists(): # pragma: no cover
pytest.skip("terminal-bench-smoke workflow not present")
text = workflow.read_text(encoding="utf-8")
match = re.search(r'python-version:\s*"?([0-9]+)\.([0-9]+)"?', text)
assert match, "no python-version pin found in the smoke workflow"
major, minor = int(match.group(1)), int(match.group(2))
assert (major, minor) >= (3, 12), (
f"terminal-bench-smoke pins Python {major}.{minor}; harbor requires "
">=3.12, so `pip install harbor` resolves to no candidate and the gate "
"can never run"
)
# ---------------------------------------------------------------------------
# (c) safety: the bypass must not weaken any tighter policy
# ---------------------------------------------------------------------------
def test_plan_mode_cannot_be_combined_with_the_bypass_flags(
monkeypatch, _isolate_approval_registry
):
"""--plan (read-only) and the approval opt-outs stay mutually exclusive, so
a bypass backend can never be installed for a planning session."""
registry = _isolate_approval_registry
registry._global_backend = None
_install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
for flags in (["--plan", "--dangerously-skip-approval"], ["--plan", "--no-safe"]):
result = runner.invoke(app, ["-p", "--output", "json", *flags, "Explore"])
assert result.exit_code == 1, result.output
assert not _global_backend_is_bypass(registry)
def test_plan_mode_still_installs_its_read_only_backend_on_the_agent(monkeypatch):
"""A per-agent backend (--plan / --agent scope) is what the core consults
first, so the bypass path cannot silently unlock a scoped session."""
agent_cls = _install_stub_agent(monkeypatch, answer="done")
runner = CliRunner()
result = runner.invoke(app, ["-p", "--output", "json", "--plan", "Explore"])
assert result.exit_code == 0, result.output
assert agent_cls.last_kwargs.get("approval") is not None
+2 -2
View File
@@ -40,7 +40,7 @@ This report compares **TypeScript/JavaScript SDK feature categories** against **
| ✅ Criteria | 1 | 1 | 162 |
| ✅ Database | 17 | 2 | 469 |
| ✅ Display | 22 | 1 | 342 |
| ✅ Embeddings | 13 | 3 | 898 |
| ✅ Embeddings | 13 | 3 | 841 |
| ✅ Evaluation | 10 | 4 | 866 |
| ✅ Events | 4 | 1 | 161 |
| ✅ Execution | 6 | 1 | 184 |
@@ -57,7 +57,7 @@ This report compares **TypeScript/JavaScript SDK feature categories** against **
| ✅ LLM | 29 | 2 | 279 |
| ✅ Loops | 11 | 1 | 241 |
| ✅ MCP | 21 | 4 | 870 |
| ✅ Memory | 24 | 4 | 840 |
| ✅ Memory | 24 | 4 | 783 |
| ✅ Middleware | 2 | 1 | 162 |
| ✅ OCR | 5 | 1 | 162 |
| ✅ Observability | 12 | 28 | 2318 |
+27 -66
View File
@@ -115,7 +115,7 @@
"version": "3.0.189",
"resolved": "https://registry.npmjs.org/@ai-sdk/gateway/-/gateway-3.0.189.tgz",
"integrity": "sha512-roPKojHenQm7U77kNNE0w586jP2vnjxaNx/+KFSQkJRLLAxrg7yUwZQfyOEizoTxdaltu+ZIlLRTw09X3XbGvQ==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/provider": "3.0.15",
@@ -133,7 +133,7 @@
"version": "4.0.50",
"resolved": "https://registry.npmjs.org/@ai-sdk/provider-utils/-/provider-utils-4.0.50.tgz",
"integrity": "sha512-YAcB+7M1JhAYsHorTrWyldCyZihjCKr/QRXH2vFrara/+lwqNE7q5KzoucKLZ7ktFiUonhnhFhRoiymsq/2K2Q==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/provider": "3.0.15",
@@ -224,7 +224,7 @@
"version": "3.0.15",
"resolved": "https://registry.npmjs.org/@ai-sdk/provider/-/provider-3.0.15.tgz",
"integrity": "sha512-XeZW1CcDF2GMbH4wejW6xBRI2QCOgnkVYUnxoeDadB1mf85riL2bMUeDoh+6gJ/r4mjNfzUPW8OjLjvwTP0u1Q==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"json-schema": "^0.4.0"
@@ -3002,7 +3002,7 @@
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.1.tgz",
"integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"engines": {
"node": ">=8.0.0"
@@ -3199,7 +3199,7 @@
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz",
"integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==",
"devOptional": true,
"dev": true,
"license": "MIT"
},
"node_modules/@tsconfig/node10": {
@@ -3648,7 +3648,7 @@
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/@vercel/oidc/-/oidc-3.2.0.tgz",
"integrity": "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"engines": {
"node": ">= 20"
@@ -3751,7 +3751,7 @@
"version": "6.0.277",
"resolved": "https://registry.npmjs.org/ai/-/ai-6.0.277.tgz",
"integrity": "sha512-fYlPj2QsisCH66SsEfWM8gV0ZR+6NLuRYSnVw3GKUKlvP+iIAUQR1bUJn1Y6lcAKeoEWQeZaann2ahXZY2HXtQ==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/gateway": "3.0.189",
@@ -3770,7 +3770,7 @@
"version": "4.0.50",
"resolved": "https://registry.npmjs.org/@ai-sdk/provider-utils/-/provider-utils-4.0.50.tgz",
"integrity": "sha512-YAcB+7M1JhAYsHorTrWyldCyZihjCKr/QRXH2vFrara/+lwqNE7q5KzoucKLZ7ktFiUonhnhFhRoiymsq/2K2Q==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@ai-sdk/provider": "3.0.15",
@@ -3919,19 +3919,6 @@
"node": ">= 8"
}
},
"node_modules/anymatch/node_modules/picomatch": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/arg": {
"version": "4.1.3",
"resolved": "https://registry.npmjs.org/arg/-/arg-4.1.3.tgz",
@@ -7492,19 +7479,6 @@
"node": "^14.15.0 || ^16.10.0 || >=18.0.0"
}
},
"node_modules/jest-util/node_modules/picomatch": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/jest-validate": {
"version": "29.7.0",
"resolved": "https://registry.npmjs.org/jest-validate/-/jest-validate-29.7.0.tgz",
@@ -7658,7 +7632,7 @@
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.4.0.tgz",
"integrity": "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA==",
"devOptional": true,
"dev": true,
"license": "(AFL-2.1 OR BSD-3-Clause)"
},
"node_modules/json-schema-ref-resolver": {
@@ -7962,19 +7936,6 @@
"node": ">=8.6"
}
},
"node_modules/micromatch/node_modules/picomatch": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/mime-db": {
"version": "1.54.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.54.0.tgz",
@@ -8570,13 +8531,13 @@
"license": "ISC"
},
"node_modules/picomatch": {
"version": "4.0.7",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.7.tgz",
"integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==",
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12"
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
@@ -8984,19 +8945,6 @@
"node": ">=8.10.0"
}
},
"node_modules/readdirp/node_modules/picomatch": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz",
"integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8.6"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/real-require": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/real-require/-/real-require-0.2.0.tgz",
@@ -9914,6 +9862,19 @@
"url": "https://github.com/sponsors/SuperchupuDev"
}
},
"node_modules/tinyglobby/node_modules/picomatch": {
"version": "4.0.7",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.7.tgz",
"integrity": "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/tmpl": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/tmpl/-/tmpl-1.0.5.tgz",
@@ -10306,7 +10267,7 @@
"version": "6.28.1",
"resolved": "https://registry.npmjs.org/undici/-/undici-6.28.1.tgz",
"integrity": "sha512-zWpdTVD54H48CIybL0rWQ3ukpb9d23wM7eH5RtfdmeP70cWHNjtfo7P4vZX+5CoDcO53J4Pu5uXp7lNfjc6DRA==",
"devOptional": true,
"dev": true,
"license": "MIT",
"engines": {
"node": ">=18.17"
+66 -679
View File
@@ -1,145 +1,10 @@
/**
* RealtimeAgent - Real-time voice/audio interaction agent
*
*
* Python parity with praisonaiagents/agent/realtime_agent.py
* Enables real-time voice conversations over a real WebSocket connection.
*
* ## Why the WebSocket is loaded the way it is
*
* This module is exported from the package root, so it has to stay loadable in
* a browser/webview bundle (see `scripts/webview-gate.mjs`). A static
* `import WebSocket from 'ws'` would put an unresolvable bare specifier on that
* bundle. So the implementation is resolved at call time:
*
* 1. an explicitly injected constructor (`{ webSocket }`), for tests and for
* runtimes that ship their own,
* 2. `globalThis.WebSocket` -- browsers, React Native, and Node >= 22,
* 3. the optional `ws` package, imported through a computed specifier so no
* bundler tries to resolve it statically. `ws` is NOT a dependency of this
* package; this branch only fires if a consumer installed it themselves,
* 4. otherwise a clear error naming all three ways to fix it. It never
* pretends to have connected.
* Enables real-time voice conversations using WebSocket connections.
*/
// ============================================================================
// WebSocket transport abstraction
// ============================================================================
/**
* The subset of the WebSocket API this agent uses.
*
* Both the WHATWG `WebSocket` (browsers, Node >= 22) and the `ws` package
* satisfy this, which is why a single code path can drive either.
*/
export interface WebSocketLike {
readyState?: number;
send(data: string | ArrayBufferLike | ArrayBufferView): void;
close(code?: number, reason?: string): void;
addEventListener(type: string, listener: (event: any) => void): void;
removeEventListener?(type: string, listener: (event: any) => void): void;
}
/** Constructor shape for a {@link WebSocketLike}. */
export type WebSocketConstructorLike = new (
url: string,
protocolsOrOptions?: any,
options?: any
) => WebSocketLike;
interface ResolvedWebSocket {
ctor: WebSocketConstructorLike;
/** Can this implementation send custom HTTP headers on the handshake? */
supportsHeaders: boolean;
/** Human-readable origin, used in error messages only. */
source: string;
}
/** WebSocket readyState constants (avoids depending on the ctor's statics). */
const WS_OPEN = 1;
let cachedWebSocket: ResolvedWebSocket | null = null;
/** Reset the cached WebSocket resolution. Exported for tests. */
export function _resetWebSocketResolution(): void {
cachedWebSocket = null;
}
/**
* Resolve a WebSocket implementation, preferring a global one.
*
* @throws if no implementation can be found - never silently degrades.
*/
export async function resolveWebSocketImplementation(
injected?: WebSocketConstructorLike,
/**
* Loader for the optional `ws` package. Defaults to a dynamic import through
* a computed specifier. Overridable in tests so the "no implementation
* available" path can be exercised without uninstalling a transitive `ws`.
*/
wsLoader?: () => Promise<any>
): Promise<ResolvedWebSocket> {
if (injected) {
// An injected constructor is assumed to accept the same options object the
// Node/`ws` implementations do; tests assert on the headers it receives.
return { ctor: injected, supportsHeaders: true, source: 'injected WebSocket' };
}
if (cachedWebSocket) return cachedWebSocket;
const globalObj = globalThis as any;
if (typeof globalObj.WebSocket === 'function') {
// Node's global WebSocket is undici's, which accepts a non-standard
// `{ headers }` option. A browser's does not - passing an object there is
// coerced to a protocol list and throws - so browsers use OpenAI's
// documented subprotocol credentials instead.
const isNode =
typeof globalObj.process !== 'undefined' &&
!!globalObj.process?.versions?.node;
cachedWebSocket = {
ctor: globalObj.WebSocket as WebSocketConstructorLike,
supportsHeaders: isNode,
source: isNode ? 'global WebSocket (Node)' : 'global WebSocket',
};
return cachedWebSocket;
}
// Computed specifier: keeps `ws` off every bundler's static import graph.
const specifier = ['w', 's'].join('');
let loadError: any = null;
try {
const mod: any = wsLoader
? await wsLoader()
: await import(/* webpackIgnore: true */ specifier);
const ctor = mod?.WebSocket ?? mod?.default?.WebSocket ?? mod?.default ?? mod;
if (typeof ctor === 'function') {
cachedWebSocket = {
ctor: ctor as WebSocketConstructorLike,
supportsHeaders: true,
source: "'ws' package",
};
return cachedWebSocket;
}
loadError = new Error("'ws' resolved but exported no WebSocket constructor");
} catch (error: any) {
// Only a module-not-found means "not installed". Anything else is a real
// failure inside an installed `ws` and must not be reported as absence.
const code = error?.code;
if (code !== 'MODULE_NOT_FOUND' && code !== 'ERR_MODULE_NOT_FOUND') {
loadError = error;
}
}
throw new Error(
'RealtimeAgent requires a WebSocket implementation and none is available. ' +
'This runtime has no global WebSocket (Node >= 22, browsers and React Native ' +
"provide one) and the optional 'ws' package " +
(loadError ? `failed to load: ${loadError?.message ?? loadError}. ` : 'is not installed. ') +
'Fix by one of: upgrade to Node 22+, run `npm install ws`, or pass ' +
'`new RealtimeAgent({ webSocket: YourWebSocketConstructor })`.'
);
}
// ============================================================================
// Configuration Types
// ============================================================================
@@ -154,7 +19,7 @@ export interface RealtimeConfig {
inputFormat?: 'pcm16' | 'g711_ulaw' | 'g711_alaw';
/** Audio output format */
outputFormat?: 'pcm16' | 'g711_ulaw' | 'g711_alaw';
/** Sample rate in Hz (local playback hint; not sent to the API) */
/** Sample rate in Hz */
sampleRate?: number;
/** Enable voice activity detection */
vadEnabled?: boolean;
@@ -165,11 +30,9 @@ export interface RealtimeConfig {
}
/**
* Event types the agent knows by name. The server may send others; they are
* emitted verbatim rather than dropped, which is why {@link RealtimeEventType}
* also admits arbitrary strings.
* Event types for realtime sessions.
*/
export type KnownRealtimeEventType =
export type RealtimeEventType =
| 'session.created'
| 'session.updated'
| 'input_audio_buffer.append'
@@ -183,20 +46,11 @@ export type KnownRealtimeEventType =
| 'response.text.done'
| 'error';
/**
* Event type for realtime sessions.
*
* `'*'` is a local wildcard: a handler registered for it receives every event.
* Any other string is a server event type, passed through untouched.
*/
export type RealtimeEventType = KnownRealtimeEventType | '*' | (string & {});
/**
* Realtime event.
*/
export interface RealtimeEvent {
type: RealtimeEventType;
/** The full decoded server payload for this event. */
data?: any;
timestamp?: number;
}
@@ -217,18 +71,6 @@ export interface RealtimeAgentConfig {
verbose?: boolean;
/** API key */
apiKey?: string;
/**
* Full WebSocket endpoint override (e.g. a local test server or a proxy).
* When set, the `model` query parameter is not appended and a missing API
* key is not an error.
*/
url?: string;
/** Extra handshake headers (ignored by implementations that cannot send them) */
headers?: Record<string, string>;
/** Inject a WebSocket constructor instead of resolving one */
webSocket?: WebSocketConstructorLike;
/** How long to wait for the handshake before failing. Default 30000ms. */
connectTimeoutMs?: number;
}
// ============================================================================
@@ -245,138 +87,47 @@ const DEFAULT_REALTIME_CONFIG: Required<RealtimeConfig> = {
timeout: 300,
};
const DEFAULT_ENDPOINT = 'wss://api.openai.com/v1/realtime';
const DEFAULT_CONNECT_TIMEOUT_MS = 30000;
// ============================================================================
// Helpers
// ============================================================================
/** Encode bytes as base64 without depending on any Node builtin import. */
function toBase64(bytes: Uint8Array): string {
const globalObj = globalThis as any;
if (typeof globalObj.btoa === 'function') {
let binary = '';
const chunkSize = 0x8000;
for (let i = 0; i < bytes.length; i += chunkSize) {
binary += String.fromCharCode(...bytes.subarray(i, i + chunkSize));
}
return globalObj.btoa(binary);
}
if (typeof globalObj.Buffer !== 'undefined') {
return globalObj.Buffer.from(bytes).toString('base64');
}
throw new Error('No base64 encoder available in this runtime (no btoa, no Buffer)');
}
/** Decode base64 to bytes without depending on any Node builtin import. */
function fromBase64(value: string): Uint8Array {
const globalObj = globalThis as any;
if (typeof globalObj.atob === 'function') {
const binary = globalObj.atob(value);
const out = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) out[i] = binary.charCodeAt(i);
return out;
}
if (typeof globalObj.Buffer !== 'undefined') {
return new Uint8Array(globalObj.Buffer.from(value, 'base64'));
}
throw new Error('No base64 decoder available in this runtime (no atob, no Buffer)');
}
function toUint8Array(data: ArrayBuffer | Uint8Array | ArrayBufferView): Uint8Array {
if (data instanceof Uint8Array) return data;
if (data instanceof ArrayBuffer) return new Uint8Array(data);
const view = data as ArrayBufferView;
return new Uint8Array(view.buffer, view.byteOffset, view.byteLength);
}
/** Turn whatever a WebSocket handed us into a string, or null if we cannot. */
function messageToString(data: any): string | null {
if (typeof data === 'string') return data;
if (data == null) return null;
if (typeof ArrayBuffer !== 'undefined' && data instanceof ArrayBuffer) {
return new TextDecoder().decode(new Uint8Array(data));
}
if (ArrayBuffer.isView(data)) {
return new TextDecoder().decode(toUint8Array(data));
}
if (Array.isArray(data)) {
// `ws` can deliver a fragmented message as an array of Buffers.
return data.map((part) => messageToString(part) ?? '').join('');
}
if (typeof data.toString === 'function') return String(data);
return null;
}
/** Best-effort description of a WebSocket error/close event, for messages. */
function describeFailure(event: any): string {
if (!event) return 'unknown error';
const parts: string[] = [];
const message = event.message ?? event.error?.message ?? event.reason;
if (message) parts.push(String(message));
if (typeof event.code === 'number') parts.push(`close code ${event.code}`);
if (parts.length === 0 && event.type) parts.push(`${event.type} event`);
return parts.length > 0 ? parts.join(', ') : 'unknown error';
}
// ============================================================================
// RealtimeAgent Class
// ============================================================================
/**
* Agent for real-time voice conversations.
*
* Uses a real WebSocket connection for low-latency audio streaming.
* `connect()` resolves only once the handshake has actually completed; a bad
* key, a bad URL or a refused connection rejects, and `isConnected()` stays
* false.
*
*
* Uses WebSocket connections for low-latency audio streaming.
*
* @example
* ```typescript
* import { RealtimeAgent } from 'praisonai';
*
*
* const agent = new RealtimeAgent({
* instructions: 'You are a helpful voice assistant',
* realtime: { voice: 'nova' }
* });
*
* // Listen for what the server actually sends
*
* // Connect to realtime session
* await agent.connect();
*
* // Send audio
* agent.sendAudio(audioBuffer);
*
* // Listen for responses
* agent.on('response.audio.delta', (event) => {
* playAudio(event.data);
* });
*
* await agent.connect(); // rejects if the handshake fails
* agent.sendAudio(audioBuffer);
* await agent.disconnect();
* ```
*/
export class RealtimeAgent {
static readonly DEFAULT_MODEL = 'gpt-4o-realtime-preview';
readonly name: string;
readonly llm: string;
private readonly llm: string;
private readonly instructions?: string;
private readonly verbose: boolean;
private readonly realtimeConfig: Required<RealtimeConfig>;
private readonly apiKey?: string;
private readonly urlOverride?: string;
private readonly extraHeaders: Record<string, string>;
private readonly webSocketImpl?: WebSocketConstructorLike;
private readonly connectTimeoutMs: number;
private connected: boolean = false;
private ws: WebSocketLike | null = null;
/** In-flight connect(), so concurrent callers share one handshake. */
private connecting: Promise<void> | null = null;
/** The socket being opened by the in-flight connect(), for cancellation. */
private connectingSocket: WebSocketLike | null = null;
/** Bumped on every teardown/disconnect so a stale handshake can abort. */
private connectGeneration: number = 0;
private eventHandlers: Map<RealtimeEventType, Array<(event: RealtimeEvent) => void>> = new Map();
private messageCallbacks: Array<(text: string) => void> = [];
private audioCallbacks: Array<(audio: Uint8Array) => void> = [];
private errorCallbacks: Array<(error: Error) => void> = [];
constructor(config: RealtimeAgentConfig) {
this.name = config.name || 'RealtimeAgent';
@@ -384,10 +135,6 @@ export class RealtimeAgent {
this.instructions = config.instructions;
this.verbose = config.verbose ?? true;
this.apiKey = config.apiKey || process.env.OPENAI_API_KEY;
this.urlOverride = config.url;
this.extraHeaders = { ...(config.headers ?? {}) };
this.webSocketImpl = config.webSocket;
this.connectTimeoutMs = config.connectTimeoutMs ?? DEFAULT_CONNECT_TIMEOUT_MS;
// Resolve realtime configuration
if (config.realtime === undefined || config.realtime === true || config.realtime === false) {
@@ -403,440 +150,96 @@ export class RealtimeAgent {
}
}
/** The endpoint this agent will dial. Never contains the API key. */
getUrl(): string {
if (this.urlOverride) return this.urlOverride;
return `${DEFAULT_ENDPOINT}?model=${encodeURIComponent(this.llm)}`;
}
/** The `session.update` payload derived from the local config. */
private sessionPayload(): Record<string, any> {
const payload: Record<string, any> = {
voice: this.realtimeConfig.voice,
input_audio_format: this.realtimeConfig.inputFormat,
output_audio_format: this.realtimeConfig.outputFormat,
turn_detection: this.realtimeConfig.vadEnabled
? { type: 'server_vad', threshold: this.realtimeConfig.vadThreshold }
: null,
};
if (this.instructions) payload.instructions = this.instructions;
return payload;
}
// =========================================================================
// Connection
// =========================================================================
/**
* Connect to the realtime session.
*
* Resolves only after the WebSocket handshake has completed. Rejects on a
* refused connection, a non-101 response (which is what a bad API key looks
* like), a close during the handshake, or a timeout.
*/
async connect(): Promise<void> {
if (this.isConnected()) {
this.log('Already connected to realtime session');
return;
}
// Concurrent callers share the one in-flight handshake instead of each
// opening a socket and racing to overwrite `this.ws`.
if (this.connecting) return this.connecting;
const attempt = this.doConnect();
this.connecting = attempt;
try {
await attempt;
} finally {
if (this.connecting === attempt) this.connecting = null;
}
}
private async doConnect(): Promise<void> {
// A half-dead socket from a previous attempt must not linger. Bumping the
// generation here invalidates any earlier handshake still in flight.
this.teardown();
const generation = this.connectGeneration;
const url = this.getUrl();
const usingDefaultEndpoint = !this.urlOverride;
if (!this.apiKey && usingDefaultEndpoint) {
throw new Error(
'OPENAI_API_KEY is required to connect to the OpenAI Realtime API. ' +
'Set the environment variable, pass `{ apiKey }`, or point `{ url }` at ' +
'your own endpoint.'
);
}
const impl = await resolveWebSocketImplementation(this.webSocketImpl);
this.log(`Connecting to realtime session at ${url} ...`);
let socket: WebSocketLike;
try {
if (impl.supportsHeaders) {
const headers: Record<string, string> = { ...this.extraHeaders };
if (this.apiKey) {
headers['Authorization'] = `Bearer ${this.apiKey}`;
headers['OpenAI-Beta'] = 'realtime=v1';
}
socket = new impl.ctor(url, { headers });
} else {
// Browsers cannot set handshake headers. OpenAI's documented browser
// path carries the credentials as subprotocols instead. Note this puts
// the API key in the page; use an ephemeral key.
const protocols = ['realtime'];
if (this.apiKey) protocols.push(`openai-insecure-api-key.${this.apiKey}`);
protocols.push('openai-beta.realtime-v1');
Object.keys(this.extraHeaders).length > 0 &&
this.log(
`Note: ${impl.source} cannot send custom headers; ` +
`${Object.keys(this.extraHeaders).join(', ')} were not sent.`
);
socket = new impl.ctor(url, protocols);
}
} catch (error: any) {
throw new Error(
`Failed to create a WebSocket for ${url} using ${impl.source}: ${error?.message ?? error}`
);
}
/** Was this handshake superseded by a teardown/disconnect/newer connect? */
const superseded = () => this.connectGeneration !== generation;
// A disconnect() may have landed before the socket even existed. If so, the
// generation has already moved on; abandon this socket immediately.
if (superseded()) {
try {
socket.close();
} catch {
/* already gone */
}
throw new Error('Connection was cancelled by disconnect() during handshake');
}
// Expose the opening socket so a concurrent disconnect() can cancel it.
this.connectingSocket = socket;
try {
await new Promise<void>((resolve, reject) => {
let settled = false;
const timer = setTimeout(() => {
if (settled) return;
settled = true;
try {
socket.close();
} catch {
/* already gone */
}
reject(new Error(`Timed out after ${this.connectTimeoutMs}ms connecting to ${url}`));
}, this.connectTimeoutMs);
if (typeof (timer as any).unref === 'function') (timer as any).unref();
const settle = (error?: Error) => {
if (settled) return false;
settled = true;
clearTimeout(timer);
if (error) reject(error);
else resolve();
return true;
};
socket.addEventListener('open', () => {
if (superseded()) {
// disconnect()/a newer connect() won the race; drop this socket.
settle(new Error('Connection superseded before handshake completed'));
try {
socket.close();
} catch {
/* already gone */
}
return;
}
settle();
});
socket.addEventListener('error', (event: any) => {
const error = new Error(
`WebSocket connection to ${url} failed: ${describeFailure(event)}`
);
if (!settle(error)) {
// Post-handshake error: report it, do not claim a connection.
this.connected = false;
this.reportError(error);
}
});
socket.addEventListener('close', (event: any) => {
if (
!settle(
new Error(
`WebSocket to ${url} closed during handshake: ${describeFailure(event)}`
)
)
) {
this.handleClose(event);
}
});
socket.addEventListener('message', (event: any) => {
this.handleMessage(event);
});
});
} finally {
if (this.connectingSocket === socket) this.connectingSocket = null;
}
// A disconnect() that arrived during the handshake must win: never mark
// ourselves connected against a socket the caller already asked us to drop.
if (superseded()) {
try {
socket.close();
} catch {
/* already gone */
}
throw new Error('Connection was cancelled by disconnect() during handshake');
}
this.ws = socket;
this.log('Connecting to realtime session...');
// Placeholder - real implementation would establish WebSocket connection
this.connected = true;
// Mirror the Python agent: push the session configuration immediately.
await this.sendEvent({ type: 'session.update', session: this.sessionPayload() });
this.log(`Connected to realtime session with model ${this.llm}`);
this.emit({ type: 'session.created', timestamp: Date.now() });
this.log('Connected to realtime session');
}
/**
* Disconnect from the realtime session.
*/
async disconnect(): Promise<void> {
// Cancel any handshake still in flight so it cannot mark us connected
// after the caller asked to disconnect. teardown() bumps the generation,
// which the pending doConnect() checks; awaiting its promise below makes
// disconnect() a hard barrier even when the socket did not exist yet.
const pendingConnect = this.connecting;
const pendingSocket = this.connectingSocket;
const socket = this.ws;
if (!socket && !pendingSocket && !pendingConnect) {
this.connected = false;
return;
}
this.log('Disconnecting from realtime session...');
// Invalidate the in-flight attempt first, then close its socket.
this.teardown();
if (pendingSocket) {
try {
pendingSocket.close(1000, 'client disconnect');
} catch {
/* already gone */
}
}
if (pendingConnect) {
// The pending connect() will reject (superseded); do not let that reject
// escape from disconnect().
await pendingConnect.catch(() => undefined);
}
if (!socket) return;
await new Promise<void>((resolve) => {
let done = false;
const finish = () => {
if (done) return;
done = true;
clearTimeout(timer);
resolve();
};
const timer = setTimeout(finish, 5000);
if (typeof (timer as any).unref === 'function') (timer as any).unref();
try {
socket.addEventListener('close', finish);
socket.close(1000, 'client disconnect');
} catch {
finish();
}
});
this.teardown();
}
/**
* Drop the socket reference and mark the agent disconnected.
*
* Bumps {@link connectGeneration} so any handshake still in flight is
* invalidated: it will close its socket and reject rather than flip the
* agent back to connected.
*/
private teardown(): void {
this.connectGeneration++;
this.ws = null;
this.connected = false;
this.connectingSocket = null;
}
/**
* Check if connected. Reflects the live socket state, not an intention.
* Check if connected.
*/
isConnected(): boolean {
if (!this.connected || !this.ws) return false;
const readyState = this.ws.readyState;
return typeof readyState === 'number' ? readyState === WS_OPEN : true;
}
// =========================================================================
// Sending
// =========================================================================
/**
* Send a raw event over the socket.
*
* @throws if not connected - it never silently drops the event.
*/
async sendEvent(event: Record<string, any>): Promise<void> {
if (!this.isConnected() || !this.ws) {
throw new Error(
`Not connected to realtime session; cannot send "${event.type}". Call connect() first.`
);
}
this.ws.send(JSON.stringify(event));
return this.connected;
}
/**
* Send audio data to the session (base64-encoded, per the Realtime API).
* Send audio data to the session.
*/
sendAudio(audioData: ArrayBuffer | Uint8Array): void {
if (!this.isConnected() || !this.ws) {
if (!this.connected) {
throw new Error('Not connected to realtime session');
}
this.ws.send(
JSON.stringify({
type: 'input_audio_buffer.append',
audio: toBase64(toUint8Array(audioData)),
})
);
this.emit({
type: 'input_audio_buffer.append',
data: audioData,
timestamp: Date.now(),
});
}
/**
* Commit the audio buffer for processing.
*/
commitAudio(): void {
if (!this.isConnected() || !this.ws) {
if (!this.connected) {
throw new Error('Not connected to realtime session');
}
this.ws.send(JSON.stringify({ type: 'input_audio_buffer.commit' }));
this.emit({
type: 'input_audio_buffer.commit',
timestamp: Date.now(),
});
}
/**
* Clear the audio buffer.
*/
clearAudio(): void {
if (!this.isConnected() || !this.ws) {
if (!this.connected) {
throw new Error('Not connected to realtime session');
}
this.ws.send(JSON.stringify({ type: 'input_audio_buffer.clear' }));
this.emit({
type: 'input_audio_buffer.clear',
timestamp: Date.now(),
});
}
/**
* Send a text message and request a response (Python `asend_text` parity).
* Send a text message.
*/
async sendText(text: string): Promise<void> {
if (!this.isConnected() || !this.ws) {
if (!this.connected) {
throw new Error('Not connected to realtime session');
}
this.log(`Sending text: ${text}`);
await this.sendEvent({
type: 'conversation.item.create',
item: {
type: 'message',
role: 'user',
content: [{ type: 'input_text', text }],
},
this.emit({
type: 'response.create',
data: { text },
timestamp: Date.now(),
});
await this.sendEvent({ type: 'response.create' });
}
// =========================================================================
// Receiving
// =========================================================================
private handleMessage(event: any): void {
const raw = messageToString(event?.data ?? event);
if (raw === null) return;
let parsed: any;
try {
parsed = JSON.parse(raw);
} catch {
return; // Python's receive_loop ignores non-JSON frames too.
}
if (!parsed || typeof parsed.type !== 'string') return;
const type: string = parsed.type;
// Python-parity convenience callbacks.
if (type === 'response.text.delta') {
const delta = typeof parsed.delta === 'string' ? parsed.delta : '';
this.invokeCallbacks(this.messageCallbacks, delta);
} else if (type === 'response.audio.delta') {
const delta = typeof parsed.delta === 'string' ? parsed.delta : '';
if (delta) {
let bytes: Uint8Array | null = null;
try {
bytes = fromBase64(delta);
} catch {
bytes = null;
}
if (bytes) this.invokeCallbacks(this.audioCallbacks, bytes);
}
} else if (type === 'error') {
const error = new Error(
parsed.error?.message ?? `Realtime API error: ${JSON.stringify(parsed.error ?? parsed)}`
);
this.reportError(error);
}
this.emit({ type, data: parsed, timestamp: Date.now() });
}
private handleClose(event: any): void {
const wasConnected = this.connected;
this.teardown();
if (wasConnected) {
this.log(`Realtime session closed: ${describeFailure(event)}`);
}
}
private reportError(error: Error): void {
this.invokeCallbacks(this.errorCallbacks, error);
}
/**
* Invoke user callbacks over a snapshot of the list, isolating each one so a
* throwing handler cannot skip later handlers or stop the frame from reaching
* `emit`. Mirrors the isolation already applied to error callbacks.
*/
private invokeCallbacks<T>(callbacks: Array<(value: T) => void>, value: T): void {
for (const callback of [...callbacks]) {
try {
callback(value);
} catch {
/* a bad handler must not break the socket */
}
}
}
// =========================================================================
// Handlers
// =========================================================================
/**
* Register an event handler. Use `'*'` to receive every server event.
* Register an event handler.
*/
on(eventType: RealtimeEventType, handler: (event: RealtimeEvent) => void): void {
if (!this.eventHandlers.has(eventType)) {
@@ -858,34 +261,14 @@ export class RealtimeAgent {
}
}
/** Register a callback for text deltas (Python `on_message` parity). */
onMessage(callback: (text: string) => void): void {
this.messageCallbacks.push(callback);
}
/** Register a callback for decoded audio deltas (Python `on_audio` parity). */
onAudio(callback: (audio: Uint8Array) => void): void {
this.audioCallbacks.push(callback);
}
/** Register a callback for errors (Python `on_error` parity). */
onError(callback: (error: Error) => void): void {
this.errorCallbacks.push(callback);
}
/**
* Emit an event to handlers.
*/
private emit(event: RealtimeEvent): void {
const targets = [
...(this.eventHandlers.get(event.type) ?? []),
...(this.eventHandlers.get('*') ?? []),
];
for (const handler of targets) {
try {
const handlers = this.eventHandlers.get(event.type);
if (handlers) {
for (const handler of handlers) {
handler(event);
} catch {
/* a bad handler must not break the socket */
}
}
}
@@ -898,13 +281,17 @@ export class RealtimeAgent {
}
/**
* Update the session configuration. Pushes `session.update` when connected.
* Update the session configuration.
*/
async updateConfig(config: Partial<RealtimeConfig>): Promise<void> {
Object.assign(this.realtimeConfig, config);
if (this.isConnected()) {
await this.sendEvent({ type: 'session.update', session: this.sessionPayload() });
if (this.connected) {
this.emit({
type: 'session.updated',
data: this.realtimeConfig,
timestamp: Date.now(),
});
}
}
}
+25 -56
View File
@@ -1,41 +1,21 @@
/**
* Database Module - Exports for persistence layer
*
*
* Usage (Python-like simplicity):
* import { db } from 'praisonai';
*
*
* const agent = new Agent({
* instructions: "You are helpful",
* db: db("sqlite:./data.db"), // URL-style string, persists to a file
* db: db("sqlite:./data.db"), // URL-style string
* sessionId: "my-session"
* });
*
* What is durable today:
* - db("sqlite:./data.db") persists to disk and survives the process
* - db("sqlite::memory:") real SQLite, but process-local
* - db("memory:") in-process Maps, lost when the process exits
*
* postgres:// and redis:// still throw -- see createDbAdapter below.
*/
export * from './types';
export { MemoryDbAdapter } from './memory-adapter';
export {
SqliteDbAdapter,
createSqliteDbAdapter,
SQLITE_SCHEMA_VERSION,
} from './sqlite-adapter';
export type {
SqliteDbAdapterOptions,
SqliteDriver,
SqliteDriverName,
SqliteConnection,
SqliteStatement,
} from './sqlite-adapter';
import type { DbAdapter, DbConfig } from './types';
import { MemoryDbAdapter } from './memory-adapter';
import { SqliteDbAdapter } from './sqlite-adapter';
// Default adapter instance
let defaultAdapter: DbAdapter | null = null;
@@ -103,32 +83,28 @@ export function createDbAdapter(config: DbConfig): DbAdapter {
case 'memory':
return new MemoryDbAdapter();
case 'sqlite':
// Implements the full DbAdapter session/message/run contract on a real
// SQLite file, so history survives the process. The database is opened
// lazily on first use (the Agent never calls connect()); if no SQLite
// driver can open the file the operation REJECTS with the reason and the
// fix. It never degrades to memory -- persistence that silently stops
// persisting is the failure this replaced.
return new SqliteDbAdapter({ filename: config.path || ':memory:' });
// The SQLite/Postgres/Redis modules ship low-level transports
// (SQLiteAdapter, NeonPostgresAdapter, UpstashRedisAdapter) that do NOT
// yet implement the full DbAdapter session/message/run contract used by
// the Agent. Returning one here would satisfy the type at the untyped
// require() boundary but crash at runtime ("... is not a function") on
// first use. Fail loudly with a clear path forward instead of handing
// back a non-conforming object.
throw new Error(
'db("sqlite:...") is not yet wired to the DbAdapter contract. ' +
'Use db("memory:") for now, or import { createSQLiteAdapter } from ' +
'"praisonai/db/sqlite" for the low-level SQLite transport.'
);
case 'postgres':
// Still unwired, deliberately. src/db/postgres.ts is a Neon HTTP
// transport (POST https://<host>/sql): it has no local mode, cannot be
// exercised without a live Neon endpoint, and exposes query/execute
// rather than the session/message/run contract the Agent calls. Wiring
// it up untested would trade a clear error for a runtime surprise.
throw new Error(
'db("postgres://...") is not yet wired to the DbAdapter contract. ' +
'Use db("sqlite:./data.db") for durable local persistence, db("memory:") ' +
'for ephemeral, or import { createNeonPostgres } from ' +
'Use db("memory:") for now, or import { createNeonPostgres } from ' +
'"praisonai/db/postgres" for the low-level Postgres transport.'
);
case 'redis':
// Same reasoning: src/db/redis.ts is an Upstash REST transport with
// key/value and hash operations, not sessions, messages and runs.
throw new Error(
'db("redis://...") is not yet wired to the DbAdapter contract. ' +
'Use db("sqlite:./data.db") for durable local persistence, db("memory:") ' +
'for ephemeral, or import { createUpstashRedis } from ' +
'Use db("memory:") for now, or import { createUpstashRedis } from ' +
'"praisonai/db/redis" for the low-level Redis transport.'
);
default:
@@ -155,24 +131,17 @@ export function setDefaultDbAdapter(adapter: DbAdapter): void {
/**
* Factory function for creating a database adapter
*
*
* Accepts either:
* - URL string: db("sqlite:./data.db"), db("memory:")
* - URL string: db("sqlite:./data.db"), db("postgres://..."), db("redis://...")
* - Config object: db({ type: 'sqlite', path: './data.db' })
*
*
* Examples:
* db("sqlite:./data.db") // SQLite file -- durable, survives restarts
* db("sqlite::memory:") // real SQLite, process-local
* db("memory:") // in-process Maps (default), lost on exit
* db() // same as db("memory:")
*
* Not yet implemented -- these parse, then throw with the reason:
* db("postgres://localhost/mydb") // throws: transport is Neon-HTTP only
* db("redis://localhost:6379") // throws: transport is Upstash-REST only
*
* The returned adapter connects lazily, so a sqlite URL is cheap to build and
* any driver problem surfaces on the first read or write -- as a rejection,
* never as a silent fall back to memory.
* db("sqlite:./data.db") // SQLite file
* db("postgres://localhost/mydb") // PostgreSQL
* db("redis://localhost:6379") // Redis
* db("memory:") // In-memory (default)
* db() // In-memory (default)
*/
export function db(configOrUrl: string | DbConfig = { type: 'memory' }): DbAdapter {
if (typeof configOrUrl === 'string') {
-975
View File
@@ -1,975 +0,0 @@
/**
* SQLite DbAdapter -- durable persistence for `db("sqlite:./data.db")`.
*
* This is the full {@link DbAdapter} contract (sessions, messages, runs, tool
* calls, traces, spans) on top of a real SQLite file, so a session written by
* one process is readable by the next one. `MemoryDbAdapter` dies with the
* process; this does not.
*
* NOT the same thing as `./sqlite.ts`. That module ships `SQLiteAdapter`, a
* low-level transport with its own narrower table shapes and its own method
* names (`addMessage`, `addTrace`, ...). It is kept for callers that already
* use it; it is not what the `Agent` talks to, and it silently degrades to an
* in-process Map when the native binding will not load. This adapter never
* does that -- see "Driver loading" below.
*
* ## Driver loading
*
* Two drivers can back this adapter, tried in order:
*
* 1. `better-sqlite3` -- the declared dependency. It is a native module, so
* its prebuilt binding is compiled against one specific Node ABI
* (`NODE_MODULE_VERSION`). Installed under one Node major and run under
* another, `new Database(...)` throws `ERR_DLOPEN_FAILED`.
* 2. `node:sqlite` -- Node's own SQLite, built into Node >= 22.5. No native
* build step, so no ABI to mismatch. It is flagged experimental by Node
* and prints one `ExperimentalWarning` when first loaded. Same on-disk
* format, so a file written by either driver reads in the other.
*
* If neither loads, every operation REJECTS with a message naming what each
* driver did and how to fix it. It never falls back to memory: a persistence
* layer that quietly stops persisting is the bug this module exists to fix.
*
* `PRAISONAI_SQLITE_DRIVER=better-sqlite3|node` pins the choice (useful to
* reproduce a driver-specific failure); `driver` on the constructor does the
* same per-instance.
*
* ## Schema (version 1.0)
*
* Table and column names mirror what the Python side persists
* (`praisonaiagents/db/protocol.py`, `runs/sqlite_ledger.py`) so the two are
* conceptually aligned: snake_case columns, JSON-encoded `metadata`,
* epoch-millisecond timestamps. Only the fields TypeScript's `DbAdapter`
* actually has are stored -- Python's `input_content`/`output_content`/
* `events` have no TypeScript counterpart and are not invented here.
*
* sessions(id PK, created_at, updated_at, metadata)
* messages(id PK, session_id, run_id, role, content, name, tool_call_id,
* tool_calls, created_at, metadata)
* runs(id PK, session_id, agent_name, status, started_at, completed_at,
* error, metadata, prompt_tokens, completion_tokens, total_tokens)
* tool_calls(id PK, run_id, name, arguments, result, status, started_at,
* completed_at, error)
* traces(id PK, session_id, run_id, agent_name, started_at, completed_at,
* status, metadata)
* spans(id PK, trace_id, parent_id, name, started_at, completed_at, status,
* attributes)
* praisonai_meta(key PK, value) -- holds schema_version
*
* Timestamps are epoch milliseconds (`Date.now()`), matching the `DbMessage`/
* `DbRun` types, NOT Python's float seconds.
*/
import type {
DbAdapter,
DbSession,
DbMessage,
DbRun,
DbToolCall,
DbTrace,
DbSpan,
} from './types';
/** Schema version stored in `praisonai_meta`. Mirrors Python's SCHEMA_VERSION. */
export const SQLITE_SCHEMA_VERSION = '1.0';
/** The only value types bound to a statement. `undefined` is never bound: */
/** `node:sqlite` rejects it outright and better-sqlite3 rejects it too. */
export type SqliteParam = string | number | null;
/** A prepared statement, as both drivers expose one. */
export interface SqliteStatement {
run(...params: SqliteParam[]): unknown;
get(...params: SqliteParam[]): any;
all(...params: SqliteParam[]): any[];
}
/** An open database handle, normalised across drivers. */
export interface SqliteConnection {
exec(sql: string): void;
prepare(sql: string): SqliteStatement;
close(): void;
}
/** A driver is a name plus a way to open a file. */
export interface SqliteDriver {
name: string;
open(filename: string): SqliteConnection;
}
export type SqliteDriverName = 'better-sqlite3' | 'node';
export interface SqliteDbAdapterOptions {
/** File path, or `':memory:'` for a real but process-local SQLite database. */
filename: string;
/** Pin the driver instead of trying each in turn. */
driver?: SqliteDriverName;
/**
* Override driver loading. Exists so tests can prove the failure path (an
* unusable native binding must throw, not degrade); not part of the public
* contract.
*/
driverLoader?: () => Promise<SqliteDriver>;
}
// ---------------------------------------------------------------------------
// Driver loading
// ---------------------------------------------------------------------------
/**
* better-sqlite3 defers loading its native binding until `new Database(...)`,
* so a successful `import` proves nothing. The driver is only considered
* loaded once a connection has actually opened.
*/
async function loadBetterSqlite3(): Promise<SqliteDriver> {
// @ts-ignore -- optional native dependency, ships no bundled type declarations
const mod: any = await import('better-sqlite3');
const Database = mod?.default ?? mod;
if (typeof Database !== 'function') {
throw new Error('better-sqlite3 did not export a Database constructor');
}
return {
name: 'better-sqlite3',
open(filename: string): SqliteConnection {
const handle = new Database(filename);
return {
exec: (sql: string) => handle.exec(sql),
prepare: (sql: string) => handle.prepare(sql) as SqliteStatement,
close: () => handle.close(),
};
},
};
}
/** Node's built-in SQLite (>= 22.5). No native build, so no ABI to mismatch. */
async function loadNodeSqlite(): Promise<SqliteDriver> {
const mod: any = await import('node:sqlite');
const DatabaseSync = mod?.DatabaseSync ?? mod?.default?.DatabaseSync;
if (typeof DatabaseSync !== 'function') {
throw new Error('node:sqlite did not export DatabaseSync (Node >= 22.5 required)');
}
return {
name: 'node:sqlite',
open(filename: string): SqliteConnection {
const handle = new DatabaseSync(filename);
return {
exec: (sql: string) => handle.exec(sql),
prepare: (sql: string) => {
const stmt = handle.prepare(sql);
return {
run: (...params: SqliteParam[]) => stmt.run(...params),
// node:sqlite returns undefined for no row; normalise to null so
// callers can use a single `?? null` shape across drivers.
get: (...params: SqliteParam[]) => stmt.get(...params) ?? null,
all: (...params: SqliteParam[]) => stmt.all(...params) as any[],
};
},
close: () => handle.close(),
};
},
};
}
function envDriverPreference(): SqliteDriverName | undefined {
const raw =
typeof process !== 'undefined' && process.env
? process.env.PRAISONAI_SQLITE_DRIVER
: undefined;
if (!raw) return undefined;
const value = raw.trim().toLowerCase();
if (value === 'better-sqlite3' || value === 'better') return 'better-sqlite3';
if (value === 'node' || value === 'node:sqlite' || value === 'builtin') return 'node';
return undefined;
}
function describeError(error: unknown): string {
if (error instanceof Error) return error.message.split('\n').slice(0, 3).join(' ');
return String(error);
}
/**
* Turn "no driver worked" into a message someone can act on: what was tried,
* what each one said, and the specific fix for the failure that was seen.
*/
function unavailableError(
filename: string,
failures: Array<{ driver: string; error: unknown }>,
): Error {
const joined = failures.map((f) => describeError(f.error)).join(' | ');
const hints: string[] = [];
if (/NODE_MODULE_VERSION|ERR_DLOPEN_FAILED|was compiled against/i.test(joined)) {
hints.push(
'better-sqlite3\'s native binding was built for a different Node ABI. ' +
'Run `npm rebuild better-sqlite3` (or reinstall) under the Node version you run with.',
);
}
if (/Cannot find module|MODULE_NOT_FOUND|ERR_MODULE_NOT_FOUND|ERR_UNKNOWN_BUILTIN_MODULE/i.test(joined)) {
hints.push(
'Install the driver (`npm install better-sqlite3`), or run Node >= 22.5 so the ' +
'built-in `node:sqlite` can be used.',
);
}
if (/unable to open database file|SQLITE_CANTOPEN/i.test(joined)) {
hints.push(
`SQLite could not open "${filename}". Check the parent directory exists and is writable.`,
);
}
hints.push('db("memory:") works everywhere, but does not survive the process.');
const tried = failures.map((f) => ` - ${f.driver}: ${describeError(f.error)}`).join('\n');
return new Error(
`SQLite persistence is unavailable for "${filename}". No driver could open it.\n` +
`${tried}\n` +
hints.map((h) => ` → ${h}`).join('\n'),
);
}
/**
* Open `filename` with the first driver that works.
*
* Opening is part of loading on purpose: better-sqlite3's ABI failure only
* surfaces at `new Database(...)`, so a loader that stopped at `import` would
* hand back a driver that throws on first use.
*/
async function openWithAnyDriver(
filename: string,
preference?: SqliteDriverName,
customLoader?: () => Promise<SqliteDriver>,
): Promise<{ driver: SqliteDriver; connection: SqliteConnection }> {
const loaders: Array<{ name: string; load: () => Promise<SqliteDriver> }> = customLoader
? [{ name: 'custom', load: customLoader }]
: [
{ name: 'better-sqlite3', load: loadBetterSqlite3 },
{ name: 'node:sqlite', load: loadNodeSqlite },
];
const pinned = preference ?? envDriverPreference();
const selected = pinned
? loaders.filter((l) =>
pinned === 'better-sqlite3' ? l.name === 'better-sqlite3' : l.name === 'node:sqlite',
)
: loaders;
const candidates = selected.length > 0 ? selected : loaders;
const failures: Array<{ driver: string; error: unknown }> = [];
for (const candidate of candidates) {
try {
const driver = await candidate.load();
const connection = driver.open(filename);
return { driver, connection };
} catch (error) {
failures.push({ driver: candidate.name, error });
}
}
throw unavailableError(filename, failures);
}
// ---------------------------------------------------------------------------
// Schema
// ---------------------------------------------------------------------------
const SCHEMA_SQL = `
CREATE TABLE IF NOT EXISTS praisonai_meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
metadata TEXT
);
CREATE TABLE IF NOT EXISTS messages (
id TEXT PRIMARY KEY,
session_id TEXT NOT NULL,
run_id TEXT,
role TEXT NOT NULL,
content TEXT,
name TEXT,
tool_call_id TEXT,
tool_calls TEXT,
created_at INTEGER NOT NULL,
metadata TEXT
);
CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id, created_at);
CREATE TABLE IF NOT EXISTS runs (
id TEXT PRIMARY KEY,
session_id TEXT NOT NULL,
agent_name TEXT,
status TEXT NOT NULL,
started_at INTEGER NOT NULL,
completed_at INTEGER,
error TEXT,
metadata TEXT,
prompt_tokens INTEGER,
completion_tokens INTEGER,
total_tokens INTEGER
);
CREATE INDEX IF NOT EXISTS idx_runs_session ON runs(session_id, started_at);
CREATE TABLE IF NOT EXISTS tool_calls (
id TEXT PRIMARY KEY,
run_id TEXT NOT NULL,
name TEXT NOT NULL,
arguments TEXT NOT NULL,
result TEXT,
status TEXT NOT NULL,
started_at INTEGER NOT NULL,
completed_at INTEGER,
error TEXT
);
CREATE INDEX IF NOT EXISTS idx_tool_calls_run ON tool_calls(run_id, started_at);
CREATE TABLE IF NOT EXISTS traces (
id TEXT PRIMARY KEY,
session_id TEXT NOT NULL,
run_id TEXT,
agent_name TEXT,
started_at INTEGER NOT NULL,
completed_at INTEGER,
status TEXT NOT NULL,
metadata TEXT
);
CREATE INDEX IF NOT EXISTS idx_traces_session ON traces(session_id);
CREATE TABLE IF NOT EXISTS spans (
id TEXT PRIMARY KEY,
trace_id TEXT NOT NULL,
parent_id TEXT,
name TEXT NOT NULL,
started_at INTEGER NOT NULL,
completed_at INTEGER,
status TEXT NOT NULL,
attributes TEXT
);
CREATE INDEX IF NOT EXISTS idx_spans_trace ON spans(trace_id, started_at);
`;
/**
* Columns each table must have for this adapter's statements to work.
*
* `CREATE TABLE IF NOT EXISTS` is a no-op against a file that already holds a
* table of the same name and a different shape -- notably one written by the
* older `SQLiteAdapter` in `./sqlite.ts`, whose `runs` has `agent_id` and no
* `agent_name`. Without this check the first INSERT would fail deep in the
* driver with "table runs has no column named agent_name"; with it, the file
* is diagnosed on open.
*
* Every column named by an INSERT or UPDATE is listed, not just the NOT NULL
* core: a file carrying the core columns but missing, say, `runs.total_tokens`
* or `spans.attributes` would otherwise pass open() and fail on the first write
* that touches the missing column. The lists are kept in lockstep with the
* INSERT statements below.
*/
const REQUIRED_COLUMNS: Record<string, string[]> = {
sessions: ['id', 'created_at', 'updated_at', 'metadata'],
messages: [
'id',
'session_id',
'run_id',
'role',
'content',
'name',
'tool_call_id',
'tool_calls',
'created_at',
'metadata',
],
runs: [
'id',
'session_id',
'agent_name',
'status',
'started_at',
'completed_at',
'error',
'metadata',
'prompt_tokens',
'completion_tokens',
'total_tokens',
],
tool_calls: [
'id',
'run_id',
'name',
'arguments',
'result',
'status',
'started_at',
'completed_at',
'error',
],
traces: [
'id',
'session_id',
'run_id',
'agent_name',
'started_at',
'completed_at',
'status',
'metadata',
],
spans: [
'id',
'trace_id',
'parent_id',
'name',
'started_at',
'completed_at',
'status',
'attributes',
],
};
// ---------------------------------------------------------------------------
// Value coercion
// ---------------------------------------------------------------------------
function toJson(value: unknown): string | null {
if (value === undefined || value === null) return null;
try {
return JSON.stringify(value);
} catch {
// A metadata bag with a circular reference must not lose the whole row.
return JSON.stringify(String(value));
}
}
function fromJson(value: unknown): any | undefined {
if (value === undefined || value === null || value === '') return undefined;
try {
return JSON.parse(String(value));
} catch {
return undefined;
}
}
/** `undefined` -> `null`; both drivers reject `undefined` as a bound value. */
function text(value: string | null | undefined): string | null {
return value === undefined || value === null ? null : value;
}
function num(value: number | null | undefined): number | null {
return value === undefined || value === null ? null : value;
}
/** `null` -> `undefined`, so an optional field round-trips as it went in. */
function optText(value: unknown): string | undefined {
return value === null || value === undefined ? undefined : String(value);
}
function optNum(value: unknown): number | undefined {
return value === null || value === undefined ? undefined : Number(value);
}
// ---------------------------------------------------------------------------
// Adapter
// ---------------------------------------------------------------------------
export class SqliteDbAdapter implements DbAdapter {
private readonly filename: string;
private readonly driverPreference?: SqliteDriverName;
private readonly driverLoader?: () => Promise<SqliteDriver>;
private connection: SqliteConnection | null = null;
private driver: SqliteDriver | null = null;
private connecting: Promise<void> | null = null;
constructor(options: SqliteDbAdapterOptions | string) {
const opts = typeof options === 'string' ? { filename: options } : options;
this.filename = opts.filename || ':memory:';
this.driverPreference = opts.driver;
this.driverLoader = opts.driverLoader;
}
/** The file this adapter persists to (`':memory:'` for the in-process form). */
getFilename(): string {
return this.filename;
}
/** Which driver opened the database, once connected. */
getDriverName(): string | null {
return this.driver?.name ?? null;
}
// -- Lifecycle ------------------------------------------------------------
async connect(): Promise<void> {
if (this.connection) return;
if (!this.connecting) {
this.connecting = this.openAndMigrate().finally(() => {
this.connecting = null;
});
}
return this.connecting;
}
private async openAndMigrate(): Promise<void> {
const { driver, connection } = await openWithAnyDriver(
this.filename,
this.driverPreference,
this.driverLoader,
);
try {
if (this.filename !== ':memory:') {
try {
// WAL keeps a reader in another process from blocking a writer.
// Unsupported on some filesystems; never worth failing the open for.
connection.exec('PRAGMA journal_mode = WAL;');
} catch {
/* keep the default journal mode */
}
}
connection.exec(SCHEMA_SQL);
this.verifySchema(connection);
connection
.prepare('INSERT OR REPLACE INTO praisonai_meta (key, value) VALUES (?, ?)')
.run('schema_version', SQLITE_SCHEMA_VERSION);
} catch (error) {
try {
connection.close();
} catch {
/* already closing down */
}
throw error;
}
this.connection = connection;
this.driver = driver;
}
private verifySchema(connection: SqliteConnection): void {
for (const [table, columns] of Object.entries(REQUIRED_COLUMNS)) {
const present = new Set(
connection.prepare(`PRAGMA table_info(${table})`).all().map((row: any) => String(row.name)),
);
const missing = columns.filter((c) => !present.has(c));
if (missing.length > 0) {
throw new Error(
`SQLite file "${this.filename}" already has a "${table}" table with an incompatible ` +
`shape (missing: ${missing.join(', ')}). It was most likely written by the ` +
`low-level SQLiteAdapter in "praisonai/db/sqlite", whose tables differ. ` +
`Point db("sqlite:...") at a different file, or migrate that one.`,
);
}
}
}
async disconnect(): Promise<void> {
if (this.connecting) {
// Do not leave a half-open handle behind if disconnect races connect.
await this.connecting.catch(() => undefined);
}
if (this.connection) {
this.connection.close();
this.connection = null;
this.driver = null;
}
}
isConnected(): boolean {
return this.connection !== null;
}
/**
* The Agent never calls `connect()` -- it goes straight to `saveMessage` /
* `getMessages` -- so every operation opens on demand. When the database
* cannot be opened this rejects with {@link unavailableError}, which is the
* point: no operation ever quietly succeeds against memory.
*/
private async db(): Promise<SqliteConnection> {
if (!this.connection) await this.connect();
if (!this.connection) throw unavailableError(this.filename, []);
return this.connection;
}
// -- Sessions -------------------------------------------------------------
async createSession(session: DbSession): Promise<void> {
const db = await this.db();
db.prepare(
'INSERT OR REPLACE INTO sessions (id, created_at, updated_at, metadata) VALUES (?, ?, ?, ?)',
).run(session.id, num(session.createdAt) ?? Date.now(), num(session.updatedAt) ?? Date.now(), toJson(session.metadata));
}
async getSession(id: string): Promise<DbSession | null> {
const db = await this.db();
const row = db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) ?? null;
return row ? this.rowToSession(row) : null;
}
async updateSession(id: string, updates: Partial<DbSession>): Promise<void> {
const db = await this.db();
const fields: string[] = [];
const values: SqliteParam[] = [];
if (updates.createdAt !== undefined) {
fields.push('created_at = ?');
values.push(updates.createdAt);
}
if (updates.metadata !== undefined) {
fields.push('metadata = ?');
values.push(toJson(updates.metadata));
}
// MemoryDbAdapter stamps updatedAt on every update; match that, and let an
// explicit updates.updatedAt win.
fields.push('updated_at = ?');
values.push(num(updates.updatedAt) ?? Date.now());
values.push(id);
db.prepare(`UPDATE sessions SET ${fields.join(', ')} WHERE id = ?`).run(...values);
}
async deleteSession(id: string): Promise<void> {
const db = await this.db();
db.prepare('DELETE FROM messages WHERE session_id = ?').run(id);
db.prepare('DELETE FROM sessions WHERE id = ?').run(id);
}
async listSessions(limit = 100, offset = 0): Promise<DbSession[]> {
const db = await this.db();
const rows = db
.prepare('SELECT * FROM sessions ORDER BY created_at ASC, rowid ASC LIMIT ? OFFSET ?')
.all(limit, offset);
return rows.map((row) => this.rowToSession(row));
}
private rowToSession(row: any): DbSession {
return {
id: String(row.id),
createdAt: Number(row.created_at),
updatedAt: Number(row.updated_at),
metadata: fromJson(row.metadata),
};
}
// -- Messages -------------------------------------------------------------
async saveMessage(message: DbMessage): Promise<void> {
const db = await this.db();
db.prepare(
'INSERT OR REPLACE INTO messages ' +
'(id, session_id, run_id, role, content, name, tool_call_id, tool_calls, created_at, metadata) ' +
'VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)',
).run(
message.id,
message.sessionId,
text(message.runId),
message.role,
message.content === undefined ? null : message.content,
text(message.name),
text(message.toolCallId),
toJson(message.toolCalls),
num(message.createdAt) ?? Date.now(),
toJson(message.metadata),
);
}
async getMessages(sessionId: string, limit?: number): Promise<DbMessage[]> {
const db = await this.db();
// MemoryDbAdapter returns the LAST `limit` messages, still in chronological
// order (`slice(-limit)`). Reproduce that: newest-first with a LIMIT, then
// reverse. rowid breaks ties, because two messages in the same turn share a
// Date.now() millisecond often enough to scramble a conversation.
if (limit !== undefined && limit > 0) {
const rows = db
.prepare(
'SELECT * FROM messages WHERE session_id = ? ORDER BY created_at DESC, rowid DESC LIMIT ?',
)
.all(sessionId, limit);
return rows.reverse().map((row) => this.rowToMessage(row));
}
const rows = db
.prepare('SELECT * FROM messages WHERE session_id = ? ORDER BY created_at ASC, rowid ASC')
.all(sessionId);
return rows.map((row) => this.rowToMessage(row));
}
async deleteMessages(sessionId: string): Promise<void> {
const db = await this.db();
db.prepare('DELETE FROM messages WHERE session_id = ?').run(sessionId);
}
private rowToMessage(row: any): DbMessage {
return {
id: String(row.id),
sessionId: String(row.session_id),
runId: optText(row.run_id),
role: String(row.role) as DbMessage['role'],
content: row.content === null || row.content === undefined ? null : String(row.content),
name: optText(row.name),
toolCallId: optText(row.tool_call_id),
toolCalls: fromJson(row.tool_calls),
createdAt: Number(row.created_at),
metadata: fromJson(row.metadata),
};
}
// -- Runs -----------------------------------------------------------------
async createRun(run: DbRun): Promise<void> {
const db = await this.db();
db.prepare(
'INSERT OR REPLACE INTO runs ' +
'(id, session_id, agent_name, status, started_at, completed_at, error, metadata, ' +
'prompt_tokens, completion_tokens, total_tokens) ' +
'VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)',
).run(
run.id,
run.sessionId,
text(run.agentName),
run.status,
num(run.startedAt) ?? Date.now(),
num(run.completedAt),
text(run.error),
toJson(run.metadata),
num(run.tokenUsage?.promptTokens),
num(run.tokenUsage?.completionTokens),
num(run.tokenUsage?.totalTokens),
);
}
async getRun(id: string): Promise<DbRun | null> {
const db = await this.db();
const row = db.prepare('SELECT * FROM runs WHERE id = ?').get(id) ?? null;
return row ? this.rowToRun(row) : null;
}
async updateRun(id: string, updates: Partial<DbRun>): Promise<void> {
const db = await this.db();
const fields: string[] = [];
const values: SqliteParam[] = [];
const set = (column: string, value: SqliteParam) => {
fields.push(`${column} = ?`);
values.push(value);
};
if (updates.sessionId !== undefined) set('session_id', updates.sessionId);
if (updates.agentName !== undefined) set('agent_name', text(updates.agentName));
if (updates.status !== undefined) set('status', updates.status);
if (updates.startedAt !== undefined) set('started_at', updates.startedAt);
if (updates.completedAt !== undefined) set('completed_at', num(updates.completedAt));
if (updates.error !== undefined) set('error', text(updates.error));
if (updates.metadata !== undefined) set('metadata', toJson(updates.metadata));
if (updates.tokenUsage !== undefined) {
set('prompt_tokens', num(updates.tokenUsage?.promptTokens));
set('completion_tokens', num(updates.tokenUsage?.completionTokens));
set('total_tokens', num(updates.tokenUsage?.totalTokens));
}
if (fields.length === 0) return;
values.push(id);
db.prepare(`UPDATE runs SET ${fields.join(', ')} WHERE id = ?`).run(...values);
}
async listRuns(sessionId: string, limit = 100): Promise<DbRun[]> {
const db = await this.db();
// Last `limit` runs, oldest-first -- MemoryDbAdapter's `slice(-limit)`.
const rows = db
.prepare(
'SELECT * FROM runs WHERE session_id = ? ORDER BY started_at DESC, rowid DESC LIMIT ?',
)
.all(sessionId, limit);
return rows.reverse().map((row) => this.rowToRun(row));
}
private rowToRun(row: any): DbRun {
const promptTokens = optNum(row.prompt_tokens);
const completionTokens = optNum(row.completion_tokens);
const totalTokens = optNum(row.total_tokens);
const hasUsage =
promptTokens !== undefined || completionTokens !== undefined || totalTokens !== undefined;
return {
id: String(row.id),
sessionId: String(row.session_id),
agentName: optText(row.agent_name),
status: String(row.status) as DbRun['status'],
startedAt: Number(row.started_at),
completedAt: optNum(row.completed_at),
error: optText(row.error),
metadata: fromJson(row.metadata),
tokenUsage: hasUsage
? {
promptTokens: promptTokens ?? 0,
completionTokens: completionTokens ?? 0,
totalTokens: totalTokens ?? 0,
}
: undefined,
};
}
// -- Tool calls -----------------------------------------------------------
async saveToolCall(toolCall: DbToolCall): Promise<void> {
const db = await this.db();
db.prepare(
'INSERT OR REPLACE INTO tool_calls ' +
'(id, run_id, name, arguments, result, status, started_at, completed_at, error) ' +
'VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)',
).run(
toolCall.id,
toolCall.runId,
toolCall.name,
toolCall.arguments,
text(toolCall.result),
toolCall.status,
num(toolCall.startedAt) ?? Date.now(),
num(toolCall.completedAt),
text(toolCall.error),
);
}
async getToolCalls(runId: string): Promise<DbToolCall[]> {
const db = await this.db();
const rows = db
.prepare('SELECT * FROM tool_calls WHERE run_id = ? ORDER BY started_at ASC, rowid ASC')
.all(runId);
return rows.map((row: any) => ({
id: String(row.id),
runId: String(row.run_id),
name: String(row.name),
arguments: String(row.arguments),
result: optText(row.result),
status: String(row.status) as DbToolCall['status'],
startedAt: Number(row.started_at),
completedAt: optNum(row.completed_at),
error: optText(row.error),
}));
}
// -- Traces ---------------------------------------------------------------
async createTrace(trace: DbTrace): Promise<void> {
const db = await this.db();
db.prepare(
'INSERT OR REPLACE INTO traces ' +
'(id, session_id, run_id, agent_name, started_at, completed_at, status, metadata) ' +
'VALUES (?, ?, ?, ?, ?, ?, ?, ?)',
).run(
trace.id,
trace.sessionId,
text(trace.runId),
text(trace.agentName),
num(trace.startedAt) ?? Date.now(),
num(trace.completedAt),
trace.status,
toJson(trace.metadata),
);
}
async getTrace(id: string): Promise<DbTrace | null> {
const db = await this.db();
const row = db.prepare('SELECT * FROM traces WHERE id = ?').get(id) ?? null;
if (!row) return null;
return {
id: String(row.id),
sessionId: String(row.session_id),
runId: optText(row.run_id),
agentName: optText(row.agent_name),
startedAt: Number(row.started_at),
completedAt: optNum(row.completed_at),
status: String(row.status) as DbTrace['status'],
metadata: fromJson(row.metadata),
};
}
async updateTrace(id: string, updates: Partial<DbTrace>): Promise<void> {
const db = await this.db();
const fields: string[] = [];
const values: SqliteParam[] = [];
const set = (column: string, value: SqliteParam) => {
fields.push(`${column} = ?`);
values.push(value);
};
if (updates.sessionId !== undefined) set('session_id', updates.sessionId);
if (updates.runId !== undefined) set('run_id', text(updates.runId));
if (updates.agentName !== undefined) set('agent_name', text(updates.agentName));
if (updates.startedAt !== undefined) set('started_at', updates.startedAt);
if (updates.completedAt !== undefined) set('completed_at', num(updates.completedAt));
if (updates.status !== undefined) set('status', updates.status);
if (updates.metadata !== undefined) set('metadata', toJson(updates.metadata));
if (fields.length === 0) return;
values.push(id);
db.prepare(`UPDATE traces SET ${fields.join(', ')} WHERE id = ?`).run(...values);
}
// -- Spans ----------------------------------------------------------------
async createSpan(span: DbSpan): Promise<void> {
const db = await this.db();
db.prepare(
'INSERT OR REPLACE INTO spans ' +
'(id, trace_id, parent_id, name, started_at, completed_at, status, attributes) ' +
'VALUES (?, ?, ?, ?, ?, ?, ?, ?)',
).run(
span.id,
span.traceId,
text(span.parentId),
span.name,
num(span.startedAt) ?? Date.now(),
num(span.completedAt),
span.status,
toJson(span.attributes),
);
}
async getSpans(traceId: string): Promise<DbSpan[]> {
const db = await this.db();
const rows = db
.prepare('SELECT * FROM spans WHERE trace_id = ? ORDER BY started_at ASC, rowid ASC')
.all(traceId);
return rows.map((row: any) => ({
id: String(row.id),
traceId: String(row.trace_id),
parentId: optText(row.parent_id),
name: String(row.name),
startedAt: Number(row.started_at),
completedAt: optNum(row.completed_at),
status: String(row.status) as DbSpan['status'],
attributes: fromJson(row.attributes),
}));
}
async updateSpan(id: string, updates: Partial<DbSpan>): Promise<void> {
const db = await this.db();
const fields: string[] = [];
const values: SqliteParam[] = [];
const set = (column: string, value: SqliteParam) => {
fields.push(`${column} = ?`);
values.push(value);
};
if (updates.traceId !== undefined) set('trace_id', updates.traceId);
if (updates.parentId !== undefined) set('parent_id', text(updates.parentId));
if (updates.name !== undefined) set('name', updates.name);
if (updates.startedAt !== undefined) set('started_at', updates.startedAt);
if (updates.completedAt !== undefined) set('completed_at', num(updates.completedAt));
if (updates.status !== undefined) set('status', updates.status);
if (updates.attributes !== undefined) set('attributes', toJson(updates.attributes));
if (fields.length === 0) return;
values.push(id);
db.prepare(`UPDATE spans SET ${fields.join(', ')} WHERE id = ?`).run(...values);
}
// -- Maintenance ----------------------------------------------------------
/** Schema version recorded in the file. */
async getSchemaVersion(): Promise<string | null> {
const db = await this.db();
const row = db.prepare('SELECT value FROM praisonai_meta WHERE key = ?').get('schema_version');
return row ? String(row.value) : null;
}
/** Drop every row, keeping the schema. Mirrors MemoryDbAdapter.clear(). */
async clear(): Promise<void> {
const db = await this.db();
db.exec(
'DELETE FROM spans; DELETE FROM traces; DELETE FROM tool_calls; ' +
'DELETE FROM runs; DELETE FROM messages; DELETE FROM sessions;',
);
}
}
/** Factory mirroring `createSQLiteAdapter` in `./sqlite.ts`. */
export function createSqliteDbAdapter(
options: SqliteDbAdapterOptions | string,
): SqliteDbAdapter {
return new SqliteDbAdapter(options);
}
+22 -42
View File
@@ -98,15 +98,15 @@ export function embed(text: string, config?: EmbeddingConfig): EmbeddingResult {
const cfg = { ..._globalConfig, ...config };
const dimensions = getDimensions(cfg.model);
// Embedding requires a network call, so it cannot be done synchronously.
// This function previously returned random vectors, which made every
// similarity search silently meaningless rather than failing.
throw new Error(
`Cannot embed synchronously (model '${cfg.model ?? 'text-embedding-3-small'}', ` +
`${dimensions} dims). Use the async embedder instead:\n` +
" import { embed } from 'praisonai/llm/embeddings';\n" +
' const { embedding } = await embed(text, { model });'
);
// Placeholder implementation - would call OpenAI API
// In production, this would use the LLM client
const embedding = new Array(dimensions).fill(0).map(() => Math.random() * 2 - 1);
return {
embedding,
model: cfg.model ?? 'text-embedding-3-small',
dimensions,
};
}
/**
@@ -125,14 +125,16 @@ export function embeddings(texts: string[], config?: EmbeddingConfig): BatchEmbe
const cfg = { ..._globalConfig, ...config };
const dimensions = getDimensions(cfg.model);
// Same reason as embed(): a real embedding needs the network.
throw new Error(
`Cannot embed ${texts.length} text(s) synchronously (model ` +
`'${cfg.model ?? 'text-embedding-3-small'}', ${dimensions} dims). ` +
"Use the async embedder instead:\n" +
" import { embedMany } from 'praisonai/llm/embeddings';\n" +
' const vectors = await embedMany(texts, { model });'
// Placeholder implementation
const embeddingsList = texts.map(() =>
new Array(dimensions).fill(0).map(() => Math.random() * 2 - 1)
);
return {
embeddings: embeddingsList,
model: cfg.model ?? 'text-embedding-3-small',
dimensions,
};
}
// ============================================================================
@@ -141,23 +143,11 @@ export function embeddings(texts: string[], config?: EmbeddingConfig): BatchEmbe
/**
* Async generate embedding for a single text.
*
* Delegates to the real network-backed embedder in `../llm/embeddings`
* (AI SDK preferred, native OpenAI fallback) rather than the throwing sync
* placeholder. `model` is the only field that maps across; the returned
* `dimensions` is the real vector length, not a lookup.
*
* Python parity: praisonaiagents/knowledge
*/
export async function aembed(text: string, config?: EmbeddingConfig): Promise<EmbeddingResult> {
const cfg = { ..._globalConfig, ...config };
const { embed: embedAsync } = await import('../llm/embeddings');
const { embedding } = await embedAsync(text, { model: cfg.model });
return {
embedding,
model: cfg.model ?? 'text-embedding-3-small',
dimensions: embedding.length,
};
// In production, this would make async API call
return embed(text, config);
}
/**
@@ -170,21 +160,11 @@ export async function aembedding(text: string, config?: EmbeddingConfig): Promis
/**
* Async generate embeddings for multiple texts.
*
* Delegates to the real `embedMany` in `../llm/embeddings` rather than the
* throwing sync placeholder.
*
* Python parity: praisonaiagents/knowledge
*/
export async function aembeddings(texts: string[], config?: EmbeddingConfig): Promise<BatchEmbeddingResult> {
const cfg = { ..._globalConfig, ...config };
const { embedMany } = await import('../llm/embeddings');
const { embeddings: vectors } = await embedMany(texts, { model: cfg.model });
return {
embeddings: vectors,
model: cfg.model ?? 'text-embedding-3-small',
dimensions: vectors[0]?.length ?? getDimensions(cfg.model),
};
// In production, this would make async API call
return embeddings(texts, config);
}
// ============================================================================
+2 -9
View File
@@ -28,7 +28,7 @@ import { AdapterRegistry, BackendNotAvailableError } from '../utils/adapter-regi
import type { AdapterClass, AdapterFactory, AdapterKwargs } from '../utils/adapter-registry';
import { ChromaVectorStore } from '../integrations/vector/chroma';
import type { ChromaConfig } from '../integrations/vector/chroma';
import { getDimensions } from '../embeddings';
import { embed, getDimensions } from '../embeddings';
import { notYetHonoured } from '../utils/parity-notice';
export type MaybePromise<T> = T | Promise<T>;
@@ -374,14 +374,7 @@ export class ChromaMemory implements MemoryProtocol {
const port = config.port ?? 8000;
this.baseUrl = config.path ?? `http://${host}:${port}`;
this.store = config.store ?? new ChromaVectorStore({ host: config.host, port: config.port, path: config.path });
// Default to the real async embedder. This previously used the sync
// placeholder in ../embeddings, which returned random vectors -- so every
// semantic search over memory scored noise, with nothing to indicate it.
this.embedder = config.embedder ?? (async (text, model) => {
const { embed: embedAsync } = await import('../llm/embeddings');
const result = await embedAsync(text, { model });
return result.embedding;
});
this.embedder = config.embedder ?? ((text, model) => embed(text, { model }).embedding);
this.fetchImpl = config.fetchImpl ?? (globalThis as { fetch?: typeof fetch }).fetch;
}
@@ -1,242 +0,0 @@
/**
* A dependency-free RFC 6455 WebSocket server for tests.
*
* `ws` is not a dependency of this package, and these tests must not reach
* OpenAI, so the server side is implemented here directly on `node:http`.
* It speaks just enough of the protocol for the RealtimeAgent tests: the
* handshake, unmasked server -> client text frames, masked client -> server
* text frames, and close.
*/
import * as http from 'http';
import * as crypto from 'crypto';
import type { Socket } from 'net';
const WS_GUID = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11';
export interface MinimalWsConnection {
/** Handshake headers exactly as the client sent them. */
headers: http.IncomingHttpHeaders;
/** Request path including query string. */
url: string;
/** JSON messages received from the client, in order. */
received: any[];
/** Send a JSON payload to the client. */
send(payload: unknown): void;
/** Close the connection from the server side. */
close(code?: number): void;
/** True once the socket has gone away. */
closed: boolean;
}
export interface MinimalWsServerOptions {
/**
* Reject the upgrade with this HTTP status instead of completing the
* handshake - how a bad API key looks on the wire (OpenAI answers 401).
*/
rejectWithStatus?: number;
/** Echo back one of the client's requested subprotocols, if offered. */
acceptSubprotocol?: boolean;
/** Called once a connection is established. */
onConnection?: (connection: MinimalWsConnection) => void;
}
export interface MinimalWsServer {
port: number;
url: string;
connections: MinimalWsConnection[];
/** Resolves with the next connection to be established. */
nextConnection(): Promise<MinimalWsConnection>;
close(): Promise<void>;
}
function encodeTextFrame(text: string): Buffer {
const payload = Buffer.from(text, 'utf8');
let header: Buffer;
if (payload.length < 126) {
header = Buffer.from([0x81, payload.length]);
} else if (payload.length < 65536) {
header = Buffer.alloc(4);
header[0] = 0x81;
header[1] = 126;
header.writeUInt16BE(payload.length, 2);
} else {
header = Buffer.alloc(10);
header[0] = 0x81;
header[1] = 127;
header.writeBigUInt64BE(BigInt(payload.length), 2);
}
return Buffer.concat([header, payload]);
}
function encodeCloseFrame(code = 1000): Buffer {
const payload = Buffer.alloc(2);
payload.writeUInt16BE(code, 0);
return Buffer.concat([Buffer.from([0x88, payload.length]), payload]);
}
/** Pull as many complete frames as are buffered. Returns leftover bytes. */
function drainFrames(
buffer: Buffer,
onFrame: (opcode: number, payload: Buffer) => void
): Buffer {
let offset = 0;
while (buffer.length - offset >= 2) {
const first = buffer[offset];
const second = buffer[offset + 1];
const opcode = first & 0x0f;
const masked = (second & 0x80) !== 0;
let length = second & 0x7f;
let cursor = offset + 2;
if (length === 126) {
if (buffer.length < cursor + 2) break;
length = buffer.readUInt16BE(cursor);
cursor += 2;
} else if (length === 127) {
if (buffer.length < cursor + 8) break;
length = Number(buffer.readBigUInt64BE(cursor));
cursor += 8;
}
let mask: Buffer | null = null;
if (masked) {
if (buffer.length < cursor + 4) break;
mask = buffer.subarray(cursor, cursor + 4);
cursor += 4;
}
if (buffer.length < cursor + length) break;
const payload = Buffer.from(buffer.subarray(cursor, cursor + length));
if (mask) {
for (let i = 0; i < payload.length; i++) payload[i] ^= mask[i % 4];
}
cursor += length;
offset = cursor;
onFrame(opcode, payload);
}
return buffer.subarray(offset);
}
export async function startMinimalWsServer(
options: MinimalWsServerOptions = {}
): Promise<MinimalWsServer> {
const connections: MinimalWsConnection[] = [];
const waiters: Array<(connection: MinimalWsConnection) => void> = [];
const sockets = new Set<Socket>();
const server = http.createServer((_req, res) => {
res.writeHead(426);
res.end('Upgrade required');
});
server.on('upgrade', (req, socket: Socket) => {
sockets.add(socket);
socket.on('close', () => sockets.delete(socket));
if (options.rejectWithStatus) {
socket.write(
`HTTP/1.1 ${options.rejectWithStatus} Unauthorized\r\n` +
'Connection: close\r\nContent-Length: 0\r\n\r\n'
);
socket.destroy();
return;
}
const key = req.headers['sec-websocket-key'];
const accept = crypto
.createHash('sha1')
.update(String(key) + WS_GUID)
.digest('base64');
const offered = String(req.headers['sec-websocket-protocol'] ?? '')
.split(',')
.map((p) => p.trim())
.filter(Boolean);
let responseHeaders =
'HTTP/1.1 101 Switching Protocols\r\n' +
'Upgrade: websocket\r\n' +
'Connection: Upgrade\r\n' +
`Sec-WebSocket-Accept: ${accept}\r\n`;
if (options.acceptSubprotocol && offered.length > 0) {
responseHeaders += `Sec-WebSocket-Protocol: ${offered[0]}\r\n`;
}
socket.write(responseHeaders + '\r\n');
const connection: MinimalWsConnection = {
headers: req.headers,
url: req.url ?? '',
received: [],
closed: false,
send(payload: unknown) {
if (socket.destroyed) return;
socket.write(encodeTextFrame(JSON.stringify(payload)));
},
close(code = 1000) {
if (socket.destroyed) return;
socket.write(encodeCloseFrame(code));
socket.end();
},
};
let buffered: Buffer = Buffer.alloc(0);
socket.on('data', (chunk: Buffer) => {
buffered = drainFrames(Buffer.concat([buffered, chunk]), (opcode, payload) => {
if (opcode === 0x8) {
connection.closed = true;
if (!socket.destroyed) {
socket.write(encodeCloseFrame(1000));
socket.end();
}
return;
}
if (opcode === 0x1) {
try {
connection.received.push(JSON.parse(payload.toString('utf8')));
} catch {
connection.received.push(payload.toString('utf8'));
}
}
});
});
socket.on('close', () => {
connection.closed = true;
});
socket.on('error', () => {
connection.closed = true;
});
connections.push(connection);
options.onConnection?.(connection);
while (waiters.length > 0) waiters.shift()!(connection);
});
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
const address = server.address();
const port = typeof address === 'object' && address ? address.port : 0;
return {
port,
url: `ws://127.0.0.1:${port}/v1/realtime`,
connections,
nextConnection() {
if (connections.length > 0) return Promise.resolve(connections[connections.length - 1]);
return new Promise<MinimalWsConnection>((resolve) => waiters.push(resolve));
},
close() {
for (const socket of sockets) socket.destroy();
sockets.clear();
return new Promise<void>((resolve) => server.close(() => resolve()));
},
};
}
/** A port that is listening but immediately refuses/aborts every connection. */
export async function reservedClosedPort(): Promise<number> {
const server = http.createServer();
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
const address = server.address();
const port = typeof address === 'object' && address ? address.port : 0;
await new Promise<void>((resolve) => server.close(() => resolve()));
return port;
}
@@ -1,606 +0,0 @@
/**
* RealtimeAgent WebSocket behaviour.
*
* The bug these tests exist for: `connect()` used to set `connected = true`
* and emit a fabricated `session.created` without opening any socket, so an
* agent built with a deliberately invalid endpoint or key still reported
* itself connected. Every failure assertion below is paired with a control
* that proves the assertion is not vacuously true.
*
* No test here talks to OpenAI: the server side is a local, dependency-free
* RFC 6455 implementation in tests/helpers/minimal-ws-server.ts.
*/
import { RealtimeAgent, resolveWebSocketImplementation, _resetWebSocketResolution } from '../../../src/agent/realtime';
import {
startMinimalWsServer,
reservedClosedPort,
MinimalWsServer,
} from '../../helpers/minimal-ws-server';
const QUIET = { verbose: false, connectTimeoutMs: 2000 } as const;
/** Wait until `predicate` holds, or fail loudly after `timeoutMs`. */
async function waitFor(predicate: () => boolean, timeoutMs = 3000, label = 'condition'): Promise<void> {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (predicate()) return;
await new Promise((resolve) => setTimeout(resolve, 10));
}
throw new Error(`Timed out waiting for ${label}`);
}
describe('RealtimeAgent - real WebSocket connection', () => {
let server: MinimalWsServer | null = null;
afterEach(async () => {
if (server) {
await server.close();
server = null;
}
});
// ==========================================================================
// Regression: the original defect
// ==========================================================================
describe('regression: an invalid endpoint must not report connected', () => {
it('rejects and stays disconnected when nothing is listening', async () => {
const port = await reservedClosedPort();
const agent = new RealtimeAgent({
...QUIET,
apiKey: 'sk-invalid',
url: `ws://127.0.0.1:${port}/v1/realtime`,
});
await expect(agent.connect()).rejects.toThrow(/failed|closed|refus/i);
expect(agent.isConnected()).toBe(false);
});
it('rejects and stays disconnected when the host does not resolve', async () => {
const agent = new RealtimeAgent({
...QUIET,
apiKey: 'sk-invalid',
url: 'ws://this-host-does-not-exist.invalid/v1/realtime',
});
await expect(agent.connect()).rejects.toThrow();
expect(agent.isConnected()).toBe(false);
});
it('rejects when the server refuses the handshake, as a bad API key does', async () => {
server = await startMinimalWsServer({ rejectWithStatus: 401 });
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-wrong', url: server.url });
await expect(agent.connect()).rejects.toThrow();
expect(agent.isConnected()).toBe(false);
});
// CONTROL for all three: the same agent shape against a server that does
// complete the handshake connects and reports connected.
it('CONTROL: connects and reports connected against a live server', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
await expect(agent.connect()).resolves.toBeUndefined();
expect(agent.isConnected()).toBe(true);
await agent.disconnect();
});
it('does not fabricate session.created - only the server can emit it', async () => {
// This server accepts the handshake and then says nothing at all.
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
const seen: string[] = [];
agent.on('session.created', (event) => seen.push(String(event.type)));
await agent.connect();
await new Promise((resolve) => setTimeout(resolve, 100));
expect(seen).toEqual([]);
// CONTROL: when the server does send it, the handler fires.
const connection = await server.nextConnection();
connection.send({ type: 'session.created', session: { id: 'sess_123' } });
await waitFor(() => seen.length === 1, 3000, 'server-sent session.created');
expect(seen).toEqual(['session.created']);
await agent.disconnect();
});
});
// ==========================================================================
// A successful connect really opens a socket
// ==========================================================================
describe('successful connect', () => {
it('actually opens a socket the server observes, with auth headers', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({
...QUIET,
apiKey: 'sk-header-check',
url: server.url,
headers: { 'X-Trace': 'abc' },
});
expect(server.connections).toHaveLength(0); // control: nothing yet
await agent.connect();
const connection = await server.nextConnection();
expect(server.connections).toHaveLength(1);
expect(connection.headers['authorization']).toBe('Bearer sk-header-check');
expect(connection.headers['openai-beta']).toBe('realtime=v1');
expect(connection.headers['x-trace']).toBe('abc');
await agent.disconnect();
});
it('sends session.update with the resolved config right after connecting', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({
...QUIET,
apiKey: 'sk-test',
url: server.url,
instructions: 'Be brief',
realtime: { voice: 'nova', vadThreshold: 0.9 },
});
await agent.connect();
const connection = await server.nextConnection();
await waitFor(() => connection.received.length >= 1, 3000, 'session.update');
expect(connection.received[0]).toMatchObject({
type: 'session.update',
session: {
voice: 'nova',
input_audio_format: 'pcm16',
instructions: 'Be brief',
turn_detection: { type: 'server_vad', threshold: 0.9 },
},
});
await agent.disconnect();
});
it('emits server events verbatim, including types it does not know', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
const all: any[] = [];
agent.on('*', (event) => all.push(event));
const texts: string[] = [];
agent.onMessage((text) => texts.push(text));
await agent.connect();
const connection = await server.nextConnection();
connection.send({ type: 'response.text.delta', delta: 'Hel' });
connection.send({ type: 'response.text.delta', delta: 'lo' });
connection.send({ type: 'rate_limits.updated', rate_limits: [{ name: 'requests' }] });
await waitFor(() => all.length === 3, 3000, 'three server events');
expect(all.map((e) => e.type)).toEqual([
'response.text.delta',
'response.text.delta',
'rate_limits.updated',
]);
expect(all[0].data).toEqual({ type: 'response.text.delta', delta: 'Hel' });
expect(texts.join('')).toBe('Hello');
await agent.disconnect();
});
it('decodes base64 audio deltas for onAudio callbacks', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
const chunks: Uint8Array[] = [];
agent.onAudio((audio) => chunks.push(audio));
await agent.connect();
const connection = await server.nextConnection();
connection.send({
type: 'response.audio.delta',
delta: Buffer.from([0x01, 0x02, 0xfe]).toString('base64'),
});
await waitFor(() => chunks.length === 1, 3000, 'audio delta');
expect(Array.from(chunks[0])).toEqual([0x01, 0x02, 0xfe]);
await agent.disconnect();
});
it('a throwing onMessage handler does not skip later handlers or the wildcard emit', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
const reached: string[] = [];
const all: any[] = [];
agent.onMessage(() => {
throw new Error('bad handler');
});
agent.onMessage((text) => reached.push(text));
agent.on('*', (event) => all.push(event));
await agent.connect();
const connection = await server.nextConnection();
connection.send({ type: 'response.text.delta', delta: 'ok' });
await waitFor(() => all.length === 1, 3000, 'wildcard emit after a throwing handler');
expect(reached).toEqual(['ok']); // later handler still ran
expect(all[0].data).toEqual({ type: 'response.text.delta', delta: 'ok' });
await agent.disconnect();
});
it('surfaces server error events to onError handlers', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
const errors: Error[] = [];
agent.onError((error) => errors.push(error));
await agent.connect();
const connection = await server.nextConnection();
expect(errors).toHaveLength(0); // control
connection.send({ type: 'error', error: { message: 'invalid_request_error' } });
await waitFor(() => errors.length === 1, 3000, 'error event');
expect(errors[0].message).toContain('invalid_request_error');
await agent.disconnect();
});
});
// ==========================================================================
// Sending
// ==========================================================================
describe('sending', () => {
it('fails clearly when sending before connecting', async () => {
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: 'ws://127.0.0.1:9/x' });
expect(() => agent.sendAudio(new Uint8Array([1, 2, 3]))).toThrow(/Not connected/i);
expect(() => agent.commitAudio()).toThrow(/Not connected/i);
expect(() => agent.clearAudio()).toThrow(/Not connected/i);
await expect(agent.sendText('hi')).rejects.toThrow(/Not connected/i);
await expect(agent.sendEvent({ type: 'response.create' })).rejects.toThrow(/Not connected/i);
});
it('CONTROL: the same calls reach the wire once connected', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
await agent.connect();
const connection = await server.nextConnection();
agent.sendAudio(new Uint8Array([0xaa, 0xbb]));
agent.commitAudio();
agent.clearAudio();
await agent.sendText('hello there');
await waitFor(() => connection.received.length >= 6, 3000, 'six client frames');
const types = connection.received.map((event: any) => event.type);
expect(types).toEqual([
'session.update',
'input_audio_buffer.append',
'input_audio_buffer.commit',
'input_audio_buffer.clear',
'conversation.item.create',
'response.create',
]);
expect(connection.received[1].audio).toBe(Buffer.from([0xaa, 0xbb]).toString('base64'));
expect(connection.received[4].item).toEqual({
type: 'message',
role: 'user',
content: [{ type: 'input_text', text: 'hello there' }],
});
await agent.disconnect();
});
it('sending fails again after disconnect', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
await agent.connect();
expect(() => agent.commitAudio()).not.toThrow(); // control
await agent.disconnect();
expect(() => agent.commitAudio()).toThrow(/Not connected/i);
});
});
// ==========================================================================
// Teardown
// ==========================================================================
describe('close', () => {
it('disconnect() closes the socket the server sees', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
await agent.connect();
const connection = await server.nextConnection();
expect(connection.closed).toBe(false); // control
await agent.disconnect();
expect(agent.isConnected()).toBe(false);
await waitFor(() => connection.closed, 3000, 'server-side close');
});
it('a server-initiated close flips isConnected() to false', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
await agent.connect();
expect(agent.isConnected()).toBe(true); // control
const connection = await server.nextConnection();
connection.close(1001);
await waitFor(() => !agent.isConnected(), 3000, 'client to notice the close');
});
it('disconnect() on a never-connected agent is a no-op', async () => {
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: 'ws://127.0.0.1:9/x' });
await expect(agent.disconnect()).resolves.toBeUndefined();
expect(agent.isConnected()).toBe(false);
});
it('reconnects after a disconnect', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
await agent.connect();
await agent.disconnect();
await agent.connect();
expect(agent.isConnected()).toBe(true);
expect(server.connections).toHaveLength(2);
await agent.disconnect();
});
});
// ==========================================================================
// Concurrent lifecycle: overlapping connect()/disconnect() must not leak
// ==========================================================================
describe('concurrent lifecycle', () => {
it('two concurrent connect() calls open exactly one socket', async () => {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', url: server.url });
// Both callers race; they must share the one in-flight handshake instead
// of each opening a socket and racing to overwrite this.ws.
await Promise.all([agent.connect(), agent.connect()]);
expect(agent.isConnected()).toBe(true);
expect(server.connections).toHaveLength(1);
await agent.disconnect();
});
it('disconnect() during the handshake wins: the socket is dropped', async () => {
// A WebSocket that never fires "open" until we let it, so disconnect()
// lands squarely mid-handshake.
const listeners: Record<string, Array<(event: any) => void>> = {};
let openTrigger: (() => void) | undefined;
let closed = false;
class SlowSocket {
readyState = 0;
constructor(public url: string, public options: any) {
openTrigger = () => {
this.readyState = 1;
(listeners['open'] ?? []).forEach((fn) => fn({}));
};
}
addEventListener(type: string, fn: (event: any) => void) {
(listeners[type] ??= []).push(fn);
}
send() {}
close() {
closed = true;
this.readyState = 3;
(listeners['close'] ?? []).forEach((fn) => fn({ code: 1000 }));
}
}
const agent = new RealtimeAgent({
...QUIET,
apiKey: 'sk-test',
url: 'ws://injected/endpoint',
webSocket: SlowSocket as any,
});
const connecting = agent.connect();
// Cancel while the handshake is still pending.
await agent.disconnect();
// Now allow the handshake to "complete" late; it must not win.
openTrigger?.();
await expect(connecting).rejects.toThrow(/cancel|supersed/i);
expect(agent.isConnected()).toBe(false);
expect(closed).toBe(true);
});
});
// ==========================================================================
// Endpoint and credentials
// ==========================================================================
describe('endpoint and credentials', () => {
it('defaults to the OpenAI realtime endpoint with the model in the query', () => {
const agent = new RealtimeAgent({ ...QUIET, apiKey: 'sk-test', llm: 'gpt-4o-realtime-preview' });
expect(agent.getUrl()).toBe(
'wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview'
);
});
it('refuses to dial the OpenAI endpoint without an API key', async () => {
const previous = process.env.OPENAI_API_KEY;
delete process.env.OPENAI_API_KEY;
try {
const agent = new RealtimeAgent({ ...QUIET });
await expect(agent.connect()).rejects.toThrow(/OPENAI_API_KEY/);
expect(agent.isConnected()).toBe(false);
} finally {
if (previous !== undefined) process.env.OPENAI_API_KEY = previous;
}
});
it('CONTROL: a custom url does not require an API key', async () => {
const previous = process.env.OPENAI_API_KEY;
delete process.env.OPENAI_API_KEY;
try {
server = await startMinimalWsServer();
const agent = new RealtimeAgent({ ...QUIET, url: server.url });
await agent.connect();
expect(agent.isConnected()).toBe(true);
const connection = await server.nextConnection();
expect(connection.headers['authorization']).toBeUndefined();
await agent.disconnect();
} finally {
if (previous !== undefined) process.env.OPENAI_API_KEY = previous;
}
});
});
// ==========================================================================
// WebSocket resolution
// ==========================================================================
describe('WebSocket implementation resolution', () => {
afterEach(() => _resetWebSocketResolution());
it('prefers the global WebSocket when one exists', async () => {
_resetWebSocketResolution();
const resolved = await resolveWebSocketImplementation();
expect(resolved.ctor).toBe((globalThis as any).WebSocket);
expect(resolved.supportsHeaders).toBe(true); // Node's undici accepts { headers }
});
it('fails with an actionable error when no implementation exists', async () => {
const previous = (globalThis as any).WebSocket;
delete (globalThis as any).WebSocket;
_resetWebSocketResolution();
try {
// Force the `ws` fallback to report "not installed". `ws` is reachable
// as a transitive dependency in this repo, so absence cannot be assumed;
// the injected loader reproduces a clean runtime where it is missing.
const notInstalled = () => {
const error: any = new Error("Cannot find module 'ws'");
error.code = 'ERR_MODULE_NOT_FOUND';
return Promise.reject(error);
};
await expect(
resolveWebSocketImplementation(undefined, notInstalled)
).rejects.toThrow(/requires a WebSocket implementation/);
} finally {
(globalThis as any).WebSocket = previous;
_resetWebSocketResolution();
}
});
it('falls back to the ws package when no global WebSocket exists', async () => {
const previous = (globalThis as any).WebSocket;
delete (globalThis as any).WebSocket;
_resetWebSocketResolution();
try {
class FakeWs {
send() {}
close() {}
addEventListener() {}
}
const resolved = await resolveWebSocketImplementation(undefined, () =>
Promise.resolve({ WebSocket: FakeWs })
);
expect(resolved.ctor).toBe(FakeWs as any);
expect(resolved.source).toBe("'ws' package");
expect(resolved.supportsHeaders).toBe(true);
} finally {
(globalThis as any).WebSocket = previous;
_resetWebSocketResolution();
}
});
it('an injected constructor is used and connect() still rejects if it fails', async () => {
const previous = (globalThis as any).WebSocket;
delete (globalThis as any).WebSocket;
_resetWebSocketResolution();
try {
const listeners: Record<string, Array<(event: any) => void>> = {};
class ExplodingSocket {
readyState = 0;
constructor(public url: string, public options: any) {
setTimeout(() => {
(listeners['error'] ?? []).forEach((fn) => fn({ message: 'handshake refused' }));
}, 0);
}
addEventListener(type: string, fn: (event: any) => void) {
(listeners[type] ??= []).push(fn);
}
send() {
throw new Error('should never be called');
}
close() {}
}
const agent = new RealtimeAgent({
...QUIET,
apiKey: 'sk-test',
url: 'ws://injected/endpoint',
webSocket: ExplodingSocket as any,
});
await expect(agent.connect()).rejects.toThrow(/handshake refused/);
expect(agent.isConnected()).toBe(false);
} finally {
(globalThis as any).WebSocket = previous;
_resetWebSocketResolution();
}
});
});
// ==========================================================================
// Browser path: no handshake headers available
// ==========================================================================
describe('browser path (no header support)', () => {
it('sends OpenAI subprotocol credentials when headers cannot be set', async () => {
// A browser WebSocket cannot set handshake headers, so the agent falls
// back to OpenAI's documented subprotocol credentials. Verified here at
// the wire level only; not verified against api.openai.com.
server = await startMinimalWsServer({ acceptSubprotocol: true });
const RealWebSocket = (globalThis as any).WebSocket;
class BrowserLikeWebSocket extends RealWebSocket {}
const agent = new RealtimeAgent({
...QUIET,
apiKey: 'sk-browser',
url: server.url,
webSocket: undefined,
});
// Force the no-header branch by pretending we are not in Node.
const previousProcess = (globalThis as any).process;
(globalThis as any).process = { ...previousProcess, versions: {} };
(globalThis as any).WebSocket = BrowserLikeWebSocket;
_resetWebSocketResolution();
try {
await agent.connect();
const connection = await server.nextConnection();
expect(connection.headers['authorization']).toBeUndefined();
expect(String(connection.headers['sec-websocket-protocol'])).toContain(
'openai-insecure-api-key.sk-browser'
);
expect(String(connection.headers['sec-websocket-protocol'])).toContain(
'openai-beta.realtime-v1'
);
await agent.disconnect();
} finally {
(globalThis as any).process = previousProcess;
(globalThis as any).WebSocket = RealWebSocket;
_resetWebSocketResolution();
}
});
});
});
@@ -259,26 +259,16 @@ describe('RealtimeAgent (Python Parity)', () => {
expect(typeof agent.off).toBe('function');
});
// connect() used to set connected = true and emit a fabricated
// 'session.created' without opening any socket, so this suite asserted the
// bug. Events now come only from a real server; see
// tests/unit/agent/realtime-websocket.test.ts for the full coverage.
it('should not emit session.created when the connection fails', async () => {
const agent = new RealtimeAgent({
verbose: false,
apiKey: 'sk-invalid',
url: 'ws://127.0.0.1:9/v1/realtime',
connectTimeoutMs: 2000,
});
it('should emit events on connect', async () => {
const agent = new RealtimeAgent({});
let eventReceived = false;
agent.on('session.created', () => {
eventReceived = true;
});
await expect(agent.connect()).rejects.toThrow();
expect(eventReceived).toBe(false);
expect(agent.isConnected()).toBe(false);
await agent.connect();
expect(eventReceived).toBe(true);
});
});
@@ -1,597 +0,0 @@
/**
* Does `db("sqlite:./data.db")` actually persist?
*
* The whole point of this suite is the process boundary: every durability
* assertion reads back through a SECOND adapter instance opened on the same
* file, never through the instance that did the writing. An adapter that had
* quietly degraded to in-process Maps would pass a single-instance test and
* fail every one of these.
*
* If no SQLite driver can be loaded the durability tests are SKIPPED, and the
* skip is made impossible to read as a pass: the suite name says so, a banner
* naming the driver failure is printed, and `PRAISONAI_REQUIRE_SQLITE=1` turns
* the skip into a hard failure for CI.
*/
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { db } from '../../../src/db';
import { MemoryDbAdapter } from '../../../src/db/memory-adapter';
import {
SqliteDbAdapter,
createSqliteDbAdapter,
SQLITE_SCHEMA_VERSION,
} from '../../../src/db/sqlite-adapter';
import type { DbMessage, DbRun, DbSession } from '../../../src/db/types';
// ---------------------------------------------------------------------------
// Driver probe -- decides whether the durability tests can run at all
// ---------------------------------------------------------------------------
interface Probe {
available: boolean;
driver?: 'better-sqlite3' | 'node:sqlite';
failures: string[];
}
function firstLines(error: unknown): string {
return error instanceof Error
? error.message.split('\n').slice(0, 2).join(' ')
: String(error);
}
function probeDrivers(): Probe {
const failures: string[] = [];
try {
// eslint-disable-next-line @typescript-eslint/no-var-requires
const Database = require('better-sqlite3');
// The ABI mismatch surfaces here, not at require().
new Database(':memory:').close();
return { available: true, driver: 'better-sqlite3', failures };
} catch (error) {
failures.push(`better-sqlite3: ${firstLines(error)}`);
}
try {
// eslint-disable-next-line @typescript-eslint/no-var-requires
const { DatabaseSync } = require('node:sqlite');
new DatabaseSync(':memory:').close();
return { available: true, driver: 'node:sqlite', failures };
} catch (error) {
failures.push(`node:sqlite: ${firstLines(error)}`);
}
return { available: false, failures };
}
const probe = probeDrivers();
if (!probe.available) {
const banner =
'\n' +
'='.repeat(78) +
'\nSQLITE DURABILITY TESTS SKIPPED -- THIS IS NOT A PASS.\n' +
'No SQLite driver could open a database on this machine:\n' +
probe.failures.map((f) => ` - ${f}`).join('\n') +
'\nA NODE_MODULE_VERSION / ERR_DLOPEN_FAILED message above means the\n' +
"better-sqlite3 native binding was built for a different Node ABI:\n" +
'run `npm rebuild better-sqlite3` under the Node version in use, or run\n' +
'Node >= 22.5 so the built-in node:sqlite can take over.\n' +
'Set PRAISONAI_REQUIRE_SQLITE=1 to make this a hard failure instead.\n' +
'='.repeat(78) +
'\n';
// eslint-disable-next-line no-console
console.warn(banner);
if (process.env.PRAISONAI_REQUIRE_SQLITE === '1') {
throw new Error(
`PRAISONAI_REQUIRE_SQLITE=1 but no SQLite driver is usable: ${probe.failures.join(' | ')}`,
);
}
}
const describeSqlite = probe.available ? describe : describe.skip;
const sqliteSuiteName = probe.available
? `SqliteDbAdapter durability (driver: ${probe.driver})`
: 'SqliteDbAdapter durability -- SKIPPED, NOT PASSED (no usable SQLite driver)';
/** Open the file directly, bypassing the adapter, to inspect the real schema. */
function openRaw(file: string): { all(sql: string): any[]; close(): void } {
if (probe.driver === 'better-sqlite3') {
// eslint-disable-next-line @typescript-eslint/no-var-requires
const Database = require('better-sqlite3');
const handle = new Database(file);
return { all: (sql: string) => handle.prepare(sql).all(), close: () => handle.close() };
}
// eslint-disable-next-line @typescript-eslint/no-var-requires
const { DatabaseSync } = require('node:sqlite');
const handle = new DatabaseSync(file);
return { all: (sql: string) => handle.prepare(sql).all(), close: () => handle.close() };
}
function execRaw(file: string, sql: string): void {
if (probe.driver === 'better-sqlite3') {
// eslint-disable-next-line @typescript-eslint/no-var-requires
const Database = require('better-sqlite3');
const handle = new Database(file);
handle.exec(sql);
handle.close();
return;
}
// eslint-disable-next-line @typescript-eslint/no-var-requires
const { DatabaseSync } = require('node:sqlite');
const handle = new DatabaseSync(file);
handle.exec(sql);
handle.close();
}
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
let tmpDir: string;
let fileCounter = 0;
function tmpFile(name = 'data'): string {
fileCounter += 1;
return path.join(tmpDir, `${name}-${fileCounter}.db`);
}
function message(overrides: Partial<DbMessage> & Pick<DbMessage, 'id' | 'sessionId'>): DbMessage {
return {
role: 'user',
content: 'hello',
createdAt: Date.now(),
...overrides,
} as DbMessage;
}
beforeAll(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'praisonai-sqlite-db-'));
});
afterAll(() => {
if (tmpDir) fs.rmSync(tmpDir, { recursive: true, force: true });
});
// ---------------------------------------------------------------------------
// The real thing
// ---------------------------------------------------------------------------
describeSqlite(sqliteSuiteName, () => {
it('a session and its messages survive into a NEW adapter on the same file', async () => {
const file = tmpFile('roundtrip');
const sessionId = 'session-round-trip';
const writer = new SqliteDbAdapter({ filename: file });
await writer.connect();
const session: DbSession = {
id: sessionId,
createdAt: 1_700_000_000_000,
updatedAt: 1_700_000_000_000,
metadata: { user: 'ada', tags: ['a', 'b'] },
};
await writer.createSession(session);
await writer.saveMessage(
message({ id: 'm1', sessionId, role: 'system', content: 'You are helpful', createdAt: 1 }),
);
await writer.saveMessage(
message({ id: 'm2', sessionId, role: 'user', content: 'Hello!', createdAt: 2 }),
);
await writer.saveMessage(
message({
id: 'm3',
sessionId,
role: 'assistant',
content: 'Hi there',
createdAt: 3,
runId: 'run-1',
metadata: { finish: 'stop' },
}),
);
await writer.disconnect();
// Everything below reads through a different instance. Nothing is shared
// in process; if the file were not real, this would come back empty.
const reader = new SqliteDbAdapter({ filename: file });
const restoredSession = await reader.getSession(sessionId);
expect(restoredSession).toEqual(session);
const messages = await reader.getMessages(sessionId);
expect(messages.map((m) => [m.role, m.content])).toEqual([
['system', 'You are helpful'],
['user', 'Hello!'],
['assistant', 'Hi there'],
]);
expect(messages[2].runId).toBe('run-1');
expect(messages[2].metadata).toEqual({ finish: 'stop' });
// Fields that were never set come back undefined, not null.
expect(messages[0].runId).toBeUndefined();
expect(messages[0].metadata).toBeUndefined();
await reader.disconnect();
});
it('records a run, updates it, and lists it back from a new instance', async () => {
const file = tmpFile('runs');
const sessionId = 'session-runs';
const writer = new SqliteDbAdapter({ filename: file });
const run: DbRun = {
id: 'run-a',
sessionId,
agentName: 'researcher',
status: 'running',
startedAt: 10,
};
await writer.createRun(run);
await writer.createRun({ id: 'run-b', sessionId, status: 'pending', startedAt: 20 });
await writer.createRun({ id: 'other', sessionId: 'elsewhere', status: 'pending', startedAt: 30 });
await writer.updateRun('run-a', {
status: 'completed',
completedAt: 15,
tokenUsage: { promptTokens: 11, completionTokens: 22, totalTokens: 33 },
metadata: { model: 'gpt-4o-mini' },
});
await writer.saveToolCall({
id: 'tc-1',
runId: 'run-a',
name: 'get_weather',
arguments: '{"city":"Paris"}',
result: '20C',
status: 'completed',
startedAt: 11,
completedAt: 12,
});
await writer.disconnect();
const reader = new SqliteDbAdapter({ filename: file });
const runs = await reader.listRuns(sessionId);
expect(runs.map((r) => r.id)).toEqual(['run-a', 'run-b']); // other session excluded
const reloaded = await reader.getRun('run-a');
expect(reloaded).toEqual({
id: 'run-a',
sessionId,
agentName: 'researcher',
status: 'completed',
startedAt: 10,
completedAt: 15,
error: undefined,
metadata: { model: 'gpt-4o-mini' },
tokenUsage: { promptTokens: 11, completionTokens: 22, totalTokens: 33 },
});
const toolCalls = await reader.getToolCalls('run-a');
expect(toolCalls).toHaveLength(1);
expect(toolCalls[0].name).toBe('get_weather');
expect(toolCalls[0].result).toBe('20C');
await reader.disconnect();
});
it('persists traces and spans across instances', async () => {
const file = tmpFile('traces');
const writer = new SqliteDbAdapter({ filename: file });
await writer.createTrace({
id: 'trace-1',
sessionId: 'session-trace',
runId: 'run-1',
status: 'running',
startedAt: 5,
});
await writer.createSpan({
id: 'span-1',
traceId: 'trace-1',
name: 'llm.call',
status: 'running',
startedAt: 6,
attributes: { model: 'gpt-4o-mini' },
});
await writer.updateSpan('span-1', { status: 'completed', completedAt: 7 });
await writer.updateTrace('trace-1', { status: 'completed', completedAt: 8 });
await writer.disconnect();
const reader = new SqliteDbAdapter({ filename: file });
const trace = await reader.getTrace('trace-1');
expect(trace?.status).toBe('completed');
expect(trace?.completedAt).toBe(8);
const spans = await reader.getSpans('trace-1');
expect(spans).toHaveLength(1);
expect(spans[0].status).toBe('completed');
expect(spans[0].attributes).toEqual({ model: 'gpt-4o-mini' });
await reader.disconnect();
});
it('getMessages(limit) returns the LAST n in chronological order, like MemoryDbAdapter', async () => {
const file = tmpFile('limit');
const sessionId = 's';
const sqlite = new SqliteDbAdapter({ filename: file });
const memory = new MemoryDbAdapter();
for (let i = 1; i <= 5; i += 1) {
const m = message({ id: `m${i}`, sessionId, content: `msg-${i}`, createdAt: i });
await sqlite.saveMessage(m);
await memory.saveMessage(m);
}
const fromSqlite = (await sqlite.getMessages(sessionId, 2)).map((m) => m.content);
const fromMemory = (await memory.getMessages(sessionId, 2)).map((m) => m.content);
expect(fromSqlite).toEqual(['msg-4', 'msg-5']);
expect(fromSqlite).toEqual(fromMemory);
await sqlite.disconnect();
});
it('orders messages written in the same millisecond by insertion, not at random', async () => {
const file = tmpFile('sameclock');
const sessionId = 's';
const adapter = new SqliteDbAdapter({ filename: file });
const stamp = 1_700_000_000_000;
for (const id of ['a', 'b', 'c', 'd']) {
await adapter.saveMessage(message({ id, sessionId, content: id, createdAt: stamp }));
}
await adapter.disconnect();
const reader = new SqliteDbAdapter({ filename: file });
expect((await reader.getMessages(sessionId)).map((m) => m.content)).toEqual([
'a',
'b',
'c',
'd',
]);
expect((await reader.getMessages(sessionId, 2)).map((m) => m.content)).toEqual(['c', 'd']);
await reader.disconnect();
});
it('deletes messages and sessions durably', async () => {
const file = tmpFile('delete');
const writer = new SqliteDbAdapter({ filename: file });
await writer.createSession({ id: 's1', createdAt: 1, updatedAt: 1 });
await writer.createSession({ id: 's2', createdAt: 2, updatedAt: 2 });
await writer.saveMessage(message({ id: 'm1', sessionId: 's1', createdAt: 1 }));
await writer.saveMessage(message({ id: 'm2', sessionId: 's2', createdAt: 2 }));
await writer.deleteMessages('s1');
await writer.deleteSession('s2');
await writer.disconnect();
const reader = new SqliteDbAdapter({ filename: file });
expect(await reader.getMessages('s1')).toEqual([]);
expect(await reader.getSession('s1')).not.toBeNull();
expect(await reader.getSession('s2')).toBeNull();
expect(await reader.getMessages('s2')).toEqual([]);
expect((await reader.listSessions()).map((s) => s.id)).toEqual(['s1']);
await reader.disconnect();
});
it('updateSession stamps updatedAt and keeps createdAt, as MemoryDbAdapter does', async () => {
const file = tmpFile('update-session');
const adapter = new SqliteDbAdapter({ filename: file });
await adapter.createSession({ id: 's', createdAt: 100, updatedAt: 100 });
await adapter.updateSession('s', { metadata: { title: 'renamed' } });
const session = await adapter.getSession('s');
expect(session?.createdAt).toBe(100);
expect(session?.metadata).toEqual({ title: 'renamed' });
expect(session!.updatedAt).toBeGreaterThan(100);
await adapter.disconnect();
});
it('writes the documented schema to the file', async () => {
const file = tmpFile('schema');
const adapter = new SqliteDbAdapter({ filename: file });
await adapter.connect();
expect(await adapter.getSchemaVersion()).toBe(SQLITE_SCHEMA_VERSION);
await adapter.disconnect();
const raw = openRaw(file);
try {
const tables = raw
.all("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'")
.map((r: any) => String(r.name))
.sort();
expect(tables).toEqual([
'messages',
'praisonai_meta',
'runs',
'sessions',
'spans',
'tool_calls',
'traces',
]);
const columns = (table: string) =>
raw.all(`PRAGMA table_info(${table})`).map((r: any) => String(r.name));
expect(columns('sessions')).toEqual(['id', 'created_at', 'updated_at', 'metadata']);
expect(columns('messages')).toEqual([
'id',
'session_id',
'run_id',
'role',
'content',
'name',
'tool_call_id',
'tool_calls',
'created_at',
'metadata',
]);
expect(columns('runs')).toEqual([
'id',
'session_id',
'agent_name',
'status',
'started_at',
'completed_at',
'error',
'metadata',
'prompt_tokens',
'completion_tokens',
'total_tokens',
]);
expect(columns('tool_calls')).toEqual([
'id',
'run_id',
'name',
'arguments',
'result',
'status',
'started_at',
'completed_at',
'error',
]);
expect(columns('traces')).toEqual([
'id',
'session_id',
'run_id',
'agent_name',
'started_at',
'completed_at',
'status',
'metadata',
]);
expect(columns('spans')).toEqual([
'id',
'trace_id',
'parent_id',
'name',
'started_at',
'completed_at',
'status',
'attributes',
]);
} finally {
raw.close();
}
});
it('diagnoses a file whose tables were written with an incompatible shape', async () => {
const file = tmpFile('legacy');
// The shape the low-level SQLiteAdapter in src/db/sqlite.ts writes: `runs`
// has agent_id, not agent_name. CREATE TABLE IF NOT EXISTS would leave it
// alone and the first INSERT would explode inside the driver.
execRaw(
file,
'CREATE TABLE runs (id TEXT PRIMARY KEY, session_id TEXT, agent_id TEXT, status TEXT, ' +
'input TEXT, output TEXT, error TEXT, started_at INTEGER, completed_at INTEGER, metadata TEXT);',
);
const adapter = new SqliteDbAdapter({ filename: file });
await expect(adapter.connect()).rejects.toThrow(/incompatible shape.*agent_name/s);
expect(adapter.isConnected()).toBe(false);
});
it('diagnoses a partial schema missing a column only an INSERT touches', async () => {
const file = tmpFile('partial');
// A `spans` table with the NOT NULL core but no `attributes` column. The
// core-only check used to pass this file, then the first saveSpan() would
// fail deep in the driver on "table spans has no column named attributes".
// The full-column check must catch it at open().
execRaw(
file,
'CREATE TABLE spans (id TEXT PRIMARY KEY, trace_id TEXT NOT NULL, ' +
'parent_id TEXT, name TEXT NOT NULL, started_at INTEGER NOT NULL, ' +
'completed_at INTEGER, status TEXT NOT NULL);',
);
const adapter = new SqliteDbAdapter({ filename: file });
await expect(adapter.connect()).rejects.toThrow(/incompatible shape.*attributes/s);
expect(adapter.isConnected()).toBe(false);
});
it('db("sqlite:<file>") is the durable adapter, end to end', async () => {
const file = tmpFile('factory');
const writing = db(`sqlite:${file}`);
expect(writing).toBeInstanceOf(SqliteDbAdapter);
await writing.saveMessage(message({ id: 'm1', sessionId: 'via-factory', content: 'kept' }));
await writing.disconnect();
const reading = db(`sqlite:${file}`);
const messages = await reading.getMessages('via-factory');
expect(messages.map((m) => m.content)).toEqual(['kept']);
await reading.disconnect();
});
it('sqlite ":memory:" is real SQLite but, like memory:, is process-local', async () => {
const a = createSqliteDbAdapter(':memory:');
await a.saveMessage(message({ id: 'm1', sessionId: 's', content: 'ephemeral' }));
expect((await a.getMessages('s')).map((m) => m.content)).toEqual(['ephemeral']);
const b = createSqliteDbAdapter(':memory:');
expect(await b.getMessages('s')).toEqual([]);
await a.disconnect();
await b.disconnect();
});
});
// ---------------------------------------------------------------------------
// Controls -- these run on every machine, driver or not
// ---------------------------------------------------------------------------
describe('db() controls', () => {
it('db("memory:") still works and is NOT persistent across instances', async () => {
const first = db('memory:');
expect(first).toBeInstanceOf(MemoryDbAdapter);
await first.createSession({ id: 's', createdAt: 1, updatedAt: 1 });
await first.saveMessage(message({ id: 'm1', sessionId: 's', content: 'gone soon' }));
expect((await first.getMessages('s')).map((m) => m.content)).toEqual(['gone soon']);
// A second adapter is a second set of Maps: nothing crosses between them,
// which is exactly why sqlite had to exist.
const second = db('memory:');
expect(await second.getSession('s')).toBeNull();
expect(await second.getMessages('s')).toEqual([]);
});
it('postgres:// and redis:// still throw, naming what to use instead', () => {
expect(() => db('postgres://user:pass@localhost:5432/app')).toThrow(
/not yet wired to the DbAdapter contract/,
);
expect(() => db('postgres://user:pass@localhost:5432/app')).toThrow(/db\("sqlite:\.\/data\.db"\)/);
expect(() => db('redis://localhost:6379')).toThrow(/not yet wired to the DbAdapter contract/);
});
it('an unusable driver produces a clear error and NEVER a silent memory fallback', async () => {
const file = path.join(tmpDir, 'never-created.db');
const adapter = new SqliteDbAdapter({
filename: file,
// Exactly what better-sqlite3 throws when its prebuilt binding was
// compiled against a different Node ABI.
driverLoader: async () => {
throw new Error(
"The module '/x/better_sqlite3.node' was compiled against a different Node.js " +
'version using NODE_MODULE_VERSION 141. This version of Node.js requires ' +
'NODE_MODULE_VERSION 127. (ERR_DLOPEN_FAILED)',
);
},
});
await expect(adapter.connect()).rejects.toThrow(/NODE_MODULE_VERSION/);
await expect(adapter.connect()).rejects.toThrow(/npm rebuild better-sqlite3/);
// A write must FAIL, not appear to succeed against memory...
await expect(
adapter.saveMessage(message({ id: 'm1', sessionId: 's', content: 'must not vanish' })),
).rejects.toThrow(/SQLite persistence is unavailable/);
// ...and a read must FAIL rather than return a confident empty list.
await expect(adapter.getMessages('s')).rejects.toThrow(/SQLite persistence is unavailable/);
expect(adapter.isConnected()).toBe(false);
expect(fs.existsSync(file)).toBe(false);
});
it('better-sqlite3, pinned, either works or explains how to fix its native binding', async () => {
const file = path.join(tmpDir, 'pinned-better.db');
const adapter = new SqliteDbAdapter({ filename: file, driver: 'better-sqlite3' });
let failure: Error | null = null;
try {
await adapter.connect();
} catch (error) {
failure = error as Error;
}
if (failure) {
// This is the state of this checkout today: the prebuilt binding does not
// match the running Node ABI. The contract is that the message says so
// and says what to do, instead of the adapter pretending to persist.
expect(failure.message).toMatch(/better-sqlite3/);
expect(failure.message).toMatch(/npm rebuild better-sqlite3|Install the driver/);
expect(failure.message).toMatch(/does not survive the process/);
expect(adapter.isConnected()).toBe(false);
} else {
expect(adapter.getDriverName()).toBe('better-sqlite3');
await adapter.saveMessage(message({ id: 'm1', sessionId: 's', content: 'ok' }));
expect((await adapter.getMessages('s')).map((m) => m.content)).toEqual(['ok']);
}
await adapter.disconnect();
});
});
@@ -1,48 +0,0 @@
/**
* Memory must not score noise.
*
* The default memory embedder was the sync placeholder in src/embeddings,
* which returned `Math.random()` vectors. Every similarity search over memory
* was therefore meaningless, with nothing to indicate it — the module even
* claimed "Python parity" in its docstring.
*/
import { embed, embeddings, getDimensions } from '../../../src/embeddings';
describe('sync embedding placeholders', () => {
it('refuses to embed rather than returning random vectors', () => {
expect(() => embed('hello')).toThrow(/Cannot embed synchronously/);
});
it('names the async replacement in the error', () => {
expect(() => embed('hello')).toThrow(/await embed\(/);
});
it('refuses batches too', () => {
expect(() => embeddings(['a', 'b'])).toThrow(/Cannot embed 2 text\(s\) synchronously/);
});
it('still reports dimensions, which needs no network', () => {
expect(getDimensions('text-embedding-3-small')).toBe(1536);
});
});
jest.mock('../../../src/llm/embeddings', () => ({
embed: jest.fn(async (_text: string, opts?: { model?: string }) => ({
embedding: [0.1, 0.2, 0.3],
usage: { tokens: 1 },
})),
}));
describe('ChromaMemory default embedder', () => {
it('routes through the real async embedder, not the sync placeholder', async () => {
const { embed: realEmbed } = await import('../../../src/llm/embeddings');
const { ChromaMemory } = await import('../../../src/memory/adapters');
const memory = new ChromaMemory({ embeddingModel: 'text-embedding-3-small' });
const vector = await (memory as any).embedText('hello', {});
expect(realEmbed).toHaveBeenCalledWith('hello', { model: 'text-embedding-3-small' });
expect(vector).toEqual([0.1, 0.2, 0.3]);
});
});
@@ -1799,18 +1799,6 @@ def _start_execution_worker(self, tools_list, console, session_state):
)
if _agent_approval is not None:
agent_extra_kwargs['approval'] = _agent_approval
# Thread the coding-sized step budget (`praisonai code
# --max-steps`, built into args.execution by the code
# command) onto the wrapper-dispatched REPL agent, so the
# common `pip install praisonai` path does not silently
# keep the general-purpose core defaults (20 steps / 10
# tool calls per turn) and truncate a coding task.
_agent_execution = (
getattr(self.args, 'execution', None)
if hasattr(self, 'args') else None
)
if _agent_execution is not None:
agent_extra_kwargs['execution'] = _agent_execution
def _build_agent():
# Build the agent from the CURRENT conversation history
@@ -2032,13 +2020,6 @@ def _process_interactive_prompt(self, prompt, tools_list, console, show_profilin
agent_approval = getattr(self.args, 'agent_approval', None) if hasattr(self, 'args') else None
if agent_approval is not None:
agent_kwargs['approval'] = agent_approval
# Thread the coding-sized step budget (`praisonai code --max-steps`,
# built into args.execution by the code command) onto the
# wrapper-dispatched single-prompt agent so it does not silently keep
# the general-purpose core defaults (20 steps / 10 tool calls per turn).
agent_execution = getattr(self.args, 'execution', None) if hasattr(self, 'args') else None
if agent_execution is not None:
agent_kwargs['execution'] = agent_execution
thinking_budget = getattr(self.args, 'thinking_budget', None) if hasattr(self, 'args') else None
# Show thinking indicator and create agent
@@ -305,17 +305,12 @@ def handle_rules_command(self, action: str, action_args: list):
except Exception as e:
print(f"[red]ERROR: Rules command failed: {e}[/red]")
def handle_hooks_command(self, action: str) -> int:
def handle_hooks_command(self, action: str):
"""
Handle hooks subcommand actions.
Args:
action: The hooks action (list, stats, init, help)
Returns:
Process exit code: ``0`` when the action ran, non-zero when it did
not. An unknown action previously printed an error and still exited
``0``, so a script adding a hook "succeeded" while doing nothing.
action: The hooks action (list, stats, init)
"""
try:
from praisonaiagents.memory import HooksManager
@@ -334,12 +329,8 @@ def handle_hooks_command(self, action: str) -> int:
for event in stats.get('events', []):
print(f" - {event}")
else:
print(
"[yellow]No hooks configured. Create "
f"{HooksManager.CONFIG_FILE}[/yellow]"
)
return 0
print("[yellow]No hooks configured. Create .praison/hooks.json[/yellow]")
elif action == 'stats':
stats = hooks.get_stats()
table = Table(title="Hooks Statistics")
@@ -350,20 +341,14 @@ def handle_hooks_command(self, action: str) -> int:
table.add_row(str(key), str(value))
console.print(table)
return 0
elif action == 'init':
# Derive the file location from the manager itself; a literal here
# is exactly how the CLI came to advertise a path the loader never
# reads (HooksManager.CONFIG_FILE is ".praisonai/hooks.json").
hooks_file = os.path.join(
os.getcwd(), HooksManager.CONFIG_FILE.replace("/", os.sep)
)
os.makedirs(os.path.dirname(hooks_file), exist_ok=True)
hooks_dir = os.path.join(os.getcwd(), ".praison")
os.makedirs(hooks_dir, exist_ok=True)
hooks_file = os.path.join(hooks_dir, "hooks.json")
if os.path.exists(hooks_file):
print(f"[yellow]hooks.json already exists at {hooks_file}[/yellow]")
return 0
else:
template = {
"enabled": True,
@@ -382,7 +367,6 @@ def handle_hooks_command(self, action: str) -> int:
json.dump(template, f, indent=2)
print(f"[green]✅ Created hooks.json at {hooks_file}[/green]")
print("[cyan]Edit the file to configure your hooks[/cyan]")
return 0
elif action == 'help' or action == '--help':
print("[bold]Hooks Commands:[/bold]")
@@ -395,19 +379,15 @@ def handle_hooks_command(self, action: str) -> int:
print(" pre_run_command, post_run_command")
print(" pre_user_prompt, post_user_prompt")
print(" pre_mcp_tool_use, post_mcp_tool_use")
return 0
else:
print(f"[red]Unknown hooks action: {action}[/red]")
print("Use 'praisonai hooks help' for available commands")
return 1
except ImportError as e:
print(f"[red]ERROR: Failed to import hooks module: {e}[/red]")
print("Make sure praisonaiagents is installed: pip install praisonaiagents")
return 1
except Exception as e:
print(f"[red]ERROR: Hooks command failed: {e}[/red]")
return 1
def handle_knowledge_command(self, action: str, action_args: list):
"""
@@ -0,0 +1,201 @@
"""
Manifest schema and storage for the FDEP standardisation system.
Manifests are stored in .praison/features/{slug}.yaml
"""
from pathlib import Path
from typing import List, Optional
from .models import FeatureManifest, FeatureSlug
class ManifestStorage:
"""Manages feature manifest files."""
def __init__(self, manifest_dir: Optional[Path] = None):
self.manifest_dir = manifest_dir or Path(".praison/features")
def get_manifest_path(self, slug: FeatureSlug) -> Path:
"""Get the path to a manifest file."""
return self.manifest_dir / f"{slug.normalised}.yaml"
def load(self, slug: FeatureSlug) -> Optional[FeatureManifest]:
"""Load a manifest from disk."""
path = self.get_manifest_path(slug)
if not path.exists():
return None
try:
import yaml
except ImportError:
return self._load_simple(path, slug)
try:
with open(path, "r", encoding="utf-8") as f:
data = yaml.safe_load(f)
return FeatureManifest.from_dict(data)
except Exception:
return None
def _load_simple(self, path: Path, slug: FeatureSlug) -> Optional[FeatureManifest]:
"""Simple YAML loading without PyYAML dependency."""
try:
content = path.read_text(encoding="utf-8")
data = {}
for line in content.split("\n"):
line = line.strip()
if ":" in line and not line.startswith("#"):
key, value = line.split(":", 1)
key = key.strip()
value = value.strip().strip('"').strip("'")
data[key] = value
return FeatureManifest.from_dict(data)
except Exception:
return None
def save(self, manifest: FeatureManifest) -> Path:
"""Save a manifest to disk."""
path = self.get_manifest_path(manifest.slug)
path.parent.mkdir(parents=True, exist_ok=True)
try:
import yaml
content = yaml.dump(manifest.to_dict(), default_flow_style=False, sort_keys=False)
except ImportError:
content = self._to_simple_yaml(manifest)
path.write_text(content, encoding="utf-8")
return path
def _to_simple_yaml(self, manifest: FeatureManifest) -> str:
"""Convert manifest to simple YAML without PyYAML."""
lines = []
data = manifest.to_dict()
for key, value in data.items():
if value is None:
continue
if isinstance(value, list):
if value:
lines.append(f"{key}:")
for item in value:
lines.append(f" - {item}")
elif isinstance(value, dict):
if value:
lines.append(f"{key}:")
for k, v in value.items():
lines.append(f" {k}: {v}")
else:
lines.append(f"{key}: {value}")
return "\n".join(lines)
def list_all(self) -> List[FeatureManifest]:
"""List all manifests."""
manifests = []
if not self.manifest_dir.exists():
return manifests
for yaml_file in self.manifest_dir.glob("*.yaml"):
slug = FeatureSlug.from_path(yaml_file, "manifest")
manifest = self.load(slug)
if manifest:
manifests.append(manifest)
return manifests
def delete(self, slug: FeatureSlug) -> bool:
"""Delete a manifest."""
path = self.get_manifest_path(slug)
if path.exists():
path.unlink()
return True
return False
def exists(self, slug: FeatureSlug) -> bool:
"""Check if a manifest exists."""
return self.get_manifest_path(slug).exists()
def create_default(self, slug: FeatureSlug,
sdk_module: Optional[str] = None,
cli_commands: Optional[List[str]] = None) -> FeatureManifest:
"""Create a default manifest for a feature."""
name = slug.normalised.replace("-", " ").title()
return FeatureManifest(
slug=slug,
name=name,
description=f"{name} feature for PraisonAI",
status="stable",
sdk_module=sdk_module or slug.normalised,
cli_commands=cli_commands or [],
)
# Schema for manifest validation
MANIFEST_SCHEMA = {
"type": "object",
"required": ["slug", "name", "description"],
"properties": {
"slug": {
"type": "string",
"pattern": "^[a-z][a-z0-9-]{0,62}[a-z0-9]$|^[a-z]$",
"description": "Feature slug (lowercase alphanumeric + hyphens)",
},
"name": {
"type": "string",
"minLength": 1,
"maxLength": 100,
"description": "Human-readable feature name",
},
"description": {
"type": "string",
"minLength": 1,
"maxLength": 500,
"description": "Brief description of the feature",
},
"status": {
"type": "string",
"enum": ["stable", "beta", "experimental", "deprecated"],
"default": "stable",
},
"min_version": {
"type": "string",
"pattern": "^\\d+\\.\\d+\\.\\d+$",
"description": "Minimum PraisonAI version required",
},
"tags": {
"type": "array",
"items": {"type": "string"},
"description": "Tags for categorisation",
},
"related_features": {
"type": "array",
"items": {"type": "string"},
"description": "Related feature slugs",
},
"sdk_module": {
"type": "string",
"description": "SDK module name (e.g., 'guardrails')",
},
"cli_commands": {
"type": "array",
"items": {"type": "string"},
"description": "CLI commands for this feature",
},
"api_classes": {
"type": "array",
"items": {"type": "string"},
"description": "Public API classes",
},
"artifacts": {
"type": "object",
"description": "Paths to artifacts",
"additionalProperties": {"type": "string"},
},
},
}
+1 -2
View File
@@ -169,8 +169,7 @@ agno = [
"praisonai-tools>=0.1.0",
]
google-adk = [
"praisonai-frameworks>=0.1.8",
"google-adk[extensions]>=2.5,<3",
"praisonai-frameworks[google-adk]>=0.1.8",
"praisonai-tools>=0.1.0",
]
pydantic-ai = [
@@ -1,173 +0,0 @@
"""The local layer measures things; these tests prove the measurements are used.
A competitor review found PraisonAI's local *sensing* to be the strongest of five
frameworks and its local *acting* the weakest: capabilities, context windows,
embedding widths and 18 documented quirks were all measured and then discarded.
Each test here pins one measurement to the behaviour it now drives.
"""
import pytest
from praisonaiagents.local.capabilities import ApiStyle, Evidence, LocalEngine
from praisonaiagents.local.discover import Discovery
from praisonaiagents.local.target import build_target
def _target(engine="ollama", base="http://127.0.0.1:11434", extra=()):
d = Discovery(engine=LocalEngine(engine), base_url=base, api_style=ApiStyle.OPENAI_CHAT,
engine_version=None, models=("m",), raw_identity="", latency_ms=1,
blocked=None, evidence=Evidence.SERVER)
return build_target(d, "m", extra=extra) if extra else build_target(d, "m")
class TestContextWindowComesFromTheServer:
"""Compaction budgeted every local model at the generic 128000 default."""
def test_a_probed_window_beats_the_generic_default(self):
from praisonaiagents.context.budgeter import get_model_limit
from praisonaiagents.local.embed import remember_model_facts
remember_model_facts("tiny-ctx-model", context_length=4096)
assert get_model_limit("tiny-ctx-model") == 4096, (
"a 4096-token model still budgeted at the 128000 default would have "
"31x its real room and the server would truncate silently"
)
def test_a_provider_prefixed_name_still_matches(self):
from praisonaiagents.context.budgeter import get_model_limit
from praisonaiagents.local.embed import remember_model_facts
remember_model_facts("prefixed-model", context_length=8192)
assert get_model_limit("ollama/prefixed-model") == 8192
def test_an_unprobed_model_is_untouched(self):
from praisonaiagents.context.budgeter import get_model_limit
assert get_model_limit("gpt-4o") == 128000
class TestEmbeddingWidthIsCarried:
"""A store built at the wrong width rejects every write or corrupts silently."""
def test_a_probed_width_reaches_the_embedder_config(self):
from praisonaiagents.local.embed import local_embedder_config, remember_model_facts
remember_model_facts("some-embedder", embedding_dimension=768)
cfg = local_embedder_config(LocalEngine.OLLAMA, "http://127.0.0.1:11434", "some-embedder")
assert cfg["config"]["embedding_dims"] == 768
def test_an_unknown_width_is_omitted_rather_than_guessed(self):
from praisonaiagents.local.embed import local_embedder_config
cfg = local_embedder_config(LocalEngine.OLLAMA, "http://127.0.0.1:11434", "never-probed")
assert "embedding_dims" not in cfg["config"], (
"guessing a width is worse than letting the store infer it"
)
class TestToolSchemaIsMadeServerSafe:
"""`Optional[str]` produces a union type, which 400s the WHOLE request."""
UNION_TOOL = [{
"type": "function",
"function": {"name": "f", "parameters": {
"type": "object",
"properties": {"city": {"type": ["string", "null"]},
"nested": {"type": "object",
"properties": {"n": {"type": ["integer", "null"]}}}},
}},
}]
@pytest.mark.parametrize("model", ["ollama/qwen3:0.6b", "lm_studio/x", "vllm/x"])
def test_local_engines_get_a_bare_type(self, model, monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "x")
from praisonaiagents.llm.llm import LLM
out = LLM(model=model)._format_tools_for_litellm(self.UNION_TOOL)
props = out[0]["function"]["parameters"]["properties"]
assert props["city"]["type"] == "string"
assert props["nested"]["properties"]["n"]["type"] == "integer", "nested unions too"
def test_hosted_providers_keep_the_full_schema(self, monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "x")
from praisonaiagents.llm.llm import LLM
out = LLM(model="gpt-4o")._format_tools_for_litellm(self.UNION_TOOL)
assert out[0]["function"]["parameters"]["properties"]["city"]["type"] == ["string", "null"]
def test_the_transform_does_not_mutate_the_caller_s_tools(self, monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "x")
from praisonaiagents.llm.adapters import collapse_union_param_types
original = [{"function": {"parameters": {"properties": {"c": {"type": ["string", "null"]}}}}}]
collapse_union_param_types(original)
assert original[0]["function"]["parameters"]["properties"]["c"]["type"] == ["string", "null"]
def test_a_heterogeneous_union_is_preserved(self, monkeypatch):
"""Only "<type> | null" collapses; a real multi-type union must survive."""
monkeypatch.setenv("OPENAI_API_KEY", "x")
from praisonaiagents.llm.adapters import collapse_union_param_types
tools = [{"function": {"parameters": {"properties": {
"id": {"type": ["string", "integer"]}}}}}]
out = collapse_union_param_types(tools)
assert out[0]["function"]["parameters"]["properties"]["id"]["type"] == ["string", "integer"], (
"narrowing a genuine union misrepresents the tool's accepted inputs"
)
def test_a_type_plus_null_still_collapses(self, monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "x")
from praisonaiagents.llm.adapters import collapse_union_param_types
tools = [{"function": {"parameters": {"properties": {
"n": {"type": ["integer", "null"]}}}}}]
out = collapse_union_param_types(tools)
assert out[0]["function"]["parameters"]["properties"]["n"]["type"] == "integer"
class TestFormatAndToolsAreRefused:
"""Measured: together, the tool call vanishes and the model invents an answer."""
TOOLS = [{"type": "function", "function": {"name": "f",
"parameters": {"type": "object", "properties": {}}}}]
FORMAT = {"type": "json_object"}
def _params(self, model, **kw):
from praisonaiagents.llm.llm import LLM
return LLM(model=model)._build_completion_params(
messages=[{"role": "user", "content": "x"}], **kw)
def test_the_combination_raises_on_a_local_engine(self, monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "x")
with pytest.raises(ValueError, match="fabricates"):
self._params("ollama/qwen3:0.6b", tools=self.TOOLS, response_format=self.FORMAT)
@pytest.mark.parametrize("kw", [{"tools": TOOLS}, {"response_format": FORMAT}])
def test_either_alone_is_fine(self, kw, monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "x")
assert self._params("ollama/qwen3:0.6b", **kw)
def test_hosted_providers_may_combine_them(self, monkeypatch):
"""OpenAI honours both together; the refusal must not leak to them."""
monkeypatch.setenv("OPENAI_API_KEY", "x")
assert self._params("gpt-4o", tools=self.TOOLS, response_format=self.FORMAT)
class TestLocallyServedEmbedderWidths:
"""Nine local embedders silently inherited OpenAI's 1536.
Harmless while nothing read the value; wrong the moment it sizes a store.
"""
@pytest.mark.parametrize("model,width", [
("nomic-embed-text", 768), ("mxbai-embed-large", 1024), ("all-minilm", 384),
("bge-m3", 1024), ("bge-large", 1024), ("bge-base", 768), ("bge-small", 384),
("snowflake-arctic-embed", 1024), ("granite-embedding", 384),
("paraphrase-multilingual", 768), ("qwen3-embedding", 1024),
])
def test_a_local_embedder_reports_its_real_width(self, model, width):
from praisonaiagents.embedding.dimensions import get_dimensions
assert get_dimensions(model) == width
def test_an_unknown_embedder_still_reports_the_default(self):
"""The default must stay, so callers can tell 'unknown' from 'measured'."""
from praisonaiagents.embedding.dimensions import DEFAULT_DIMENSION, get_dimensions
assert get_dimensions("a-model-nobody-has-heard-of") == DEFAULT_DIMENSION
def test_a_guessed_width_is_never_sent_to_the_store(self, monkeypatch):
"""Passing the generic default would size the index wrongly and silently."""
from praisonaiagents.embedding.dimensions import DEFAULT_DIMENSION
from praisonaiagents.local.embed import local_embedder_config
from praisonaiagents.local.capabilities import LocalEngine
cfg = local_embedder_config(LocalEngine.OLLAMA, "http://127.0.0.1:11434", "unknown-embedder")
assert cfg["config"].get("embedding_dims") != DEFAULT_DIMENSION
@@ -1,20 +0,0 @@
"""Regression tests for installable wrapper extras."""
from pathlib import Path
import toml
PYPROJECT = Path(__file__).parents[2] / "pyproject.toml"
def test_google_adk_extra_avoids_legacy_frameworks_adk_pin():
metadata = toml.loads(PYPROJECT.read_text(encoding="utf-8"))
requirements = metadata["project"]["optional-dependencies"]["google-adk"]
assert "praisonai-frameworks>=0.1.8" in requirements
assert "google-adk[extensions]>=2.5,<3" in requirements
assert not any(
requirement.startswith("praisonai-frameworks[google-adk]")
for requirement in requirements
)