mirror of
https://github.com/MervinPraison/PraisonAI.git
synced 2026-11-10 11:41:32 -10:00
Compare commits
98
Commits
No files matched your search
@@ -41,4 +41,4 @@ jobs:
|
||||
|
||||
- name: Run tests
|
||||
run: |
|
||||
cd src/praisonai && python -m pytest tests/unit/ -v --tb=short --disable-warnings --timeout=60 || echo "Some tests failed"
|
||||
cd src/praisonai && python -m pytest tests/unit/ -v --tb=short --disable-warnings --timeout=60
|
||||
@@ -155,7 +155,11 @@ jobs:
|
||||
tests/unit/plugins/ tests/unit/runtime/ tests/unit/tools/
|
||||
tests/unit/bus/ tests/unit/skills/ tests/unit/agents/
|
||||
tests/unit/rag/ tests/unit/streaming/
|
||||
tests/unit/test_adapter_registry.py
|
||||
tests/unit/test_adapter_registry.py tests/unit/test_token_budget.py
|
||||
tests/unit/test_streaming_events.py tests/unit/test_tool_decorator_approval.py
|
||||
tests/unit/test_task_execution_config.py
|
||||
tests/unit/test_agent_unknown_kwarg_guidance.py
|
||||
tests/unit/test_agent_instantiation_lazy.py tests/unit/test_agent_os_exports.py
|
||||
# Both tests previously deselected here now pass in combination.
|
||||
# test_lazy_litellm_import was looking the loader module up by
|
||||
# name and could get a different object than the one get_litellm()
|
||||
|
||||
@@ -14,27 +14,145 @@
|
||||
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
|
||||
</p>
|
||||
|
||||
<div align="center">
|
||||
# PraisonAI
|
||||
|
||||
# PraisonAI 🦞
|
||||
**Private AI that earns trust one task at a time.**
|
||||
|
||||
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
PraisonAI runs AI agents on your own computers, even fully offline, so private data never has to leave the building.
|
||||
|
||||
</div>
|
||||
- **For highly regulated teams (early access):** each task moves to your own AI only after it passes a test. If quality slips, it moves back automatically, and every step is recorded for regulators. [How it works](#how-the-handover-works-early-access)
|
||||
- **For everyone else:** a free, open-source Python agent framework. With a local model you need no API key and pay no cloud bill, and your files stay yours.
|
||||
|
||||
PraisonAI 🦞 — **Hire a 24/7 AI Workforce.** Stop writing boilerplate and start shipping autonomous, self-improving agents that research, plan, and execute tasks across your apps. From one agent to an entire organization, deployed in 5 lines of code.
|
||||
[2M+ downloads](https://pepy.tech/projects/praisonai) · [9.3K stars](https://github.com/MervinPraison/PraisonAI) · [600+ releases in the last year](https://github.com/MervinPraison/PraisonAI/releases) · [80+ contributors](https://github.com/MervinPraison/PraisonAI/graphs/contributors) · Python
|
||||
|
||||
```bash
|
||||
curl -fsSL https://praison.ai/install.sh | bash
|
||||
pip install praisonai
|
||||
```
|
||||
|
||||
Only need the Python library? `pip install praisonaiagents` installs just the lightweight core.
|
||||
|
||||
## Using PraisonAI for something else?
|
||||
|
||||
PraisonAI is still a general-purpose, open-source agent framework, and everything works as before: the same packages, imports, commands and examples. Go straight to what you came for:
|
||||
|
||||
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs)
|
||||
- **Workflows:** [AgentFlow: route, parallel, loop, repeat](https://praison.ai/docs/concepts/agentflow)
|
||||
- **Memory and knowledge:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge](https://praison.ai/docs/concepts/knowledge) · [RAG](https://praison.ai/docs/concepts/rag)
|
||||
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
|
||||
- **Bots (Telegram, Discord, Slack, WhatsApp):** [Bot CLI](https://praison.ai/docs/cli/bot) · [Bot gateway](https://praison.ai/docs/features/bot-gateway)
|
||||
- **Terminal CLI:** [`praisonai code`](https://praison.ai/docs/cli/code) · [`praisonai run`](https://praison.ai/docs/cli/run) · [`praisonai chat`](https://praison.ai/docs/cli/chat) · [CLI reference](https://praison.ai/docs/cli/cli-reference)
|
||||
- **Training and fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
|
||||
- **Browser automation:** [Browser agent](https://praison.ai/docs/features/browser-agent) · [Browser CLI](https://praison.ai/docs/cli/browser)
|
||||
- **Sandboxes:** [Sandboxed code execution](https://praison.ai/docs/features/sandbox)
|
||||
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
|
||||
- **Dashboard and UI:** [Claw dashboard](https://praison.ai/docs/concepts/claw) · [Chat UI](https://praison.ai/docs/ui)
|
||||
- **YAML, no code:** [YAML agents](#-using-yaml-no-code) · [YAML reference](https://praison.ai/docs/features/yaml-configuration-reference)
|
||||
- **JavaScript and TypeScript:** [JavaScript SDK](https://praison.ai/docs/js)
|
||||
- **Desktop app:** [PraisonAI Desktop](https://praison.ai/docs/features/desktop/index) · [Downloads](#desktop-app)
|
||||
- **Everything else:** the [full feature reference](#full-feature-reference) further down this page
|
||||
|
||||
## Quick start
|
||||
|
||||
Set a key for any supported provider (OpenAI shown here), or skip the key and use a local model as in the next section.
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY="your-api-key"
|
||||
```
|
||||
|
||||
```python
|
||||
from praisonaiagents import Agent
|
||||
|
||||
agent = Agent(instructions="You are a helpful assistant.")
|
||||
print(agent.start("In two sentences, why do some companies keep their data on their own computers?"))
|
||||
```
|
||||
|
||||
## Run fully offline
|
||||
|
||||
Run the same agent on a model on your own computer with [Ollama](https://ollama.com). You need no API key, and your prompts and answers stay on the machine.
|
||||
|
||||
Install the package and pull the model while you still have internet; after that, the agent runs with no network at all. If you installed only the core library, use `pip install "praisonaiagents[llm]"`, which adds the local-model support.
|
||||
|
||||
```bash
|
||||
# Start Ollama first (the app, or "ollama serve" in another terminal)
|
||||
ollama pull llama3.2
|
||||
export DO_NOT_TRACK=true # keeps telemetry off (PraisonAI sends none unless you turn it on)
|
||||
```
|
||||
|
||||
```python
|
||||
from praisonaiagents import Agent
|
||||
|
||||
agent = Agent(instructions="You are a helpful assistant.", model="ollama/llama3.2")
|
||||
print(agent.start("Why is the sky blue?"))
|
||||
```
|
||||
|
||||
If Ollama runs on another computer on your network, pass `base_url` (or set `OLLAMA_API_BASE`):
|
||||
|
||||
```python
|
||||
agent = Agent(instructions="...", model="ollama/llama3.2", base_url="http://192.168.1.10:11434")
|
||||
```
|
||||
|
||||
vLLM and LM Studio work the same way: use `hosted_vllm/` (vLLM) or `lm_studio/` with your server's `base_url`. More in the [Ollama guide](https://praison.ai/docs/models/ollama) and [Models docs](https://praison.ai/docs/models).
|
||||
|
||||
## How the handover works (early access)
|
||||
|
||||
1. **Watch:** records how each task is done today.
|
||||
2. **Test:** your own AI takes an exam made from real past work it never studied.
|
||||
3. **Hand over:** a task moves to your own AI only when it passes.
|
||||
4. **Roll back:** if quality drops, the task moves back automatically.
|
||||
5. **Record:** every handover leaves a record your regulator can check.
|
||||
|
||||
**What is "your own AI"?** An open model, such as Llama, Qwen or Gemma, running on your own computers. Use it as it is, or train it on your own past work with [PraisonAI Train](https://praison.ai/docs/train) on your own GPU servers. Your team decides when to train; nothing trains automatically. Either way, a task moves to it only after it passes that exam.
|
||||
|
||||
Highly regulated teams, such as banks, insurers and hospitals, want AI but can't send customer data to outside models. From 6 January 2027, OpenAI also takes no new self-serve fine-tuning jobs ([notice](https://developers.openai.com/api/docs/deprecations)), so teams that customise models need models they own. The handover lets these teams move work to AI they own, one task at a time, with a record that shows why each step was trusted.
|
||||
|
||||
It is built on parts of this framework that are live today: tracing, evaluation, fine-tuning, model routing and local models. The handover itself is in early access, and we are taking on our first design partners. To take part, see [praison.ai/enterprise](https://praison.ai/enterprise/).
|
||||
|
||||
## Features
|
||||
|
||||
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs) · [Workflows](https://praison.ai/docs/concepts/agentflow)
|
||||
- **Memory:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge and RAG](https://praison.ai/docs/concepts/rag) · [Sessions](https://praison.ai/docs/concepts/session-management)
|
||||
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
|
||||
- **Guardrails and approval:** [Guardrails](https://praison.ai/docs/concepts/guardrails) · [Human approval](https://praison.ai/docs/concepts/approval) · [Policy engine](https://praison.ai/docs/features/policy-engine)
|
||||
- **Tracing and evaluation:** [Tracing](https://praison.ai/docs/observability/overview) · [Telemetry](https://praison.ai/docs/features/telemetry) · [Langfuse](https://praison.ai/docs/observability/langfuse) · [Evaluation](https://praison.ai/docs/concepts/evaluation)
|
||||
- **Your choice of model:** [Models](https://praison.ai/docs/models) · [Model router](https://praison.ai/docs/features/model-router)
|
||||
- **Fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
|
||||
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
|
||||
|
||||
## JavaScript and TypeScript
|
||||
|
||||
```bash
|
||||
npm install praisonai
|
||||
```
|
||||
|
||||
The TypeScript and JavaScript SDK has its own guide at [praison.ai/docs/js](https://praison.ai/docs/js). A short example is in [Using JavaScript Code](#-using-javascript-code) below.
|
||||
|
||||
## The company
|
||||
|
||||
- **Who:** Praison AI Limited in Cambridge, UK, building PraisonAI since September 2023. Co-founders Mervin Praison (CEO), Leebana Smith (COO) and Karthick Subramanian (CBO). [About us](https://praison.ai/about/)
|
||||
- **What we're building:** the task handover above, as a self-hosted product on top of this framework. It is in early access, and we are taking on our first design partners.
|
||||
- **Why now:** highly regulated teams want AI but can't send customer data to outside models, and from 6 January 2027 OpenAI takes no new self-serve fine-tuning jobs.
|
||||
- **How it fits together:** the framework stays free and MIT-licensed, and brings developers in. Regulated teams license the handover to run inside their own walls.
|
||||
- **Traction:** every number at the top of this page is public and linked to its source.
|
||||
- **Get in touch:** design partners, investors and press: enterprise@praison.ai or [praison.ai/enterprise](https://praison.ai/enterprise/)
|
||||
|
||||
## Docs, community and licence
|
||||
|
||||
- **Documentation:** [praison.ai/docs](https://praison.ai/docs)
|
||||
- **Questions and bugs:** [Discussions](https://github.com/MervinPraison/PraisonAI/discussions) · [Issues](https://github.com/MervinPraison/PraisonAI/issues)
|
||||
- **Contributing:** [Contributing guide](https://github.com/MervinPraison/PraisonAI/blob/main/CONTRIBUTING.md)
|
||||
- **Videos and news:** [YouTube](https://youtube.com/@MervinPraison) · [X](https://x.com/MervinPraison) · [LinkedIn](https://linkedin.com/in/mervinpraison)
|
||||
- **Desktop app:** macOS, Windows and Linux builds are on the [latest release](https://github.com/MervinPraison/PraisonAI/releases/latest)
|
||||
- **Licence:** [MIT](https://github.com/MervinPraison/PraisonAI/blob/main/LICENSE)
|
||||
|
||||
---
|
||||
|
||||
## Full feature reference
|
||||
|
||||
Everything below is the full reference that was on this page before. Nothing has been removed; the longest parts are folded, so click a heading's summary to open it.
|
||||
|
||||
### Desktop app
|
||||
|
||||
<div align="center">
|
||||
<br>
|
||||
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
|
||||
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
|
||||
</a>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
|
||||
<img src="https://img.shields.io/badge/Download_for_macOS-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Download for macOS" />
|
||||
</a>
|
||||
@@ -46,12 +164,17 @@ curl -fsSL https://praison.ai/install.sh | bash
|
||||
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
|
||||
<img src="https://img.shields.io/badge/Download_for_Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download for Linux" />
|
||||
</a>
|
||||
<br>
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
|
||||
</p>
|
||||
### Other ways to install
|
||||
|
||||
One-line installer:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://praison.ai/install.sh | bash
|
||||
```
|
||||
|
||||
The full `praisonai` package, with the CLI, bots, dashboard and integrations:
|
||||
|
||||
```
|
||||
██████╗ ██████╗ █████╗ ██╗███████╗ ██████╗ ███╗ ██╗ █████╗ ██╗
|
||||
@@ -64,10 +187,20 @@ curl -fsSL https://praison.ai/install.sh | bash
|
||||
pip install praisonai
|
||||
```
|
||||
|
||||
<p align="center">
|
||||
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
|
||||
</p>
|
||||
|
||||
<div align="center">
|
||||
<a href="https://docs.praison.ai">
|
||||
|
||||
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<a href="https://praison.ai/docs">
|
||||
<p align="center">
|
||||
<img src="https://img.shields.io/badge/📚_Documentation-Visit_docs.praison.ai-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
|
||||
<img src="https://img.shields.io/badge/📚_Documentation-Visit_praison.ai%2Fdocs-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
|
||||
</p>
|
||||
</a>
|
||||
</div>
|
||||
@@ -114,6 +247,9 @@ Most frameworks hand you one or two layers and leave the rest as homework. Prais
|
||||
|
||||
Each layer wraps the one inside it. When an agent misbehaves, the layer tells you where to look.
|
||||
|
||||
<details>
|
||||
<summary><strong>Show the five layers, examples and Managed Agents</strong></summary>
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ⬡ MANAGED AGENTS — Where does it actually run? │
|
||||
@@ -174,7 +310,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://docs.praison.ai/docs/concepts/handoffs)
|
||||
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://praison.ai/docs/concepts/handoffs)
|
||||
|
||||
### Layer 3 · Harness — *Can it act, and be checked?*
|
||||
|
||||
@@ -217,7 +353,7 @@ print(result.completion_reason)
|
||||
# surfaced here as completion_reason="error")
|
||||
```
|
||||
|
||||
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://docs.praison.ai/docs/features/doom-loop-detection)
|
||||
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://praison.ai/docs/features/doom-loop-detection)
|
||||
|
||||
### Layer 5 · Graph — *Who runs when, and who checks whom?*
|
||||
|
||||
@@ -237,7 +373,7 @@ flow = AgentFlow(steps=[
|
||||
flow.run("Ticket #123: login fails on Safari")
|
||||
```
|
||||
|
||||
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://docs.praison.ai/docs/concepts/agentflow)
|
||||
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://praison.ai/docs/concepts/agentflow)
|
||||
|
||||
### ⬡ Outside the stack: Managed Agents — *Where does it actually run?*
|
||||
|
||||
@@ -332,6 +468,8 @@ Sandboxes shut themselves down when idle (`auto_shutdown`, `idle_timeout_s`), an
|
||||
|
||||
<sub>Stack framing adapted from [The Five-Layer Agent Stack](https://mer.vin/2026/07/five-layer-agent-stack-match-bug-to-right-layer/) and [Agent Harnesses vs Orbs](https://mer.vin/2026/08/agent-harnesses-vs-orbs-why-remote-sandboxes-beat-local-agent-loops/).</sub>
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 🌌 The PraisonAI Ecosystem
|
||||
@@ -411,6 +549,11 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
</details>
|
||||
|
||||
<div align="center">
|
||||
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
|
||||
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
|
||||
</a>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
|
||||
<img src=".github/images/elon_musk_praisonai.png" alt="Highlighted by Elon Musk" width="600" />
|
||||
</a>
|
||||
@@ -426,13 +569,13 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
|--|---------|-----|
|
||||
| 🔌 | **MCP Protocol** — stdio, HTTP, WebSocket, SSE | `tools=MCP("npx ...")` |
|
||||
| 🧠 | **Planning Mode** — plan → execute → reason | `planning=True` |
|
||||
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://docs.praison.ai/docs/agents/deep-research) |
|
||||
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://docs.praison.ai/docs/code/external-agents) |
|
||||
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://praison.ai/docs/agents/deep-research) |
|
||||
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://praison.ai/docs/code/external-agents) |
|
||||
| 🔄 | **Agent Handoffs** — seamless conversation passing | `handoffs=[other_agent]` |
|
||||
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://docs.praison.ai/docs/concepts/guardrails) |
|
||||
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://praison.ai/docs/concepts/guardrails) |
|
||||
| | **Web Search + Fetch** — native browsing | `web=True` |
|
||||
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://docs.praison.ai/docs/concepts/reflection) |
|
||||
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://praison.ai/docs/concepts/reflection) |
|
||||
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://praison.ai/docs/concepts/agentflow) |
|
||||
| 🧠 | **Memory (zero deps)** — works out of the box | `memory=True` |
|
||||
|
||||
<details>
|
||||
@@ -443,18 +586,18 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
| 💡 | **Prompt Caching** — reduce latency + cost | `caching=True` |
|
||||
| 💾 | **Sessions + Auto-Save** — persistent state across restarts | `auto_save="my-project"` |
|
||||
| 💭 | **Thinking Budgets** — control reasoning depth | `agent.thinking_budget = 1024` |
|
||||
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://docs.praison.ai/docs/concepts/rag) |
|
||||
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://docs.praison.ai/docs/features/model-router) |
|
||||
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://docs.praison.ai/docs/features/checkpoints) |
|
||||
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://docs.praison.ai/docs/features/a2a) |
|
||||
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://docs.praison.ai/docs/features/context-compaction) |
|
||||
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://docs.praison.ai/docs/features/telemetry) |
|
||||
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://docs.praison.ai/docs/features/policy-engine) |
|
||||
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://docs.praison.ai/docs/features/background-tasks) |
|
||||
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://docs.praison.ai/docs/features/doom-loop-detection) |
|
||||
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://docs.praison.ai/docs/features/graph-memory) |
|
||||
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://docs.praison.ai/docs/features/sandbox) |
|
||||
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://docs.praison.ai/docs/features/bot-gateway) |
|
||||
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://praison.ai/docs/concepts/rag) |
|
||||
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://praison.ai/docs/features/model-router) |
|
||||
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://praison.ai/docs/features/checkpoints) |
|
||||
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://praison.ai/docs/features/a2a) |
|
||||
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://praison.ai/docs/features/context-compaction) |
|
||||
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://praison.ai/docs/features/telemetry) |
|
||||
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://praison.ai/docs/features/policy-engine) |
|
||||
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://praison.ai/docs/features/background-tasks) |
|
||||
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://praison.ai/docs/features/doom-loop-detection) |
|
||||
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://praison.ai/docs/features/graph-memory) |
|
||||
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://praison.ai/docs/features/sandbox) |
|
||||
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://praison.ai/docs/features/bot-gateway) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -465,6 +608,9 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
|
||||
## 📘 Using Python Code
|
||||
|
||||
<details>
|
||||
<summary><strong>Show Python examples: single agent, multi-agent, MCP, tools, databases, Claw, Langflow, UI</strong></summary>
|
||||
|
||||
> Unsure whether a result is a bug or model randomness? Check the [SDK Behavior Matrix](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/docs/sdk-behavior-matrix.md) for `stream`, memory prefetch, structured output, teams and AG-UI.
|
||||
|
||||
### 1. Single Agent
|
||||
@@ -510,7 +656,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
> 📖 [Full MCP docs](https://docs.praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
|
||||
> 📖 [Full MCP docs](https://praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
|
||||
|
||||
### 4. Custom Tools
|
||||
|
||||
@@ -562,7 +708,7 @@ agent.start("Search for AI news and calculate 15*4")
|
||||
```
|
||||
|
||||
> ⚠️ **Security Note:** Never use `eval()`, `exec()`, or `subprocess` in tool functions that process LLM-generated or user-supplied input. Always validate and sanitize inputs to prevent code injection attacks.
|
||||
> 📖 [Full tools docs](https://docs.praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
|
||||
> 📖 [Full tools docs](https://praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
|
||||
|
||||
### 5. Persistence (Databases)
|
||||
|
||||
@@ -579,7 +725,7 @@ agent = Agent(
|
||||
agent.chat("Hello!") # Auto-persists messages, runs, traces
|
||||
```
|
||||
|
||||
> 📖 [Full persistence docs](https://docs.praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
|
||||
> 📖 [Full persistence docs](https://praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
|
||||
|
||||
### 6. PraisonAI Claw 🦞 (Dashboard UI)
|
||||
|
||||
@@ -601,7 +747,7 @@ Copy `.env.example` to `.env` and configure the following variables:
|
||||
|
||||
Open **http://localhost:8082** — the dashboard comes with 13 built-in pages: Chat, Agents, Memory, Knowledge, Channels, Guardrails, Cron, and more. Add messaging channels directly from the UI.
|
||||
|
||||
> 📖 [Full Claw docs](https://docs.praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
|
||||
> 📖 [Full Claw docs](https://praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
|
||||
|
||||
### 7. Langflow Integration 🔗 (Visual Flow Builder)
|
||||
|
||||
@@ -614,7 +760,7 @@ praisonai flow
|
||||
|
||||
Open **http://localhost:7861** — use the **Agent** and **Agent Team** components to create sequential or parallel workflows. Connect Chat Input → Agent Team → Chat Output for instant multi-agent pipelines.
|
||||
|
||||
> 📖 [Full Flow docs](https://docs.praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
|
||||
> 📖 [Full Flow docs](https://praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
|
||||
|
||||
### 8. PraisonAI UI 🤖 (Clean Chat)
|
||||
|
||||
@@ -625,10 +771,15 @@ pip install "praisonai[ui]"
|
||||
praisonai ui
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 📄 Using YAML (No Code)
|
||||
|
||||
<details>
|
||||
<summary><strong>Show YAML examples</strong></summary>
|
||||
|
||||
### Example 1: Two Agents Working Together
|
||||
|
||||
Create `agents.yaml`:
|
||||
@@ -700,6 +851,8 @@ praisonai agents.yaml
|
||||
> - Tools in `tools.py` are automatically discovered
|
||||
> - The function's docstring helps the AI understand how to use it
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 🎯 CLI Quick Reference
|
||||
@@ -718,23 +871,26 @@ praisonai agents.yaml
|
||||
| **Development** | `commit`, `docs`, `checkpoint`, `hooks` |
|
||||
| **Scheduling** | `schedule start`, `schedule list`, `schedule stop` |
|
||||
|
||||
> 📖 [Full CLI reference](https://docs.praison.ai/docs/cli/cli-reference)
|
||||
> 📖 [Full CLI reference](https://praison.ai/docs/cli/cli-reference)
|
||||
|
||||
---
|
||||
|
||||
## ✨ Key Features
|
||||
|
||||
<details>
|
||||
<summary><strong>Show all features by category</strong></summary>
|
||||
|
||||
<details open>
|
||||
<summary><strong>🤖 Core Agents</strong></summary>
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://docs.praison.ai/docs/agents/single) |
|
||||
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://docs.praison.ai/docs/concepts/agents) |
|
||||
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://docs.praison.ai/docs/features/autoagents) |
|
||||
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://docs.praison.ai/docs/concepts/reflection) |
|
||||
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://docs.praison.ai/docs/features/reasoning) |
|
||||
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://docs.praison.ai/docs/features/multimodal) |
|
||||
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://praison.ai/docs/agents/single) |
|
||||
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://praison.ai/docs/concepts/agents) |
|
||||
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://praison.ai/docs/features/autoagents) |
|
||||
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://praison.ai/docs/concepts/reflection) |
|
||||
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://praison.ai/docs/features/reasoning) |
|
||||
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://praison.ai/docs/features/multimodal) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -743,16 +899,16 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://docs.praison.ai/docs/features/routing) |
|
||||
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
|
||||
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
|
||||
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
|
||||
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://praison.ai/docs/features/routing) |
|
||||
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://praison.ai/docs/features/parallelisation) |
|
||||
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://praison.ai/docs/features/repetitive) |
|
||||
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
|
||||
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -761,13 +917,13 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://docs.praison.ai/docs/features/codeagent) |
|
||||
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://docs.praison.ai/docs/code/editing) |
|
||||
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://docs.praison.ai/docs/code/external-agents) |
|
||||
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://docs.praison.ai/docs/code/claude-code) |
|
||||
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://docs.praison.ai/docs/code/gemini-cli) |
|
||||
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://docs.praison.ai/docs/code/codex-cli) |
|
||||
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://docs.praison.ai/docs/code/cursor-cli) |
|
||||
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://praison.ai/docs/features/codeagent) |
|
||||
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://praison.ai/docs/code/editing) |
|
||||
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://praison.ai/docs/code/external-agents) |
|
||||
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://praison.ai/docs/code/claude-code) |
|
||||
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://praison.ai/docs/code/gemini-cli) |
|
||||
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://praison.ai/docs/code/codex-cli) |
|
||||
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://praison.ai/docs/code/cursor-cli) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -776,18 +932,18 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
|
||||
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
|
||||
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://docs.praison.ai/docs/features/claude-memory-tool) |
|
||||
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://docs.praison.ai/docs/concepts/knowledge) |
|
||||
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://docs.praison.ai/docs/concepts/rag) |
|
||||
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://docs.praison.ai/docs/features/chat-with-pdf) |
|
||||
| Data Readers (PDF, DOCX, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/chunking-strategies) |
|
||||
| Vector Store Selection | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/knowledge-backends) |
|
||||
| Retrieval Strategies | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval-strategies) |
|
||||
| Rerankers | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/smart-retrieval) |
|
||||
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/incremental-indexing) |
|
||||
| Query Engines (Sub-Question, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval) |
|
||||
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
|
||||
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
|
||||
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://praison.ai/docs/features/claude-memory-tool) |
|
||||
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://praison.ai/docs/concepts/knowledge) |
|
||||
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://praison.ai/docs/concepts/rag) |
|
||||
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://praison.ai/docs/features/chat-with-pdf) |
|
||||
| Data Readers (PDF, DOCX, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/chunking-strategies) |
|
||||
| Vector Store Selection | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/knowledge-backends) |
|
||||
| Retrieval Strategies | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval-strategies) |
|
||||
| Rerankers | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/smart-retrieval) |
|
||||
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/incremental-indexing) |
|
||||
| Query Engines (Sub-Question, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -796,12 +952,12 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://docs.praison.ai/docs/agents/deep-research) |
|
||||
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://docs.praison.ai/docs/agents/query-rewriter) |
|
||||
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/agents/websearch) |
|
||||
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/tools/tavily) |
|
||||
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://docs.praison.ai/docs/tools/web-search) |
|
||||
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
|
||||
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://praison.ai/docs/agents/deep-research) |
|
||||
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://praison.ai/docs/agents/query-rewriter) |
|
||||
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/agents/websearch) |
|
||||
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/tools/tavily) |
|
||||
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://praison.ai/docs/tools/web-search) |
|
||||
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -810,12 +966,12 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
|
||||
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
|
||||
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
|
||||
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://docs.praison.ai/docs/features/promptchaining) |
|
||||
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
|
||||
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://docs.praison.ai/docs/concepts/orchestration) |
|
||||
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
|
||||
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
|
||||
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
|
||||
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://praison.ai/docs/features/promptchaining) |
|
||||
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
|
||||
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://praison.ai/docs/concepts/orchestration) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -824,15 +980,15 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://docs.praison.ai/docs/agents/data-analyst) |
|
||||
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://docs.praison.ai/docs/agents/finance) |
|
||||
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://docs.praison.ai/docs/agents/shopping) |
|
||||
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://docs.praison.ai/docs/agents/recommendation) |
|
||||
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://docs.praison.ai/docs/agents/wikipedia) |
|
||||
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://docs.praison.ai/docs/agents/programming) |
|
||||
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://docs.praison.ai/docs/features/mathagent) |
|
||||
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://docs.praison.ai/docs/agents/markdown) |
|
||||
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://docs.praison.ai/docs/agents/prompt-expander) |
|
||||
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://praison.ai/docs/agents/data-analyst) |
|
||||
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://praison.ai/docs/agents/finance) |
|
||||
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://praison.ai/docs/agents/shopping) |
|
||||
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://praison.ai/docs/agents/recommendation) |
|
||||
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://praison.ai/docs/agents/wikipedia) |
|
||||
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://praison.ai/docs/agents/programming) |
|
||||
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://praison.ai/docs/features/mathagent) |
|
||||
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://praison.ai/docs/agents/markdown) |
|
||||
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://praison.ai/docs/agents/prompt-expander) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -841,10 +997,10 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://docs.praison.ai/docs/features/image-generation) |
|
||||
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://docs.praison.ai/docs/agents/image-to-text) |
|
||||
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://docs.praison.ai/docs/agents/video) |
|
||||
| Camera Integration | [Example](examples/python/camera/) | [📖](https://docs.praison.ai/docs/features/camera-integration) |
|
||||
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://praison.ai/docs/features/image-generation) |
|
||||
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://praison.ai/docs/agents/image-to-text) |
|
||||
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://praison.ai/docs/agents/video) |
|
||||
| Camera Integration | [Example](examples/python/camera/) | [📖](https://praison.ai/docs/features/camera-integration) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -853,12 +1009,12 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
|
||||
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
|
||||
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Config Management | [Docs](https://docs.praison.ai/docs/cli/mcp) | [📖](https://docs.praison.ai/docs/cli/mcp) |
|
||||
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://docs.praison.ai/docs/features/langchain) |
|
||||
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://praison.ai/docs/mcp/transports) |
|
||||
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://praison.ai/docs/mcp/transports) |
|
||||
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Config Management | [Docs](https://praison.ai/docs/cli/mcp) | [📖](https://praison.ai/docs/cli/mcp) |
|
||||
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://praison.ai/docs/features/langchain) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -867,9 +1023,9 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://docs.praison.ai/docs/concepts/guardrails) |
|
||||
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://docs.praison.ai/docs/concepts/approval) |
|
||||
| Rules & Instructions | [Docs](https://docs.praison.ai/docs/features/rules) | [📖](https://docs.praison.ai/docs/features/rules) |
|
||||
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://praison.ai/docs/concepts/guardrails) |
|
||||
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://praison.ai/docs/concepts/approval) |
|
||||
| Rules & Instructions | [Docs](https://praison.ai/docs/features/rules) | [📖](https://praison.ai/docs/features/rules) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -878,16 +1034,16 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://docs.praison.ai/docs/features/async) |
|
||||
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
|
||||
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
|
||||
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://docs.praison.ai/docs/concepts/handoffs) |
|
||||
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://docs.praison.ai/docs/features/stateful-agents) |
|
||||
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://docs.praison.ai/docs/concepts/autonomy) |
|
||||
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://docs.praison.ai/docs/features/structured) |
|
||||
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://docs.praison.ai/docs/features/model-router) |
|
||||
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
|
||||
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://docs.praison.ai/docs/features/fast-context) |
|
||||
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://praison.ai/docs/features/async) |
|
||||
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://praison.ai/docs/features/parallelisation) |
|
||||
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://praison.ai/docs/features/repetitive) |
|
||||
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://praison.ai/docs/concepts/handoffs) |
|
||||
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://praison.ai/docs/features/stateful-agents) |
|
||||
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://praison.ai/docs/concepts/autonomy) |
|
||||
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://praison.ai/docs/features/structured) |
|
||||
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://praison.ai/docs/features/model-router) |
|
||||
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
|
||||
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://praison.ai/docs/features/fast-context) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -896,21 +1052,21 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://docs.praison.ai/docs/tools/tools) |
|
||||
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://docs.praison.ai/docs/developers/agents-playbook) |
|
||||
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://docs.praison.ai/docs/models) |
|
||||
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
|
||||
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
|
||||
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://docs.praison.ai/docs/features/middleware) |
|
||||
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://docs.praison.ai/docs/features/configurable-model) |
|
||||
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://docs.praison.ai/docs/features/rate-limiter) |
|
||||
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://docs.praison.ai/docs/features/injected-state) |
|
||||
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://docs.praison.ai/docs/features/checkpoints) |
|
||||
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://docs.praison.ai/docs/features/background-tasks) |
|
||||
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://docs.praison.ai/docs/features/policy-engine) |
|
||||
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://docs.praison.ai/docs/features/thinking-budgets) |
|
||||
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://docs.praison.ai/docs/features/output-styles) |
|
||||
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://docs.praison.ai/docs/features/context-compaction) |
|
||||
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://praison.ai/docs/tools/tools) |
|
||||
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://praison.ai/docs/developers/agents-playbook) |
|
||||
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://praison.ai/docs/models) |
|
||||
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://praison.ai/docs/concepts/hooks) |
|
||||
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://praison.ai/docs/concepts/hooks) |
|
||||
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://praison.ai/docs/features/middleware) |
|
||||
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://praison.ai/docs/features/configurable-model) |
|
||||
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://praison.ai/docs/features/rate-limiter) |
|
||||
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://praison.ai/docs/features/injected-state) |
|
||||
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://praison.ai/docs/features/checkpoints) |
|
||||
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://praison.ai/docs/features/background-tasks) |
|
||||
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://praison.ai/docs/features/policy-engine) |
|
||||
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://praison.ai/docs/features/thinking-budgets) |
|
||||
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://praison.ai/docs/features/output-styles) |
|
||||
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://praison.ai/docs/features/context-compaction) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -919,14 +1075,14 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://docs.praison.ai/docs/concepts/session-management) |
|
||||
| Auto-Save Sessions | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
|
||||
| History in Context | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
|
||||
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://docs.praison.ai/docs/features/telemetry) |
|
||||
| Langfuse Tracing | [Docs](https://docs.praison.ai/docs/observability/langfuse) | [📖](https://docs.praison.ai/docs/observability/langfuse) |
|
||||
| Project Docs (.praison/docs/) | [Docs](https://docs.praison.ai/docs/cli/docs) | [📖](https://docs.praison.ai/docs/cli/docs) |
|
||||
| AI Commit Messages | [Docs](https://docs.praison.ai/docs/cli/commit) | [📖](https://docs.praison.ai/docs/cli/commit) |
|
||||
| @Mentions in Prompts | [Docs](https://docs.praison.ai/docs/cli/mentions) | [📖](https://docs.praison.ai/docs/cli/mentions) |
|
||||
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://praison.ai/docs/concepts/session-management) |
|
||||
| Auto-Save Sessions | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
|
||||
| History in Context | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
|
||||
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://praison.ai/docs/features/telemetry) |
|
||||
| Langfuse Tracing | [Docs](https://praison.ai/docs/observability/langfuse) | [📖](https://praison.ai/docs/observability/langfuse) |
|
||||
| Project Docs (.praison/docs/) | [Docs](https://praison.ai/docs/cli/docs) | [📖](https://praison.ai/docs/cli/docs) |
|
||||
| AI Commit Messages | [Docs](https://praison.ai/docs/cli/commit) | [📖](https://praison.ai/docs/cli/commit) |
|
||||
| @Mentions in Prompts | [Docs](https://praison.ai/docs/cli/mentions) | [📖](https://praison.ai/docs/cli/mentions) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -935,20 +1091,20 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://docs.praison.ai/docs/cli/slash-commands) |
|
||||
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://docs.praison.ai/docs/cli/autonomy-modes) |
|
||||
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://docs.praison.ai/docs/cli/cost-tracking) |
|
||||
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://docs.praison.ai/docs/cli/repo-map) |
|
||||
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://docs.praison.ai/docs/cli/interactive-tui) |
|
||||
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://docs.praison.ai/docs/cli/git-integration) |
|
||||
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://docs.praison.ai/docs/cli/sandbox-execution) |
|
||||
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://docs.praison.ai/docs/cli/compare) |
|
||||
| Profile/Benchmark | [Docs](https://docs.praison.ai/docs/cli/profile) | [📖](https://docs.praison.ai/docs/cli/profile) |
|
||||
| Auto Mode | [Docs](https://docs.praison.ai/docs/cli/auto) | [📖](https://docs.praison.ai/docs/cli/auto) |
|
||||
| Init | [Docs](https://docs.praison.ai/docs/cli/init) | [📖](https://docs.praison.ai/docs/cli/init) |
|
||||
| File Input | [Docs](https://docs.praison.ai/docs/cli/file-input) | [📖](https://docs.praison.ai/docs/cli/file-input) |
|
||||
| Final Agent | [Docs](https://docs.praison.ai/docs/cli/final-agent) | [📖](https://docs.praison.ai/docs/cli/final-agent) |
|
||||
| Max Tokens | [Docs](https://docs.praison.ai/docs/cli/max-tokens) | [📖](https://docs.praison.ai/docs/cli/max-tokens) |
|
||||
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://praison.ai/docs/cli/slash-commands) |
|
||||
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://praison.ai/docs/cli/autonomy-modes) |
|
||||
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://praison.ai/docs/cli/cost-tracking) |
|
||||
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://praison.ai/docs/cli/repo-map) |
|
||||
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://praison.ai/docs/cli/interactive-tui) |
|
||||
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://praison.ai/docs/cli/git-integration) |
|
||||
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://praison.ai/docs/cli/sandbox-execution) |
|
||||
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://praison.ai/docs/cli/compare) |
|
||||
| Profile/Benchmark | [Docs](https://praison.ai/docs/cli/profile) | [📖](https://praison.ai/docs/cli/profile) |
|
||||
| Auto Mode | [Docs](https://praison.ai/docs/cli/auto) | [📖](https://praison.ai/docs/cli/auto) |
|
||||
| Init | [Docs](https://praison.ai/docs/cli/init) | [📖](https://praison.ai/docs/cli/init) |
|
||||
| File Input | [Docs](https://praison.ai/docs/cli/file-input) | [📖](https://praison.ai/docs/cli/file-input) |
|
||||
| Final Agent | [Docs](https://praison.ai/docs/cli/final-agent) | [📖](https://praison.ai/docs/cli/final-agent) |
|
||||
| Max Tokens | [Docs](https://praison.ai/docs/cli/max-tokens) | [📖](https://praison.ai/docs/cli/max-tokens) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -957,10 +1113,10 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -969,8 +1125,8 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
|
||||
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
|
||||
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://praison.ai/docs/concepts/skills) |
|
||||
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://praison.ai/docs/concepts/skills) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -979,7 +1135,9 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://docs.praison.ai/docs/cli/scheduler) |
|
||||
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://praison.ai/docs/cli/scheduler) |
|
||||
|
||||
</details>
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1008,7 +1166,7 @@ PraisonAI keeps framework overhead microsecond-class: creating an `Agent` (one t
|
||||
|--------------------|-----------|----------|
|
||||
| Avg Agent instantiation (1 tool, 1000 iterations) | **46 μs** | 2026-10-15 · praisonaiagents 1.7.12 |
|
||||
|
||||
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://docs.praison.ai/docs/developers/benchmarks)
|
||||
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://praison.ai/docs/developers/benchmarks)
|
||||
|
||||
---
|
||||
|
||||
@@ -1018,7 +1176,7 @@ Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md]
|
||||
|
||||
## ⭐ Star History
|
||||
|
||||
[](https://docs.praison.ai)
|
||||
[](https://praison.ai/docs)
|
||||
|
||||
---
|
||||
|
||||
@@ -1089,6 +1247,9 @@ We welcome contributions! Fork the repo, create a branch, and submit a PR → [C
|
||||
|
||||
## ❓ FAQ & Troubleshooting
|
||||
|
||||
<details>
|
||||
<summary><strong>Show FAQ and troubleshooting</strong></summary>
|
||||
|
||||
<details>
|
||||
<summary><strong>ModuleNotFoundError: No module named 'praisonaiagents'</strong></summary>
|
||||
|
||||
@@ -1107,7 +1268,7 @@ Ensure your API key is set:
|
||||
export OPENAI_API_KEY=your_key_here
|
||||
```
|
||||
|
||||
For other providers, see [Models docs](https://docs.praison.ai/docs/models).
|
||||
For other providers, see [Models docs](https://praison.ai/docs/models).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1148,7 +1309,7 @@ agent = Agent(instructions="...", llm="ollama/llama3.2", base_url="http://192.16
|
||||
The same shape works for other local runtimes — `lm_studio/`, `vllm/` and
|
||||
`hosted_vllm/` with their server's `base_url`.
|
||||
|
||||
See [Models docs](https://docs.praison.ai/docs/models) for more details.
|
||||
See [Models docs](https://praison.ai/docs/models) for more details.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1168,7 +1329,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
See [Persistence docs](https://docs.praison.ai/docs/databases/overview) for supported databases.
|
||||
See [Persistence docs](https://praison.ai/docs/databases/overview) for supported databases.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1185,7 +1346,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
See [Memory docs](https://docs.praison.ai/docs/concepts/memory) for more options.
|
||||
See [Memory docs](https://praison.ai/docs/concepts/memory) for more options.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1201,7 +1362,7 @@ agents = Agents(agents=[agent1, agent2])
|
||||
agents.start()
|
||||
```
|
||||
|
||||
See [Agents docs](https://docs.praison.ai/docs/concepts/agents) for more examples.
|
||||
See [Agents docs](https://praison.ai/docs/concepts/agents) for more examples.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1216,13 +1377,15 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport options.
|
||||
See [MCP docs](https://praison.ai/docs/mcp/transports) for all transport options.
|
||||
|
||||
</details>
|
||||
|
||||
</details>
|
||||
|
||||
### Getting Help
|
||||
|
||||
- 📚 [Full Documentation](https://docs.praison.ai)
|
||||
- 📚 [Full Documentation](https://praison.ai/docs)
|
||||
- 🐛 [Report Issues](https://github.com/MervinPraison/PraisonAI/issues)
|
||||
- 💬 [Discussions](https://github.com/MervinPraison/PraisonAI/discussions)
|
||||
|
||||
@@ -1231,7 +1394,7 @@ See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport op
|
||||
<div align="center">
|
||||
<p><strong>Made with ❤️ by the PraisonAI Team</strong></p>
|
||||
<p>
|
||||
<a href="https://docs.praison.ai">📚 Documentation</a> •
|
||||
<a href="https://praison.ai/docs">📚 Documentation</a> •
|
||||
<a href="https://github.com/MervinPraison/PraisonAI">GitHub</a> •
|
||||
<a href="https://youtube.com/@MervinPraison">▶️ YouTube</a> •
|
||||
<a href="https://x.com/MervinPraison">𝕏 X</a> •
|
||||
|
||||
@@ -5,23 +5,39 @@ PraisonAI Agents - Real Execution Benchmark
|
||||
Benchmarks ACTUAL agent execution with LLM API calls.
|
||||
Tests single agent running a simple task across frameworks.
|
||||
|
||||
Each framework gets untimed warm-up calls first (imports, client set-up and
|
||||
the first connection are not what this measures), then the timed runs are
|
||||
interleaved round by round so network drift hits every framework alike.
|
||||
A run that raises or returns an empty answer is counted as a failure, not
|
||||
timed. The summary reports the median.
|
||||
|
||||
Usage:
|
||||
export OPENAI_API_KEY=your_key
|
||||
python benchmarks/execution_benchmark.py
|
||||
python benchmarks/execution_benchmark.py --no-save
|
||||
python benchmarks/execution_benchmark.py --iterations 5
|
||||
python benchmarks/execution_benchmark.py --save
|
||||
python benchmarks/execution_benchmark.py --iterations 10
|
||||
python benchmarks/execution_benchmark.py --model gpt-4o --iterations 10
|
||||
"""
|
||||
|
||||
import time
|
||||
import os
|
||||
import argparse
|
||||
import os
|
||||
import statistics
|
||||
import time
|
||||
from importlib.metadata import version as get_version
|
||||
|
||||
# Defaults (can be overridden via CLI)
|
||||
DEFAULT_MODEL = "gpt-4o-mini"
|
||||
DEFAULT_ITERATIONS = 3
|
||||
DEFAULT_ITERATIONS = 10
|
||||
DEFAULT_WARMUP = 1
|
||||
DEFAULT_PROMPT = "What is 2+2? Reply with just the number."
|
||||
INSTRUCTIONS = "You are a helpful assistant. Be very brief."
|
||||
|
||||
METHODS = {
|
||||
'PraisonAI': 'agent.start()',
|
||||
'PraisonAI (LiteLLM)': 'agent.start()',
|
||||
'Agno': 'agent.run()',
|
||||
'CrewAI': 'crew.kickoff()',
|
||||
}
|
||||
|
||||
|
||||
def get_package_versions():
|
||||
@@ -35,260 +51,235 @@ def get_package_versions():
|
||||
for name, pkg in packages.items():
|
||||
try:
|
||||
versions[name] = get_version(pkg)
|
||||
except:
|
||||
except Exception:
|
||||
versions[name] = 'not installed'
|
||||
return versions
|
||||
|
||||
|
||||
def benchmark_praisonai(model, iterations, prompt):
|
||||
"""Benchmark PraisonAI agent execution using agent.start()"""
|
||||
def praisonai_runner(model, prompt):
|
||||
"""PraisonAI: a new agent per run; only agent.start() is timed."""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
print("\n--- PraisonAI ---")
|
||||
print("Method: agent.start()")
|
||||
|
||||
times = []
|
||||
for i in range(iterations):
|
||||
agent = Agent(
|
||||
name="Calculator",
|
||||
instructions="You are a helpful assistant. Be very brief.",
|
||||
llm=model
|
||||
)
|
||||
|
||||
start = time.perf_counter()
|
||||
response = agent.start(prompt, output="silent")
|
||||
elapsed = time.perf_counter() - start
|
||||
|
||||
times.append(elapsed)
|
||||
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
|
||||
|
||||
avg = sum(times) / len(times)
|
||||
print(f" Average: {avg:.2f}s")
|
||||
return avg
|
||||
|
||||
def build():
|
||||
agent = Agent(name="Calculator", instructions=INSTRUCTIONS, llm=model)
|
||||
return lambda: agent.start(prompt, output="silent")
|
||||
return build
|
||||
|
||||
|
||||
def benchmark_praisonai_litellm(model, iterations, prompt):
|
||||
"""Benchmark PraisonAI with LiteLLM backend using agent.start()"""
|
||||
from praisonaiagents import Agent
|
||||
|
||||
print("\n--- PraisonAI (LiteLLM) ---")
|
||||
print("Method: agent.start()")
|
||||
|
||||
times = []
|
||||
for i in range(iterations):
|
||||
agent = Agent(
|
||||
name="Calculator",
|
||||
instructions="You are a helpful assistant. Be very brief.",
|
||||
llm=f"openai/{model}"
|
||||
)
|
||||
|
||||
start = time.perf_counter()
|
||||
response = agent.start(prompt, output="silent")
|
||||
elapsed = time.perf_counter() - start
|
||||
|
||||
times.append(elapsed)
|
||||
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
|
||||
|
||||
avg = sum(times) / len(times)
|
||||
print(f" Average: {avg:.2f}s")
|
||||
return avg
|
||||
|
||||
|
||||
def benchmark_agno(model, iterations, prompt):
|
||||
"""Benchmark Agno agent execution using agent.run()"""
|
||||
def agno_runner(model, prompt):
|
||||
"""Agno: a new agent per run; only agent.run() is timed."""
|
||||
from agno.agent import Agent
|
||||
from agno.models.openai import OpenAIChat
|
||||
|
||||
print("\n--- Agno ---")
|
||||
print("Method: agent.run()")
|
||||
|
||||
times = []
|
||||
|
||||
# One model object per runner: each OpenAIChat opens its own HTTP client,
|
||||
# so a fresh one per round would put a new connection inside the timer,
|
||||
# while PraisonAI and CrewAI agents share theirs.
|
||||
llm = OpenAIChat(id=model)
|
||||
|
||||
def build():
|
||||
agent = Agent(model=llm, instructions=[INSTRUCTIONS])
|
||||
|
||||
def call():
|
||||
response = agent.run(prompt)
|
||||
# Agno reports a failed run in ``status`` and puts the error text in
|
||||
# ``content``, so the content alone would pass for an answer.
|
||||
if str(getattr(response, 'status', '')).lower().endswith('error'):
|
||||
raise RuntimeError(f"Agno run failed: {str(response.content)[:200]}")
|
||||
return getattr(response, 'content', response)
|
||||
return call
|
||||
return build
|
||||
|
||||
|
||||
def crewai_runner(model, prompt):
|
||||
"""CrewAI: a new one-agent crew per run; only crew.kickoff() is timed."""
|
||||
from crewai import Agent, Crew, Task
|
||||
|
||||
def build():
|
||||
agent = Agent(role="Calculator", goal="Answer math questions",
|
||||
backstory=INSTRUCTIONS, llm=model, verbose=False)
|
||||
task = Task(description=prompt, expected_output="A number", agent=agent)
|
||||
return Crew(agents=[agent], tasks=[task], verbose=False).kickoff
|
||||
return build
|
||||
|
||||
|
||||
def build_runners(model, prompt):
|
||||
"""Return {framework: build()} for every installed framework.
|
||||
|
||||
``build()`` creates fresh objects (untimed) and returns the call to time.
|
||||
"""
|
||||
factories = [
|
||||
('PraisonAI', lambda: praisonai_runner(model, prompt)),
|
||||
('PraisonAI (LiteLLM)', lambda: praisonai_runner(f"openai/{model}", prompt)),
|
||||
('Agno', lambda: agno_runner(model, prompt)),
|
||||
('CrewAI', lambda: crewai_runner(model, prompt)),
|
||||
]
|
||||
runners = {}
|
||||
for name, factory in factories:
|
||||
try:
|
||||
runners[name] = factory()
|
||||
except ImportError as e:
|
||||
print(f" {name}: skipped ({e})")
|
||||
return runners
|
||||
|
||||
|
||||
def answer_text(response):
|
||||
"""The answer as text, or '' when the run produced nothing usable."""
|
||||
if response is None:
|
||||
return ''
|
||||
text = str(response).strip()
|
||||
return '' if text == 'None' else text
|
||||
|
||||
|
||||
def run_benchmark(runners, iterations, warmup):
|
||||
"""Warm up each framework, then time interleaved rounds.
|
||||
|
||||
A failed warm-up is reported but the framework stays in the timed rounds,
|
||||
so one transient error does not discard it. The order within a round
|
||||
rotates every round. Object construction happens
|
||||
before the timer starts; only the execution call is timed.
|
||||
|
||||
Returns {framework: {"times": [...], "failures": [...]}}.
|
||||
"""
|
||||
results = {name: {"times": [], "failures": []} for name in runners}
|
||||
|
||||
for name, build in runners.items():
|
||||
for _ in range(warmup):
|
||||
try:
|
||||
if not answer_text(build()()):
|
||||
raise RuntimeError("empty answer")
|
||||
except Exception as e:
|
||||
print(f" {name}: warm-up failed ({type(e).__name__}: {str(e)[:120]})")
|
||||
|
||||
# Rotate the start each round so no framework always holds the same slot.
|
||||
names = list(runners)
|
||||
for i in range(iterations):
|
||||
agent = Agent(
|
||||
model=OpenAIChat(id=model),
|
||||
instructions=["You are a helpful assistant. Be very brief."],
|
||||
)
|
||||
|
||||
start = time.perf_counter()
|
||||
response = agent.run(prompt)
|
||||
elapsed = time.perf_counter() - start
|
||||
|
||||
times.append(elapsed)
|
||||
# Extract content from response
|
||||
content = str(response.content)[:50] if hasattr(response, 'content') else str(response)[:50]
|
||||
print(f" Run {i+1}: {elapsed:.2f}s - Response: {content}")
|
||||
|
||||
avg = sum(times) / len(times)
|
||||
print(f" Average: {avg:.2f}s")
|
||||
return avg
|
||||
shift = i % len(names) if names else 0
|
||||
for name in names[shift:] + names[:shift]:
|
||||
build = runners[name]
|
||||
try:
|
||||
call = build()
|
||||
start = time.perf_counter()
|
||||
text = answer_text(call())
|
||||
elapsed = time.perf_counter() - start
|
||||
error = None if text else "empty answer"
|
||||
except Exception as e:
|
||||
text, error = '', f"{type(e).__name__}: {e}"
|
||||
if error:
|
||||
results[name]["failures"].append(error)
|
||||
print(f" Round {i + 1} {name}: FAILED ({error[:120]})")
|
||||
else:
|
||||
results[name]["times"].append(elapsed)
|
||||
print(f" Round {i + 1} {name}: {elapsed:.2f}s - {text[:40]}")
|
||||
return results
|
||||
|
||||
|
||||
def benchmark_crewai(model, iterations, prompt):
|
||||
"""Benchmark CrewAI agent execution using crew.kickoff()"""
|
||||
from crewai import Agent, Task, Crew
|
||||
|
||||
print("\n--- CrewAI ---")
|
||||
print("Method: crew.kickoff()")
|
||||
|
||||
times = []
|
||||
for i in range(iterations):
|
||||
agent = Agent(
|
||||
role="Calculator",
|
||||
goal="Answer math questions",
|
||||
backstory="You are a helpful assistant.",
|
||||
llm=model,
|
||||
verbose=False
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description=prompt,
|
||||
expected_output="A number",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=False
|
||||
)
|
||||
|
||||
start = time.perf_counter()
|
||||
response = crew.kickoff()
|
||||
elapsed = time.perf_counter() - start
|
||||
|
||||
times.append(elapsed)
|
||||
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
|
||||
|
||||
avg = sum(times) / len(times)
|
||||
print(f" Average: {avg:.2f}s")
|
||||
return avg
|
||||
def summarise(results):
|
||||
"""Rows of (framework, median, mean, ok, failed), fastest median first."""
|
||||
rows = []
|
||||
for name, r in results.items():
|
||||
times = r["times"]
|
||||
if times:
|
||||
rows.append((name, statistics.median(times), statistics.fmean(times),
|
||||
len(times), len(r["failures"])))
|
||||
return sorted(rows, key=lambda row: row[1])
|
||||
|
||||
|
||||
def save_results(results: dict, model: str, iterations: int, prompt: str):
|
||||
def save_results(rows, results, model, iterations, warmup, prompt):
|
||||
"""Save results to markdown file."""
|
||||
import os
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
filepath = os.path.join(os.path.dirname(__file__), 'EXECUTION_BENCHMARK_RESULTS.md')
|
||||
versions = get_package_versions()
|
||||
|
||||
# Sort by time
|
||||
sorted_results = sorted(results.items(), key=lambda x: x[1])
|
||||
fastest = sorted_results[0][1]
|
||||
|
||||
fastest = rows[0][1]
|
||||
|
||||
with open(filepath, 'w') as f:
|
||||
f.write('# PraisonAI Agents - Real Execution Benchmark\n\n')
|
||||
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
|
||||
f.write(f'**Model:** {model}\n')
|
||||
f.write(f'**Iterations:** {iterations}\n')
|
||||
f.write(f'**Iterations:** {iterations} interleaved rounds after {warmup} warm-up call(s) per framework\n')
|
||||
f.write(f'**Prompt:** "{prompt}"\n\n')
|
||||
f.write('## Results\n\n')
|
||||
f.write('| Framework | Method | Avg Time | Relative |\n')
|
||||
f.write('|-----------|--------|----------|----------|\n')
|
||||
|
||||
methods = {
|
||||
'PraisonAI': 'agent.start()',
|
||||
'PraisonAI (LiteLLM)': 'agent.start()',
|
||||
'Agno': 'agent.run()',
|
||||
'CrewAI': 'crew.kickoff()',
|
||||
}
|
||||
|
||||
for name, avg in sorted_results:
|
||||
ratio = avg / fastest
|
||||
method = methods.get(name, 'unknown')
|
||||
f.write('| Framework | Method | Median | Mean | Runs ok / failed | Relative |\n')
|
||||
f.write('|-----------|--------|--------|------|------------------|----------|\n')
|
||||
for name, median, mean, ok, failed in rows:
|
||||
ratio = median / fastest
|
||||
method = METHODS.get(name, 'unknown')
|
||||
if ratio == 1.0:
|
||||
f.write(f'| **{name}** | `{method}` | **{avg:.2f}s** | **1.00x (fastest)** |\n')
|
||||
f.write(f'| **{name}** | `{method}` | **{median:.2f}s** | {mean:.2f}s | {ok} / {failed} | **1.00x (fastest)** |\n')
|
||||
else:
|
||||
f.write(f'| {name} | `{method}` | {avg:.2f}s | {ratio:.2f}x |\n')
|
||||
|
||||
f.write(f'| {name} | `{method}` | {median:.2f}s | {mean:.2f}s | {ok} / {failed} | {ratio:.2f}x |\n')
|
||||
excluded = [name for name, r in results.items() if not r["times"]]
|
||||
if excluded:
|
||||
f.write(f'\nNo successful runs (not ranked): {", ".join(excluded)}\n')
|
||||
|
||||
f.write('\n## Package Versions\n\n')
|
||||
f.write('| Package | Version |\n')
|
||||
f.write('|---------|--------|\n')
|
||||
for pkg, ver in versions.items():
|
||||
f.write(f'| {pkg} | {ver} |\n')
|
||||
|
||||
|
||||
f.write('\n## How to Reproduce\n\n')
|
||||
f.write('```bash\n')
|
||||
f.write('export OPENAI_API_KEY=your_key\n')
|
||||
f.write('cd praisonai-agents\n')
|
||||
f.write('python benchmarks/execution_benchmark.py\n')
|
||||
f.write('cd src/praisonai-agents\n')
|
||||
f.write('python benchmarks/execution_benchmark.py --save\n')
|
||||
f.write('```\n')
|
||||
|
||||
|
||||
print(f'\nResults saved to: {filepath}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description='PraisonAI Agents - Real Execution Benchmark')
|
||||
parser.add_argument('--save', action='store_true', help='Save results to file')
|
||||
parser.add_argument('--save', action='store_true', help='Save results to EXECUTION_BENCHMARK_RESULTS.md')
|
||||
parser.add_argument('--model', '-m', type=str, default=DEFAULT_MODEL, help=f'Model to use (default: {DEFAULT_MODEL})')
|
||||
parser.add_argument('--iterations', '-i', type=int, default=DEFAULT_ITERATIONS, help=f'Number of iterations (default: {DEFAULT_ITERATIONS})')
|
||||
parser.add_argument('--iterations', '-i', type=int, default=DEFAULT_ITERATIONS, help=f'Timed rounds (default: {DEFAULT_ITERATIONS})')
|
||||
parser.add_argument('--warmup', '-w', type=int, default=DEFAULT_WARMUP, help=f'Untimed warm-up calls per framework (default: {DEFAULT_WARMUP})')
|
||||
parser.add_argument('--prompt', '-p', type=str, default=DEFAULT_PROMPT, help=f'Prompt to use (default: "{DEFAULT_PROMPT}")')
|
||||
args = parser.parse_args()
|
||||
|
||||
# Check API key
|
||||
if args.iterations < 1:
|
||||
parser.error('--iterations must be at least 1')
|
||||
if args.warmup < 0:
|
||||
parser.error('--warmup must be 0 or more')
|
||||
|
||||
if not os.environ.get('OPENAI_API_KEY'):
|
||||
print("ERROR: Set OPENAI_API_KEY environment variable")
|
||||
exit(1)
|
||||
|
||||
model = args.model
|
||||
iterations = args.iterations
|
||||
prompt = args.prompt
|
||||
|
||||
return 1
|
||||
|
||||
print('=' * 60)
|
||||
print('PraisonAI Agents - Real Execution Benchmark')
|
||||
print('=' * 60)
|
||||
print(f'\nModel: {model}')
|
||||
print(f'Iterations: {iterations}')
|
||||
print(f'Prompt: "{prompt}"')
|
||||
|
||||
versions = get_package_versions()
|
||||
print(f'\nModel: {args.model}')
|
||||
print(f'Rounds: {args.iterations} (after {args.warmup} warm-up call(s) per framework)')
|
||||
print(f'Prompt: "{args.prompt}"')
|
||||
print('\nPackage versions:')
|
||||
for pkg, ver in versions.items():
|
||||
for pkg, ver in get_package_versions().items():
|
||||
print(f' {pkg}: {ver}')
|
||||
|
||||
results = {}
|
||||
|
||||
# Run benchmarks
|
||||
try:
|
||||
results['PraisonAI'] = benchmark_praisonai(model, iterations, prompt)
|
||||
except Exception as e:
|
||||
print(f"PraisonAI error: {e}")
|
||||
|
||||
try:
|
||||
results['PraisonAI (LiteLLM)'] = benchmark_praisonai_litellm(model, iterations, prompt)
|
||||
except Exception as e:
|
||||
print(f"PraisonAI (LiteLLM) error: {e}")
|
||||
|
||||
try:
|
||||
results['Agno'] = benchmark_agno(model, iterations, prompt)
|
||||
except Exception as e:
|
||||
print(f"Agno error: {e}")
|
||||
|
||||
try:
|
||||
results['CrewAI'] = benchmark_crewai(model, iterations, prompt)
|
||||
except Exception as e:
|
||||
print(f"CrewAI error: {e}")
|
||||
|
||||
# Summary
|
||||
print()
|
||||
|
||||
runners = build_runners(args.model, args.prompt)
|
||||
results = run_benchmark(runners, args.iterations, args.warmup)
|
||||
rows = summarise(results)
|
||||
|
||||
print('\n' + '=' * 60)
|
||||
print('SUMMARY')
|
||||
print('SUMMARY (median of successful runs)')
|
||||
print('=' * 60)
|
||||
|
||||
if results:
|
||||
sorted_results = sorted(results.items(), key=lambda x: x[1])
|
||||
fastest = sorted_results[0][1]
|
||||
|
||||
print(f"\n{'Framework':<25} {'Avg Time':<12} {'Relative':<10}")
|
||||
print('-' * 47)
|
||||
|
||||
for name, avg in sorted_results:
|
||||
ratio = avg / fastest
|
||||
print(f'{name:<25} {avg:<12.2f}s {ratio:.2f}x')
|
||||
|
||||
if rows:
|
||||
fastest = rows[0][1]
|
||||
print(f"\n{'Framework':<22} {'Median':>8} {'Mean':>8} {'ok/failed':>10} {'Relative':>9}")
|
||||
print('-' * 61)
|
||||
for name, median, mean, ok, failed in rows:
|
||||
print(f'{name:<22} {median:>7.2f}s {mean:>7.2f}s {ok:>5}/{failed:<4} {median / fastest:>8.2f}x')
|
||||
for name, r in results.items():
|
||||
if not r["times"]:
|
||||
print(f'{name:<22} no successful runs: {(r["failures"] or ["?"])[0][:80]}')
|
||||
print('\n' + '=' * 60)
|
||||
|
||||
# Save results (if --save)
|
||||
if results and args.save:
|
||||
save_results(results, model, iterations, prompt)
|
||||
|
||||
if rows and args.save:
|
||||
save_results(rows, results, args.model, args.iterations, args.warmup, args.prompt)
|
||||
elif not args.save:
|
||||
print('\nResults not saved (use --save flag to save results to file)')
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
raise SystemExit(main())
|
||||
@@ -109,17 +109,11 @@ def run_benchmark(skip_crewai: bool = False, quiet: bool = False,
|
||||
pass
|
||||
|
||||
try:
|
||||
from langchain_core.tools import tool as langchain_tool
|
||||
from langchain_openai import ChatOpenAI
|
||||
from langgraph.prebuilt import create_react_agent
|
||||
|
||||
@langchain_tool
|
||||
def get_weather_lg(city: Literal['nyc', 'sf']):
|
||||
"""Get weather info."""
|
||||
return 'sunny' if city == 'sf' else 'cloudy'
|
||||
|
||||
|
||||
results['LangGraph'] = measure(
|
||||
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
|
||||
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[])
|
||||
)
|
||||
except ImportError:
|
||||
pass
|
||||
@@ -129,19 +123,12 @@ def run_benchmark(skip_crewai: bool = False, quiet: bool = False,
|
||||
else:
|
||||
try:
|
||||
from crewai.agent import Agent as CrewAgent
|
||||
from crewai.tools import tool as crewai_tool
|
||||
|
||||
@crewai_tool("Weather Tool")
|
||||
def get_weather_crew(city: Literal['nyc', 'sf']):
|
||||
"""Get weather info."""
|
||||
return 'sunny' if city == 'sf' else 'cloudy'
|
||||
|
||||
results['CrewAI'] = measure(
|
||||
lambda: CrewAgent(
|
||||
role='Weather Agent',
|
||||
goal='Provide weather info',
|
||||
backstory='A weather expert',
|
||||
tools=[get_weather_crew],
|
||||
verbose=False
|
||||
)
|
||||
)
|
||||
|
||||
@@ -2698,6 +2698,8 @@ class Agent(GoalLoopMixin, SteeringMixin, SandboxMixin, SkillReviewMixin, Unifie
|
||||
self.system_prompt = f"""{self.backstory}\n
|
||||
Your Role: {self.role}\n
|
||||
Your Goal: {self.goal}
|
||||
""" if self.goal != self.backstory else f"""{self.backstory}\n
|
||||
Your Role: {self.role}
|
||||
"""
|
||||
|
||||
# Per-invocation system-prompt append (never persisted). Sourced from the
|
||||
@@ -3571,8 +3573,26 @@ Your Goal: {self.goal}
|
||||
|
||||
def _replace_chat_history(self, new_history: List[Dict[str, Any]]):
|
||||
"""Thread-safe replacement of entire chat history using proper locking."""
|
||||
with self._history_lock.lock():
|
||||
self._history_lock.value[:] = new_history
|
||||
# Share _auto_save_session's lock so a save cannot interleave with the
|
||||
# rewrite and the cursor update below (#6217).
|
||||
with self._auto_save_lock, self._history_lock.lock():
|
||||
history = self._history_lock.value
|
||||
# Re-sync the positional auto-save cursor (#6217): resume at the
|
||||
# first unsaved survivor, or right after the last survivor so
|
||||
# appended messages (e.g. durable replays) are saved; summaries
|
||||
# placed before survivors are not re-persisted.
|
||||
last = getattr(self, '_auto_save_last_index', 0)
|
||||
old = {id(m): i >= last for i, m in enumerate(history)}
|
||||
cursor = len(new_history)
|
||||
survivors = [i for i, m in enumerate(new_history) if id(m) in old]
|
||||
if survivors:
|
||||
cursor = survivors[-1] + 1
|
||||
for i in survivors:
|
||||
if old[id(new_history[i])]:
|
||||
cursor = i
|
||||
break
|
||||
history[:] = new_history
|
||||
self._auto_save_last_index = cursor
|
||||
|
||||
@property
|
||||
def _cache_lock(self):
|
||||
|
||||
@@ -543,17 +543,53 @@ class ChatMixin:
|
||||
return int(getattr(execution, "max_tool_calls_per_turn", default))
|
||||
return default
|
||||
|
||||
def _resolve_harness_base_prompt(self):
|
||||
_EDIT_TOOL_NAMES = frozenset({"edit_file", "apply_patch"})
|
||||
|
||||
def _offers_edit_tool(self, tools=None):
|
||||
"""True when ``tools`` (default: the agent's tools) include a file-edit primitive."""
|
||||
for tool in (tools if tools is not None else getattr(self, "tools", None)) or ():
|
||||
if isinstance(tool, list):
|
||||
# _format_tools_for_completion flattens a nested list but keeps
|
||||
# only its function-schema dicts, so only those can offer an edit tool.
|
||||
if self._offers_edit_tool([
|
||||
t for t in tool
|
||||
if isinstance(t, dict) and t.get("type") == "function"
|
||||
and isinstance(t.get("function"), dict)
|
||||
]):
|
||||
return True
|
||||
continue
|
||||
if isinstance(tool, str):
|
||||
name = tool
|
||||
elif isinstance(tool, dict):
|
||||
function = tool.get("function")
|
||||
name = function.get("name") if isinstance(function, dict) else tool.get("name")
|
||||
else:
|
||||
# A tool's ``name`` is what the model is sent (``@tool(name=...)``
|
||||
# keeps the function's own ``__name__``), so it wins.
|
||||
name = getattr(tool, "name", None)
|
||||
if not isinstance(name, str):
|
||||
name = getattr(tool, "__name__", None)
|
||||
# _prune_denied_tools drops denied tools from the payload, so a
|
||||
# denied edit tool must not earn the hint either.
|
||||
if name in self._EDIT_TOOL_NAMES and self._tool_name_allowed(name):
|
||||
return True
|
||||
return False
|
||||
|
||||
def _resolve_harness_base_prompt(self, tools=None):
|
||||
"""Resolve the model-family harness base-prompt fragment, if any.
|
||||
|
||||
Honours an explicit ``harness_base_prompt`` override on the agent when
|
||||
set; otherwise resolves lazily from the active model id. Returns an
|
||||
empty string when no fragment applies (behaviour-preserving default).
|
||||
Never raises: any resolution error collapses to no fragment.
|
||||
set; otherwise resolves lazily from the active model id. The family
|
||||
fragments are file-editing guidance, so they apply only when ``tools``
|
||||
(default: the agent's tools) include ``edit_file`` or ``apply_patch``.
|
||||
Returns an empty string when no fragment applies. Never raises: any
|
||||
resolution error collapses to no fragment.
|
||||
"""
|
||||
override = getattr(self, "harness_base_prompt", None)
|
||||
if override is not None:
|
||||
return override
|
||||
if not self._offers_edit_tool(tools):
|
||||
return ""
|
||||
try:
|
||||
from ..model_harness import resolve_harness
|
||||
# self.llm may be a string model id or an LLM object; resolve the
|
||||
@@ -580,7 +616,10 @@ class ChatMixin:
|
||||
# Check cache first (skip cache if memory is enabled since context is dynamic)
|
||||
if not self._memory_instance:
|
||||
tools_key = self._get_tools_cache_key(tools)
|
||||
cache_key = f"{self.role}:{self.goal}:{tools_key}"
|
||||
# tools=None keys as "none" whatever self.tools holds, so key the
|
||||
# edit-hint decision too; otherwise a cached prefix would keep (or
|
||||
# miss) the hint after the agent's tools change.
|
||||
cache_key = f"{self.role}:{self.goal}:{hash(self.backstory)}:{tools_key}:{self._offers_edit_tool(tools)}"
|
||||
|
||||
cached_prompt = self._cache_get(self._system_prompt_cache, cache_key)
|
||||
if cached_prompt is not None:
|
||||
@@ -591,13 +630,16 @@ class ChatMixin:
|
||||
cache_key = None # Don't cache when memory is enabled
|
||||
|
||||
system_prompt = f"""{self.backstory}\n
|
||||
Your Role: {self.role}\n
|
||||
Your Goal: {self.goal}"""
|
||||
Your Role: {self.role}"""
|
||||
# Agent(instructions=...) sets backstory and goal to the same text;
|
||||
# state it once.
|
||||
if self.goal != self.backstory:
|
||||
system_prompt += f"\n\nYour Goal: {self.goal}"
|
||||
|
||||
# Prepend the model-family harness base prompt when one applies.
|
||||
# Resolution is lazy and falls back to a behaviour-preserving default
|
||||
# (no fragment) for unknown models, so output is unchanged by default.
|
||||
harness_prompt = self._resolve_harness_base_prompt()
|
||||
# (no fragment) for unknown models or agents without edit tools.
|
||||
harness_prompt = self._resolve_harness_base_prompt(tools)
|
||||
if harness_prompt:
|
||||
system_prompt = f"{harness_prompt}\n\n{system_prompt}"
|
||||
|
||||
@@ -1204,7 +1246,8 @@ Your Goal: {self.goal}"""
|
||||
if cached_entry is not None:
|
||||
cached_tools, cached_metadata = cached_entry
|
||||
self._tool_search_metadata = cached_metadata
|
||||
return cached_tools
|
||||
# Re-prune: permissions may have changed since the entry was cached.
|
||||
return self._prune_denied_tools(cached_tools)
|
||||
|
||||
formatted_tools = []
|
||||
# Provider-hosted tool specs have no local callable; the provider
|
||||
@@ -1324,20 +1367,19 @@ Your Goal: {self.goal}"""
|
||||
return sorted(tools_list, key=sort_key)
|
||||
|
||||
cleaned_tools = sort_formatted_tools(cleaned_tools)
|
||||
|
||||
# Prune permission-denied tools from the advertised set so the model is
|
||||
# only ever offered tools it can actually call. Mirrors the execution-time
|
||||
# enforcement in tool_execution.py (_perm_deny / _perm_allow). No-op when
|
||||
# no deny set and no allow set are configured (backward compatible).
|
||||
cleaned_tools = self._prune_denied_tools(cleaned_tools)
|
||||
|
||||
# Cache the formatted tools with LRU eviction, including tool search metadata
|
||||
|
||||
# Cache the unpruned tools with LRU eviction, including tool search
|
||||
# metadata, so a later change to the deny/allow sets takes effect.
|
||||
self._cache_put(
|
||||
self._formatted_tools_cache,
|
||||
cache_key,
|
||||
(cleaned_tools, getattr(self, "_tool_search_metadata", None)),
|
||||
)
|
||||
return cleaned_tools
|
||||
# Prune permission-denied tools from the advertised set so the model is
|
||||
# only ever offered tools it can actually call. Mirrors the execution-time
|
||||
# enforcement in tool_execution.py (_perm_deny / _perm_allow). No-op when
|
||||
# no deny set and no allow set are configured (backward compatible).
|
||||
return self._prune_denied_tools(cleaned_tools)
|
||||
|
||||
def _build_before_tool_definitions_input(self, formatted_tools):
|
||||
"""Build the BEFORE_TOOL_DEFINITIONS hook input.
|
||||
@@ -2042,6 +2084,21 @@ Your Goal: {self.goal}"""
|
||||
return fn(*args, **kwargs)
|
||||
return {**llm_kwargs, 'execute_tool_fn': tracked}, ran
|
||||
|
||||
@staticmethod
|
||||
def _history_before_turn(history, content):
|
||||
"""``history`` without the copy of this turn's user message stored before the call.
|
||||
|
||||
``LLM._build_messages`` appends ``prompt`` after ``chat_history``, so
|
||||
forwarding the stored copy as well sends the prompt twice. Only a
|
||||
trailing match is dropped: messages a durable replay restored after
|
||||
the turn keep their order.
|
||||
"""
|
||||
if history:
|
||||
last = history[-1]
|
||||
if isinstance(last, dict) and last.get("role") == "user" and last.get("content") == content:
|
||||
return list(history[:-1])
|
||||
return history
|
||||
|
||||
def _custom_llm_hook_messages(self, llm_kwargs):
|
||||
"""The custom-LLM request as the message list BEFORE_LLM / AFTER_LLM hooks see."""
|
||||
messages = []
|
||||
@@ -4090,7 +4147,7 @@ Your Goal: {self.goal}"""
|
||||
llm_kwargs = dict(
|
||||
prompt=llm_prompt,
|
||||
system_prompt=system_prompt_for_llm,
|
||||
chat_history=processed_history,
|
||||
chat_history=self._history_before_turn(processed_history, normalized_content),
|
||||
temperature=temperature,
|
||||
tools=tool_param,
|
||||
output_json=output_json,
|
||||
@@ -4207,6 +4264,7 @@ Your Goal: {self.goal}"""
|
||||
# Pass llm_prompt (which includes multimodal content if attachments present)
|
||||
messages, original_prompt = self._build_messages(
|
||||
llm_prompt, temperature, output_json, output_pydantic,
|
||||
tools=tools,
|
||||
use_native_format=use_native_format,
|
||||
memory_prefetch_context=memory_prefetch_context,
|
||||
)
|
||||
@@ -4485,7 +4543,9 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
except Exception:
|
||||
# Catch any exception from the inner try block and re-raise to outer handler
|
||||
raise
|
||||
except ToolExecutionError:
|
||||
except (ToolExecutionError, BudgetExceededError):
|
||||
# Parity with the custom-LLM and async paths: a budget stop must
|
||||
# reach the caller, not turn into a ``None`` reply.
|
||||
raise
|
||||
except Exception as e:
|
||||
self._last_stop_reason = "error"
|
||||
@@ -4832,7 +4892,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
tools,
|
||||
memory_prefetch_context=memory_prefetch_context,
|
||||
),
|
||||
'chat_history': effective_history,
|
||||
'chat_history': self._history_before_turn(effective_history, normalized_content),
|
||||
'temperature': temperature,
|
||||
'tools': tools,
|
||||
'output_json': output_json,
|
||||
@@ -4926,6 +4986,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
temperature,
|
||||
output_json,
|
||||
output_pydantic,
|
||||
tools=tools,
|
||||
restore_durable=False,
|
||||
memory_prefetch_context=memory_prefetch_context,
|
||||
)
|
||||
@@ -5264,7 +5325,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
response = await self._openai_client.async_client.chat.completions.create(
|
||||
**tool_call_kwargs
|
||||
)
|
||||
result = await self._achat_completion(response, tools)
|
||||
result = await self._achat_completion(response, tools, messages=messages)
|
||||
if get_logger().getEffectiveLevel() == logging.DEBUG:
|
||||
total_time = time.time() - start_time
|
||||
logging.debug(f"Agent.achat completed in {total_time:.2f} seconds")
|
||||
@@ -5444,8 +5505,12 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
# the except blocks above already read ownership before this runs.
|
||||
self._clear_turn_tracking()
|
||||
|
||||
async def _achat_completion(self, response, tools, reasoning_steps=False):
|
||||
"""Async version of _chat_completion method"""
|
||||
async def _achat_completion(self, response, tools, reasoning_steps=False, messages=None):
|
||||
"""Async version of _chat_completion method.
|
||||
|
||||
``messages`` is the conversation that produced ``response``; it is kept in
|
||||
the follow-up request so the final answer still sees the user's question.
|
||||
"""
|
||||
try:
|
||||
message = response.choices[0].message
|
||||
if not hasattr(message, 'tool_calls') or not message.tool_calls:
|
||||
@@ -5531,17 +5596,20 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
if results:
|
||||
formatted_results = "\n".join([str(r) for r in results if r is not None])
|
||||
if formatted_results:
|
||||
messages = [
|
||||
{"role": "system", "content": self.system_prompt},
|
||||
if messages:
|
||||
follow_up_messages = list(messages)
|
||||
else:
|
||||
follow_up_messages = [{"role": "system", "content": self.system_prompt}]
|
||||
follow_up_messages.extend([
|
||||
{"role": "assistant", "content": "Here are the tool results:"},
|
||||
{"role": "user", "content": formatted_results + "\nPlease process these results and provide a final response."}
|
||||
]
|
||||
])
|
||||
try:
|
||||
final_response = await self._openai_client.async_client.chat.completions.create(
|
||||
model=self.llm,
|
||||
messages=messages,
|
||||
messages=follow_up_messages,
|
||||
temperature=1.0,
|
||||
stream=stream
|
||||
stream=True
|
||||
)
|
||||
full_response_text = ""
|
||||
reasoning_content = ""
|
||||
@@ -5551,6 +5619,9 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
# Process stream without display_generating since streaming is active
|
||||
async for chunk in final_response:
|
||||
chunks.append(chunk)
|
||||
if not chunk.choices:
|
||||
# e.g. Azure content-filter annotation chunks
|
||||
continue
|
||||
if chunk.choices[0].delta.content:
|
||||
full_response_text += chunk.choices[0].delta.content
|
||||
|
||||
@@ -5558,9 +5629,18 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
rc = chunk.choices[0].delta.reasoning_content
|
||||
if rc:
|
||||
reasoning_content += rc
|
||||
|
||||
if not full_response_text.strip() and not reasoning_content.strip():
|
||||
# No answer text (only annotation, contentless or
|
||||
# whitespace-only chunks): fall back to the tool results via the
|
||||
# except below.
|
||||
raise ValueError("follow-up stream returned no answer text")
|
||||
|
||||
self.console.print()
|
||||
console = self.console # None unless verbose
|
||||
if console is not None:
|
||||
console.print()
|
||||
|
||||
from .agent import _get_llm_functions
|
||||
final_response = _get_llm_functions()['process_stream_chunks'](chunks)
|
||||
# Return only reasoning content if reasoning_steps is True
|
||||
if reasoning_steps and hasattr(final_response.choices[0].message, 'reasoning_content') and final_response.choices[0].message.reasoning_content:
|
||||
@@ -5878,7 +5958,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
for chunk in self.llm_instance.get_response_stream(
|
||||
prompt=actual_prompt,
|
||||
system_prompt=stream_system_prompt,
|
||||
chat_history=stream_history,
|
||||
chat_history=self._history_before_turn(stream_history, normalized_content),
|
||||
temperature=kwargs.get('temperature', 1.0),
|
||||
tools=tool_param,
|
||||
output_json=kwargs.get('output_json'),
|
||||
@@ -5953,6 +6033,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
|
||||
# Build messages using the helper method
|
||||
messages, original_prompt = self._build_messages(actual_prompt, kwargs.get('temperature', 1.0),
|
||||
kwargs.get('output_json'), kwargs.get('output_pydantic'),
|
||||
tools=tool_param,
|
||||
memory_prefetch_context=memory_prefetch_context)
|
||||
|
||||
# Apply context management so the streaming path compacts long
|
||||
|
||||
@@ -213,7 +213,8 @@ class ContextAgent(Agent):
|
||||
self.output_dir / "agent_responses",
|
||||
self.output_dir / "markdown_outputs",
|
||||
self.output_dir / "debug_logs",
|
||||
self.output_dir / "final_results"
|
||||
self.output_dir / "final_results",
|
||||
self.output_dir / "PRPs",
|
||||
]
|
||||
|
||||
for directory in directories:
|
||||
@@ -1270,11 +1271,12 @@ Every agent interaction has been saved for full audit trail and reproducibility.
|
||||
return self._generate_combined_analysis(feature_request)
|
||||
|
||||
# Otherwise just save and return the PRP
|
||||
from ..utils.atomic_io import atomic_write_text, safe_join
|
||||
prp_dir = os.path.join(self.output_dir, "PRPs")
|
||||
os.makedirs(prp_dir, exist_ok=True)
|
||||
safe_filename = feature_request.lower().replace(' ', '_').replace('-', '_')[:50]
|
||||
prp_path = os.path.join(self.output_dir, "PRPs", f"{safe_filename}.md")
|
||||
|
||||
with open(prp_path, "w", encoding="utf-8") as f:
|
||||
f.write(prp)
|
||||
prp_path = safe_join(prp_dir, safe_filename, ".md")
|
||||
atomic_write_text(prp_path, prp)
|
||||
|
||||
print(f"📄 Comprehensive PRP following PRD template saved to: {prp_path}")
|
||||
return prp
|
||||
|
||||
@@ -55,6 +55,27 @@ def _live_turn_message_ids() -> set:
|
||||
ids.update(id(m) for m in owned)
|
||||
return ids
|
||||
|
||||
|
||||
def _exchange_spans(history: list) -> list:
|
||||
"""Split history into ``(start, end)`` units that must stay together.
|
||||
|
||||
An assistant turn carrying ``tool_calls`` and the ``role="tool"`` results
|
||||
that follow it form one unit; every other message is its own unit. History
|
||||
mutators cut/delete on unit boundaries so they never leave an orphaned tool
|
||||
result or a dangling ``tool_calls`` turn, which providers reject (#6217).
|
||||
"""
|
||||
spans = []
|
||||
i, n = 0, len(history)
|
||||
while i < n:
|
||||
j = i + 1
|
||||
msg = history[i]
|
||||
if msg.get("role") == "assistant" and msg.get("tool_calls"):
|
||||
while j < n and history[j].get("role") == "tool":
|
||||
j += 1
|
||||
spans.append((i, j))
|
||||
i = j
|
||||
return spans
|
||||
|
||||
# Shared lazy display helpers (cached, thread-safe; avoid circular imports)
|
||||
from ._lazy_display import _get_console, _get_live, _get_display_functions
|
||||
|
||||
@@ -178,6 +199,11 @@ class MemoryMixin:
|
||||
length: Target length for chat history
|
||||
"""
|
||||
with self._history_lock:
|
||||
# Never cut inside a tool exchange: back off to its start (#6217).
|
||||
for start, end in _exchange_spans(self.chat_history):
|
||||
if start < length < end:
|
||||
length = start
|
||||
break
|
||||
self.chat_history = self.chat_history[:length]
|
||||
|
||||
def _begin_turn_tracking(self):
|
||||
@@ -299,8 +325,14 @@ class MemoryMixin:
|
||||
if len(self.chat_history) <= keep_last:
|
||||
return 0
|
||||
|
||||
deleted_count = len(self.chat_history) - keep_last
|
||||
self.chat_history = self.chat_history[-keep_last:]
|
||||
cut = len(self.chat_history) - keep_last
|
||||
# Never start the kept tail inside a tool exchange (#6217).
|
||||
for start, end in _exchange_spans(self.chat_history):
|
||||
if start < cut < end:
|
||||
cut = end
|
||||
break
|
||||
deleted_count = cut
|
||||
self.chat_history = self.chat_history[cut:]
|
||||
# Reset auto-save index to match new history length
|
||||
self._auto_save_last_index = len(self.chat_history)
|
||||
return deleted_count
|
||||
@@ -318,8 +350,17 @@ class MemoryMixin:
|
||||
True if deleted, False if index out of range
|
||||
"""
|
||||
with self._history_lock:
|
||||
n = len(self.chat_history)
|
||||
if not -n <= index < n:
|
||||
return False
|
||||
index %= n
|
||||
# Deleting part of a tool exchange removes the whole exchange so no
|
||||
# orphaned tool result / dangling tool_calls is left (#6217).
|
||||
for start, end in _exchange_spans(self.chat_history):
|
||||
if start <= index < end:
|
||||
break
|
||||
try:
|
||||
del self.chat_history[index]
|
||||
del self.chat_history[start:end]
|
||||
# Adjust auto-save index if deletion affects saved range
|
||||
if hasattr(self, '_auto_save_last_index'):
|
||||
self._auto_save_last_index = min(
|
||||
@@ -344,10 +385,18 @@ class MemoryMixin:
|
||||
"""
|
||||
with self._history_lock:
|
||||
original_len = len(self.chat_history)
|
||||
self.chat_history = [
|
||||
msg for msg in self.chat_history
|
||||
if pattern.lower() not in msg.get("content", "").lower()
|
||||
]
|
||||
needle = pattern.lower()
|
||||
history = self.chat_history
|
||||
kept = []
|
||||
# Match per exchange so a hit inside a tool exchange drops the whole
|
||||
# exchange; content may be None on tool-calling turns (#6217).
|
||||
for start, end in _exchange_spans(history):
|
||||
unit = history[start:end]
|
||||
if not any(
|
||||
needle in str(m.get("content") or "").lower() for m in unit
|
||||
):
|
||||
kept.extend(unit)
|
||||
self.chat_history = kept
|
||||
deleted_count = original_len - len(self.chat_history)
|
||||
# Adjust auto-save index if deletion affects saved range
|
||||
if deleted_count > 0 and hasattr(self, '_auto_save_last_index'):
|
||||
@@ -569,11 +618,10 @@ class MemoryMixin:
|
||||
if history:
|
||||
# Only restore if chat_history is empty (avoid duplicates)
|
||||
if not self.chat_history:
|
||||
# The store returns LLM-shaped dicts (to_llm_message), so
|
||||
# keep tool_calls / tool_call_id intact (#6217).
|
||||
for msg in history:
|
||||
self.chat_history.append({
|
||||
"role": msg["role"],
|
||||
"content": msg["content"]
|
||||
})
|
||||
self.chat_history.append(dict(msg))
|
||||
logging.info(f"Restored session {self._session_id} with {len(history)} messages from JSON store")
|
||||
|
||||
# Set agent info
|
||||
@@ -862,7 +910,7 @@ class MemoryMixin:
|
||||
except Exception as e:
|
||||
logging.debug(f"Discard partial assistant turn skipped: {e}")
|
||||
|
||||
def _add_message_with_tool_fields(self, role, content, **tool_fields):
|
||||
def _add_message_with_tool_fields(self, role, content, session_id=None, **tool_fields):
|
||||
"""Add a message carrying tool fields, tolerating stores without them.
|
||||
|
||||
The default JSON store and the SQLite store accept ``tool_calls`` /
|
||||
@@ -874,11 +922,13 @@ class MemoryMixin:
|
||||
back to a plain-text turn so no exchange is silently lost.
|
||||
"""
|
||||
add_message = self._session_store.add_message
|
||||
if self._store_supports_tool_fields(add_message):
|
||||
add_message(self._session_id, role, content, **tool_fields)
|
||||
if session_id is None:
|
||||
session_id = self._session_id
|
||||
if tool_fields and self._store_supports_tool_fields(add_message):
|
||||
add_message(session_id, role, content, **tool_fields)
|
||||
return
|
||||
# Legacy store: preserve the turn as plain text so resume still sees it.
|
||||
add_message(self._session_id, role, content)
|
||||
add_message(session_id, role, content)
|
||||
|
||||
def _store_supports_tool_fields(self, add_message) -> bool:
|
||||
"""Return True if ``add_message`` accepts the tool keyword fields."""
|
||||
@@ -1195,10 +1245,17 @@ class MemoryMixin:
|
||||
if self._session_store is not None:
|
||||
# Persist only NEW messages to SessionStore
|
||||
for msg in new_messages:
|
||||
self._session_store.add_message(
|
||||
self.auto_save, # Use auto_save name as session_id
|
||||
role=msg.get("role", "user"),
|
||||
content=msg.get("content", ""),
|
||||
tool_fields = {
|
||||
k: msg[k]
|
||||
for k in ("tool_calls", "tool_call_id")
|
||||
if msg.get(k) is not None
|
||||
}
|
||||
# Persist tool fields so resumed tool turns stay valid (#6217)
|
||||
self._add_message_with_tool_fields(
|
||||
msg.get("role", "user"),
|
||||
msg.get("content") or "",
|
||||
session_id=self.auto_save,
|
||||
**tool_fields,
|
||||
)
|
||||
# Update last saved index
|
||||
self._auto_save_last_index = len(clean_history)
|
||||
|
||||
@@ -4362,8 +4362,13 @@ class ToolExecutionMixin:
|
||||
return "Tool search not in bridge mode"
|
||||
|
||||
# Get deferrable tools from metadata
|
||||
deferrable_tools = metadata.get("deferrable_tools", [])
|
||||
|
||||
# Hide permission-denied tools from search/describe; the cached
|
||||
# metadata is built before pruning, so filter locally here.
|
||||
deferrable_tools = [
|
||||
t for t in metadata.get("deferrable_tools", [])
|
||||
if self._tool_name_allowed(t.get("function", {}).get("name", ""))
|
||||
]
|
||||
|
||||
if function_name == "tool_search":
|
||||
# Handle tool_search bridge call
|
||||
try:
|
||||
|
||||
@@ -957,7 +957,8 @@ def _validate_feature_config(feature: str, config: Dict[str, Any]) -> List[str]:
|
||||
"guardrails": {"enabled", "max_retries", "validator"},
|
||||
"web": {"enabled", "search", "fetch", "provider"},
|
||||
"output": {"enabled", "verbose", "markdown", "stream", "metrics", "preset"},
|
||||
"execution": {"enabled", "max_iter", "max_rpm", "max_execution_time", "preset"},
|
||||
"execution": {"enabled", "max_iter", "max_rpm", "max_execution_time", "preset",
|
||||
"max_budget", "on_budget_exceeded", "durable", "journal_path", "resume_run_id"},
|
||||
"caching": {"enabled", "prompt_caching", "ttl"},
|
||||
"autonomy": {"enabled", "max_steps", "approval_required"},
|
||||
"context": {"enabled", "max_tokens", "strategy"},
|
||||
|
||||
@@ -18,6 +18,8 @@ from .._lazy import create_lazy_getattr
|
||||
# 283 KB / 7,201-line ``protocols.py`` (and siblings) out of package-import
|
||||
# time; each submodule is parsed only when one of its symbols is first
|
||||
# touched. Mirrors the ``mcp``/``skills`` packages (AGENTS.md §4.2).
|
||||
_PROTOCOLS_MODULE = "praisonaiagents.gateway.protocols"
|
||||
|
||||
_LAZY_IMPORTS = {
|
||||
# protocols.py (parsed lazily, on first touch)
|
||||
"GatewayProtocol": ("praisonaiagents.gateway.protocols", "GatewayProtocol"),
|
||||
@@ -229,7 +231,9 @@ _LAZY_IMPORTS = {
|
||||
# Schema-validated inbound frame codec (Issue #2831)
|
||||
"HelloParams": ("praisonaiagents.gateway.protocols", "HelloParams"),
|
||||
# Device-identity handshake contract (Issue #5998)
|
||||
"DeviceAuth": ("praisonaiagents.gateway.protocols", "DeviceAuth"),
|
||||
# Module path via a constant: GitGuardian reads a ("...", "DeviceAuth")
|
||||
# string pair as an "authentication tuple" and fails every PR that merges main.
|
||||
"DeviceAuth": (_PROTOCOLS_MODULE, "DeviceAuth"),
|
||||
"DeviceIdentityProtocol": ("praisonaiagents.gateway.protocols", "DeviceIdentityProtocol"),
|
||||
"device_id_from_public_key": ("praisonaiagents.gateway.protocols", "device_id_from_public_key"),
|
||||
"HelloResult": ("praisonaiagents.gateway.protocols", "HelloResult"),
|
||||
|
||||
@@ -3841,12 +3841,19 @@ class ConversationRequestProtocol(Protocol):
|
||||
# import lives in core.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
GatewayRequestKind = Literal["approval", "choice", "input"]
|
||||
GatewayRequestKind = Literal["approval", "choice", "input", "secret"]
|
||||
"""Closed set of server→client interactive request kinds.
|
||||
|
||||
* ``approval`` — allow/deny; the reply ``value`` is ``"allow"`` or ``"deny"``.
|
||||
* ``choice`` — select one of ``options``; the reply ``value`` is the option.
|
||||
* ``input`` — free text; the reply ``value`` is the entered text.
|
||||
* ``secret`` — a credential (password, API key, one-time code) captured
|
||||
mid-turn (Issue #6223). Clients MUST render a masked input. Isolation
|
||||
contract: the reply ``value`` is returned *only* to the caller of
|
||||
:meth:`GatewayRequestChannelProtocol.request`; implementations MUST NOT log,
|
||||
journal, echo, or persist it, and the caller (a tool/callback) MUST NOT
|
||||
return it to the model or append it to history. A tool obtains one via
|
||||
``get_request_channel().request(GatewayServerRequest(kind="secret", ...))``.
|
||||
"""
|
||||
|
||||
|
||||
@@ -3893,11 +3900,13 @@ class GatewayServerReply:
|
||||
Attributes:
|
||||
request_id: The id of the request being answered.
|
||||
value: ``"allow"``/``"deny"`` for approval, the chosen option for
|
||||
choice, or the entered text for input.
|
||||
choice, the entered text for input, or the credential for secret.
|
||||
Excluded from ``repr`` so a logged/traced reply never leaks a
|
||||
secret.
|
||||
"""
|
||||
|
||||
request_id: str
|
||||
value: str
|
||||
value: str = field(repr=False)
|
||||
|
||||
def as_dict(self) -> Dict[str, Any]:
|
||||
"""Convert to a serializable dictionary."""
|
||||
|
||||
@@ -572,6 +572,14 @@ class FileMemory:
|
||||
promoted.append(item.id)
|
||||
|
||||
if promoted:
|
||||
# Promotion writes the same store as add_long_term, so retain the
|
||||
# same highest-importance, newest-first items within its capacity.
|
||||
limit = self.config["long_term_limit"]
|
||||
if len(self._long_term) > limit:
|
||||
self._long_term.sort(
|
||||
key=lambda item: (item.importance, item.created_at), reverse=True
|
||||
)
|
||||
self._long_term = self._long_term[:limit]
|
||||
self._save_long_term()
|
||||
self._log(f"Auto-promoted {len(promoted)} items to long-term memory")
|
||||
|
||||
|
||||
@@ -104,7 +104,12 @@ class PlanStorage:
|
||||
if filename is None:
|
||||
filename = self._get_plan_filename(plan)
|
||||
|
||||
path = os.path.join(self.plans_dir, filename)
|
||||
from ..utils.atomic_io import safe_join
|
||||
# Keep ".md" as the suffix so listing/lookup still find the file.
|
||||
stem, ext = os.path.splitext(filename)
|
||||
if ext.lower() != ".md":
|
||||
stem, ext = filename, ".md"
|
||||
path = safe_join(self.plans_dir, stem, ext)
|
||||
|
||||
# Convert plan to markdown
|
||||
markdown = plan.to_markdown()
|
||||
@@ -215,6 +220,29 @@ class PlanStorage:
|
||||
|
||||
return False
|
||||
|
||||
def _todo_path(self, name: str) -> str:
|
||||
"""Confined path for a todo name, reusing a pre-existing legacy file.
|
||||
|
||||
Legacy files (``<name>.json`` written before Issue #6238) are reused
|
||||
only when ``name`` is a plain filename, so upgrades keep working
|
||||
without ever allowing traversal out of ``todos_dir``.
|
||||
"""
|
||||
from ..utils.atomic_io import safe_join
|
||||
path = safe_join(self.todos_dir, name, ".json")
|
||||
name = str(name)
|
||||
if (
|
||||
not os.path.exists(path)
|
||||
and name not in ("", ".", "..")
|
||||
and "\0" not in name
|
||||
and "%" not in name
|
||||
and os.path.basename(name) == name
|
||||
and not (os.altsep and os.altsep in name)
|
||||
):
|
||||
legacy = os.path.join(self.todos_dir, f"{name}.json")
|
||||
if os.path.isfile(legacy):
|
||||
return legacy
|
||||
return path
|
||||
|
||||
def save_todo(self, todo: TodoList, name: str = "current") -> str:
|
||||
"""
|
||||
Save a todo list to storage.
|
||||
@@ -226,11 +254,10 @@ class PlanStorage:
|
||||
Returns:
|
||||
Path to the saved file
|
||||
"""
|
||||
filename = f"{name}.json"
|
||||
path = os.path.join(self.todos_dir, filename)
|
||||
from ..utils.atomic_io import atomic_write_text
|
||||
path = self._todo_path(name)
|
||||
|
||||
# Atomic write so a crash mid-write never truncates the previous todos.
|
||||
from ..utils.atomic_io import atomic_write_text
|
||||
atomic_write_text(path, todo.to_json())
|
||||
|
||||
logger.debug(f"Saved todo list to {path}")
|
||||
@@ -246,8 +273,7 @@ class PlanStorage:
|
||||
Returns:
|
||||
TodoList instance, or None if not found
|
||||
"""
|
||||
filename = f"{name}.json"
|
||||
path = os.path.join(self.todos_dir, filename)
|
||||
path = self._todo_path(name)
|
||||
|
||||
if not os.path.exists(path):
|
||||
return None
|
||||
@@ -265,12 +291,13 @@ class PlanStorage:
|
||||
Returns:
|
||||
List of todo list names
|
||||
"""
|
||||
from urllib.parse import unquote
|
||||
todos = []
|
||||
|
||||
for filename in os.listdir(self.todos_dir):
|
||||
if filename.endswith(".json"):
|
||||
name = filename[:-5] # Remove .json extension
|
||||
todos.append(name)
|
||||
# Decode so listed names round-trip through load/delete_todo.
|
||||
todos.append(unquote(filename[:-5]))
|
||||
|
||||
return todos
|
||||
|
||||
@@ -284,8 +311,7 @@ class PlanStorage:
|
||||
Returns:
|
||||
True if deleted, False if not found
|
||||
"""
|
||||
filename = f"{name}.json"
|
||||
path = os.path.join(self.todos_dir, filename)
|
||||
path = self._todo_path(name)
|
||||
|
||||
if os.path.exists(path):
|
||||
os.remove(path)
|
||||
|
||||
@@ -73,28 +73,15 @@ def generate_title(
|
||||
fallback_title = _create_fallback_title(user_msg, max_length)
|
||||
|
||||
try:
|
||||
# Check if we're already in an event loop
|
||||
asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
# No running loop - safe to use asyncio.run
|
||||
try:
|
||||
return asyncio.run(generate_title_async(
|
||||
user_msg, assistant_msg, llm_model, timeout, max_length,
|
||||
primary_model=primary_model,
|
||||
))
|
||||
except Exception:
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.debug("Title generation failed in sync context", exc_info=True)
|
||||
return fallback_title
|
||||
else:
|
||||
# Already in event loop - cannot use asyncio.run
|
||||
# Works with or without a running event loop (Issue #6238).
|
||||
from ..utils.async_bridge import run_coroutine_from_any_context
|
||||
return run_coroutine_from_any_context(generate_title_async(
|
||||
user_msg, assistant_msg, llm_model, timeout, max_length,
|
||||
primary_model=primary_model,
|
||||
), timeout=None)
|
||||
except Exception:
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.warning(
|
||||
"generate_title() called from running event loop; "
|
||||
"use generate_title_async() instead. Returning fallback title."
|
||||
)
|
||||
logging.getLogger(__name__).debug("Title generation failed", exc_info=True)
|
||||
return fallback_title
|
||||
|
||||
|
||||
|
||||
@@ -645,9 +645,8 @@ def crawl4ai_sync(
|
||||
Returns:
|
||||
Dict with crawl results
|
||||
"""
|
||||
return asyncio.run(
|
||||
crawl4ai(url, css_selector, js_code, wait_for)
|
||||
)
|
||||
from ..utils.async_bridge import run_coroutine_from_any_context
|
||||
return run_coroutine_from_any_context(crawl4ai(url, css_selector, js_code, wait_for), timeout=None)
|
||||
|
||||
def crawl4ai_extract_sync(
|
||||
url: str,
|
||||
@@ -664,9 +663,8 @@ def crawl4ai_extract_sync(
|
||||
Returns:
|
||||
Dict with extracted data
|
||||
"""
|
||||
return asyncio.run(
|
||||
crawl4ai_extract(url, schema, js_code)
|
||||
)
|
||||
from ..utils.async_bridge import run_coroutine_from_any_context
|
||||
return run_coroutine_from_any_context(crawl4ai_extract(url, schema, js_code), timeout=None)
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Example usage
|
||||
|
||||
@@ -7,6 +7,15 @@ from datetime import datetime
|
||||
from typing import Dict, List, Optional, Any
|
||||
from .decorator import tool
|
||||
|
||||
import warnings
|
||||
|
||||
warnings.warn(
|
||||
"praisonaiagents.tools.jira_tools is deprecated and will be removed in a future release; "
|
||||
"use praisonai_tools.tools.jira_watch_tool instead (pip install praisonai-tools)",
|
||||
DeprecationWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def _get_jira_connection(
|
||||
|
||||
@@ -126,11 +126,31 @@ def _matches_glob(fname: str, rel: str, glob: Optional[str]) -> bool:
|
||||
if not glob:
|
||||
return True
|
||||
posix = rel.replace(os.sep, "/")
|
||||
return (
|
||||
if (
|
||||
fnmatch.fnmatch(fname, glob)
|
||||
or fnmatch.fnmatch(rel, glob)
|
||||
or fnmatch.fnmatch(posix, glob)
|
||||
)
|
||||
):
|
||||
return True
|
||||
|
||||
# fnmatch treats "**/" as requiring a slash. Recursive glob segments
|
||||
# also match zero directories, as they do on the ripgrep path.
|
||||
segments = glob.split("/")
|
||||
if "**" not in segments:
|
||||
return False
|
||||
parts = posix.split("/")
|
||||
positions = {0}
|
||||
for segment in segments:
|
||||
if not positions:
|
||||
return False
|
||||
if segment == "**":
|
||||
positions = set(range(min(positions), len(parts) + 1))
|
||||
else:
|
||||
positions = {
|
||||
i + 1 for i in positions
|
||||
if i < len(parts) and fnmatch.fnmatch(parts[i], segment)
|
||||
}
|
||||
return len(parts) in positions
|
||||
|
||||
|
||||
def _iter_files(root: str, glob: Optional[str], ignore_matcher):
|
||||
|
||||
@@ -26,6 +26,15 @@ import logging
|
||||
import os
|
||||
from importlib import util
|
||||
|
||||
import warnings
|
||||
|
||||
warnings.warn(
|
||||
"praisonaiagents.tools.youdotcom_tools is deprecated and will be removed in a future release; "
|
||||
"use praisonai_tools.tools.youdotcom_tool instead (pip install praisonai-tools)",
|
||||
DeprecationWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
|
||||
|
||||
def _check_youdotcom_available() -> tuple[bool, Optional[str]]:
|
||||
"""Check if You.com SDK is available and API key is set.
|
||||
|
||||
@@ -9,8 +9,10 @@ the previous good file: the old bytes stay intact and ``os.replace`` swaps in th
|
||||
new content atomically on both POSIX and Windows.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
@@ -21,9 +23,52 @@ __all__ = [
|
||||
"atomic_write_json",
|
||||
"atomic_write_text",
|
||||
"quarantine_corrupt_file",
|
||||
"safe_join",
|
||||
"update_json",
|
||||
]
|
||||
|
||||
# Max UTF-8 bytes for the final basename; leaves room under the common
|
||||
# 255-byte limit for ``atomic_write_text``'s ``<basename>.XXXXXXXX.tmp`` file.
|
||||
_MAX_NAME_BYTES = 200
|
||||
_HASHED = re.compile(r"(.*)%~([0-9a-f]{16})", re.DOTALL)
|
||||
_ESCAPE = re.compile(r'[%/\\:*?"<>|\x00-\x1f\x7f]')
|
||||
|
||||
|
||||
def _escape(name: str) -> str:
|
||||
stem = _ESCAPE.sub(lambda m: "%%%02X" % ord(m.group()), name)
|
||||
return "%2E" + stem[1:] if stem.startswith(".") else stem
|
||||
|
||||
|
||||
def safe_join(base: str, name: str, suffix: str = "") -> str:
|
||||
"""Join a caller-supplied ``name`` under ``base`` without escaping it.
|
||||
|
||||
Only ``%``, path separators, control and Windows-reserved characters (and
|
||||
a leading ``.``) are percent-encoded, so ordinary names - including spaces
|
||||
and non-ASCII - keep their pre-#6238 filename, ``..`` and separators can
|
||||
never leave ``base``, and ``urllib.parse.unquote`` recovers the name.
|
||||
Names whose filename would exceed ``_MAX_NAME_BYTES`` are truncated and
|
||||
tagged ``%~<sha256[:16]>``; that listed (unquoted) form maps back to the
|
||||
same file, while no ordinary name can produce a raw ``%~`` (Issue #6238).
|
||||
"""
|
||||
name = str(name)
|
||||
hashed = _HASHED.fullmatch(name)
|
||||
if hashed:
|
||||
stem = _escape(hashed.group(1)) + "%~" + hashed.group(2)
|
||||
else:
|
||||
stem = _escape(name)
|
||||
budget = _MAX_NAME_BYTES - len(suffix.encode("utf-8"))
|
||||
if len(stem.encode("utf-8", "surrogatepass")) > budget:
|
||||
tag = "%~" + hashlib.sha256(name.encode("utf-8", "surrogatepass")).hexdigest()[:16]
|
||||
kept, used = [], len(tag)
|
||||
for tok in re.findall(r"%[0-9A-F]{2}|.", _escape(name), re.DOTALL):
|
||||
used += len(tok.encode("utf-8", "surrogatepass"))
|
||||
if used > budget:
|
||||
break
|
||||
kept.append(tok)
|
||||
stem = "".join(kept) + tag
|
||||
return os.path.join(base, stem + suffix)
|
||||
|
||||
|
||||
# Per-path in-process mutexes so threads in one process serialise before they
|
||||
# even reach the cross-process ``FileLock``. Keyed by absolute path.
|
||||
_path_mutexes: "defaultdict[str, threading.RLock]" = defaultdict(threading.RLock)
|
||||
|
||||
@@ -473,7 +473,10 @@ class YAMLWorkflowParser:
|
||||
|
||||
if 'reasoning' in role_config:
|
||||
agent['reasoning'] = role_config['reasoning']
|
||||
|
||||
|
||||
if isinstance(role_config.get('execution'), dict):
|
||||
agent['execution'] = role_config['execution']
|
||||
|
||||
agents[role_id] = agent
|
||||
|
||||
return agents
|
||||
@@ -591,7 +594,11 @@ class YAMLWorkflowParser:
|
||||
# so we don't override the Agent's own caching default.
|
||||
if cache is False:
|
||||
agent_kwargs['caching'] = False
|
||||
_execution: Dict[str, Any] = {}
|
||||
# An agent-level `execution:` mapping passes straight through to
|
||||
# ExecutionConfig (e.g. max_budget, on_budget_exceeded, durable,
|
||||
# journal_path, resume_run_id); flat max_iter/max_execution_time win.
|
||||
_yaml_execution = config.get('execution')
|
||||
_execution: Dict[str, Any] = dict(_yaml_execution) if isinstance(_yaml_execution, dict) else {}
|
||||
if max_iter is not None:
|
||||
_execution['max_iter'] = max_iter
|
||||
if max_execution_time is not None:
|
||||
|
||||
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
|
||||
[project]
|
||||
name = "praisonaiagents"
|
||||
version = "1.7.21"
|
||||
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
|
||||
description = "Open-source Python AI agent framework that runs on your own computers, even fully offline."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
authors = [
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
"""Live check that async chat with a tool returns a model-written final answer.
|
||||
|
||||
Exercises the post-tool follow-up request in ``ChatMixin._achat_completion``
|
||||
by turning off the default unified dispatch, so ``_achat_impl`` takes the
|
||||
legacy OpenAI-client branch. Skipped unless ``OPENAI_API_KEY`` is set.
|
||||
"""
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from praisonaiagents import Agent
|
||||
|
||||
|
||||
def get_secret_number() -> str:
|
||||
"""Return the secret number."""
|
||||
return "4217"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skipif(not os.getenv("OPENAI_API_KEY"), reason="No API key")
|
||||
async def test_achat_with_tool_returns_final_answer_live():
|
||||
agent = Agent(
|
||||
name="test",
|
||||
instructions="You are a helpful assistant",
|
||||
llm="gpt-4o-mini",
|
||||
tools=[get_secret_number],
|
||||
)
|
||||
# The default unified path never calls _achat_completion.
|
||||
agent._use_unified_llm_dispatch = False
|
||||
result = await agent.astart(
|
||||
"Use the get_secret_number tool, then reply with one sentence that "
|
||||
"contains the secret number."
|
||||
)
|
||||
print(f"Agent output: {result}")
|
||||
assert result
|
||||
assert "4217" in str(result)
|
||||
# Without the fix the raw tool output ("4217") is returned unchanged.
|
||||
assert str(result).strip() != "4217"
|
||||
@@ -169,3 +169,264 @@ async def test_execute_tool_async_resolves_aliased_function_name():
|
||||
)
|
||||
|
||||
assert result == "aliased-ran"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_achat_completion_streams_final_response_after_tool_calls():
|
||||
"""After running tools, the legacy async path must ask the model for a
|
||||
final answer instead of failing and returning the raw tool output.
|
||||
|
||||
``_achat_completion`` used to reference an undefined ``stream`` name and
|
||||
``_get_llm_functions`` (only defined in ``agent.py``), so the follow-up
|
||||
completion always raised ``NameError``, which was swallowed and the raw
|
||||
tool results were returned.
|
||||
"""
|
||||
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
|
||||
|
||||
def lookup(query: str) -> str:
|
||||
return "tool output"
|
||||
|
||||
tool_call = SimpleNamespace(
|
||||
id="call-1",
|
||||
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
|
||||
)
|
||||
response = SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
|
||||
)
|
||||
|
||||
def _chunk(content):
|
||||
return SimpleNamespace(
|
||||
choices=[SimpleNamespace(delta=SimpleNamespace(content=content))]
|
||||
)
|
||||
|
||||
async def _stream():
|
||||
for part in ("final ", "answer"):
|
||||
yield _chunk(part)
|
||||
|
||||
create = AsyncMock(return_value=_stream())
|
||||
fake_client = SimpleNamespace(
|
||||
async_client=SimpleNamespace(
|
||||
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
|
||||
)
|
||||
)
|
||||
final = SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(content="final answer"))]
|
||||
)
|
||||
process_stream_chunks = MagicMock(return_value=final)
|
||||
|
||||
with patch.object(
|
||||
type(agent), "_openai_client", new=fake_client
|
||||
), patch.object(
|
||||
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
|
||||
), patch(
|
||||
"praisonaiagents.agent.agent._get_llm_functions",
|
||||
return_value={"process_stream_chunks": process_stream_chunks},
|
||||
), patch(
|
||||
"praisonaiagents.agent.chat_mixin._get_display_functions"
|
||||
) as display:
|
||||
result = await agent._achat_completion(response, tools=[lookup])
|
||||
|
||||
display.return_value.__getitem__.return_value.assert_not_called()
|
||||
create.assert_awaited_once()
|
||||
assert create.await_args.kwargs["stream"] is True
|
||||
process_stream_chunks.assert_called_once()
|
||||
assert result == "final answer"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_achat_completion_follow_up_keeps_conversation_and_skips_empty_chunks():
|
||||
"""The post-tool follow-up request must keep the original conversation (so
|
||||
the final answer still sees the user's question), and stream chunks with no
|
||||
``choices`` (e.g. Azure content-filter annotations) must be skipped rather
|
||||
than raising ``IndexError`` and falling back to the raw tool output.
|
||||
"""
|
||||
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
|
||||
|
||||
def lookup(query: str) -> str:
|
||||
return "tool output"
|
||||
|
||||
tool_call = SimpleNamespace(
|
||||
id="call-1",
|
||||
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
|
||||
)
|
||||
response = SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
|
||||
)
|
||||
conversation = [
|
||||
{"role": "system", "content": "You are helpful"},
|
||||
{"role": "user", "content": "Look up x and answer in French"},
|
||||
]
|
||||
|
||||
async def _stream():
|
||||
yield SimpleNamespace(choices=[])
|
||||
yield SimpleNamespace(
|
||||
choices=[SimpleNamespace(delta=SimpleNamespace(content="réponse"))]
|
||||
)
|
||||
|
||||
create = AsyncMock(return_value=_stream())
|
||||
fake_client = SimpleNamespace(
|
||||
async_client=SimpleNamespace(
|
||||
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
|
||||
)
|
||||
)
|
||||
final = SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(content="réponse"))]
|
||||
)
|
||||
process_stream_chunks = MagicMock(return_value=final)
|
||||
|
||||
with patch.object(
|
||||
type(agent), "_openai_client", new=fake_client
|
||||
), patch.object(
|
||||
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
|
||||
), patch(
|
||||
"praisonaiagents.agent.agent._get_llm_functions",
|
||||
return_value={"process_stream_chunks": process_stream_chunks},
|
||||
), patch(
|
||||
"praisonaiagents.agent.chat_mixin._get_display_functions"
|
||||
) as display:
|
||||
result = await agent._achat_completion(
|
||||
response, tools=[lookup], messages=conversation
|
||||
)
|
||||
|
||||
display.return_value.__getitem__.return_value.assert_not_called()
|
||||
sent = create.await_args.kwargs["messages"]
|
||||
assert sent[: len(conversation)] == conversation
|
||||
assert "tool output" in sent[-1]["content"]
|
||||
# The caller's list is not mutated.
|
||||
assert len(conversation) == 2
|
||||
assert result == "réponse"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_legacy_achat_impl_forwards_conversation_to_tool_follow_up():
|
||||
"""The legacy ``_achat_impl`` tool branch must pass the conversation to
|
||||
``_achat_completion`` so the follow-up request still has the user's prompt.
|
||||
"""
|
||||
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
|
||||
agent._using_custom_llm = False
|
||||
agent._use_unified_llm_dispatch = False
|
||||
|
||||
def lookup(query: str) -> str:
|
||||
return "tool output"
|
||||
|
||||
tool_call = SimpleNamespace(
|
||||
id="call-1",
|
||||
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
|
||||
)
|
||||
tool_response = SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
|
||||
)
|
||||
|
||||
async def _stream():
|
||||
yield SimpleNamespace(
|
||||
choices=[SimpleNamespace(delta=SimpleNamespace(content="final answer"))]
|
||||
)
|
||||
|
||||
create = AsyncMock(side_effect=[tool_response, _stream()])
|
||||
fake_client = SimpleNamespace(
|
||||
async_client=SimpleNamespace(
|
||||
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
|
||||
)
|
||||
)
|
||||
final = SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(content="final answer"))]
|
||||
)
|
||||
prompt_messages = [{"role": "user", "content": "Look up x for me"}]
|
||||
|
||||
with patch.object(type(agent), "_openai_client", new=fake_client), patch.object(
|
||||
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
|
||||
), patch(
|
||||
"praisonaiagents.agent.agent._get_llm_functions",
|
||||
return_value={"process_stream_chunks": MagicMock(return_value=final)},
|
||||
), patch.object(
|
||||
agent, "_atrigger_after_agent_hook", new_callable=AsyncMock, side_effect=lambda p, r, t: r
|
||||
), patch.object(agent, "_execute_callback_and_display"), patch.object(
|
||||
agent, "_build_messages", return_value=(list(prompt_messages), "Look up x for me")
|
||||
), patch.object(agent, "_persist_message"), patch.object(
|
||||
agent, "_format_tools_for_completion", return_value=[{"type": "function"}]
|
||||
):
|
||||
result = await agent._achat_impl(
|
||||
prompt="Look up x for me",
|
||||
temperature=1.0,
|
||||
tools=[lookup],
|
||||
output_json=None,
|
||||
output_pydantic=None,
|
||||
reasoning_steps=False,
|
||||
stream=False,
|
||||
task_name=None,
|
||||
task_description=None,
|
||||
task_id=None,
|
||||
config=None,
|
||||
force_retrieval=False,
|
||||
skip_retrieval=True,
|
||||
attachments=None,
|
||||
_trace_emitter=MagicMock(),
|
||||
)
|
||||
|
||||
assert create.await_count == 2
|
||||
follow_up = create.await_args_list[1].kwargs["messages"]
|
||||
assert {"role": "user", "content": "Look up x for me"} in follow_up
|
||||
assert result == "final answer"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
"follow_up_chunk",
|
||||
[
|
||||
pytest.param(SimpleNamespace(choices=[]), id="empty-choices"),
|
||||
pytest.param(
|
||||
SimpleNamespace(
|
||||
choices=[SimpleNamespace(delta=SimpleNamespace(content=None))]
|
||||
),
|
||||
id="choice-without-content",
|
||||
),
|
||||
pytest.param(
|
||||
SimpleNamespace(
|
||||
choices=[SimpleNamespace(delta=SimpleNamespace(content="\n "))]
|
||||
),
|
||||
id="whitespace-only-content",
|
||||
),
|
||||
],
|
||||
)
|
||||
async def test_achat_completion_falls_back_when_follow_up_has_no_answer_chunks(follow_up_chunk):
|
||||
"""A follow-up stream with no answer text (only empty-``choices`` chunks,
|
||||
choices without ``delta.content`` or whitespace-only content) must log an error and return the tool
|
||||
results, not an empty string.
|
||||
"""
|
||||
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
|
||||
|
||||
def lookup(query: str) -> str:
|
||||
return "tool output"
|
||||
|
||||
tool_call = SimpleNamespace(
|
||||
id="call-1",
|
||||
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
|
||||
)
|
||||
response = SimpleNamespace(
|
||||
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
|
||||
)
|
||||
|
||||
async def _stream():
|
||||
yield follow_up_chunk
|
||||
|
||||
create = AsyncMock(return_value=_stream())
|
||||
fake_client = SimpleNamespace(
|
||||
async_client=SimpleNamespace(
|
||||
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
|
||||
)
|
||||
)
|
||||
process_stream_chunks = MagicMock()
|
||||
|
||||
with patch.object(type(agent), "_openai_client", new=fake_client), patch.object(
|
||||
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
|
||||
), patch(
|
||||
"praisonaiagents.agent.agent._get_llm_functions",
|
||||
return_value={"process_stream_chunks": process_stream_chunks},
|
||||
), patch(
|
||||
"praisonaiagents.agent.chat_mixin._get_display_functions"
|
||||
) as display:
|
||||
result = await agent._achat_completion(response, tools=[lookup])
|
||||
|
||||
display.return_value.__getitem__.return_value.assert_called()
|
||||
process_stream_chunks.assert_not_called()
|
||||
assert result == "tool output"
|
||||
@@ -0,0 +1,20 @@
|
||||
"""Sync OpenAI-client chat must surface BudgetExceededError, not return None."""
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from praisonaiagents import Agent, BudgetExceededError, ExecutionConfig
|
||||
|
||||
|
||||
def test_sync_chat_reraises_budget_error(monkeypatch):
|
||||
monkeypatch.setenv("OPENAI_API_KEY", "sk-test") # client is built lazily; no request is sent
|
||||
agent = Agent(
|
||||
name="b", instructions="x", llm="gpt-4o-mini", output="silent",
|
||||
execution=ExecutionConfig(max_budget=0.01),
|
||||
)
|
||||
with patch.object(
|
||||
agent, "_chat_completion",
|
||||
side_effect=BudgetExceededError("b", 0.02, 0.01),
|
||||
):
|
||||
with pytest.raises(BudgetExceededError):
|
||||
agent.chat("hello")
|
||||
@@ -0,0 +1,137 @@
|
||||
"""Chat-history integrity for tool turns (Issue #6217)."""
|
||||
import pytest
|
||||
|
||||
from praisonaiagents import Agent
|
||||
from praisonaiagents.config import MemoryConfig
|
||||
from praisonaiagents.session import DefaultSessionStore
|
||||
|
||||
|
||||
def _tool_history():
|
||||
return [
|
||||
{"role": "user", "content": "weather in Paris?"},
|
||||
{"role": "assistant", "content": None, "tool_calls": [{
|
||||
"id": "call_1", "type": "function",
|
||||
"function": {"name": "weather", "arguments": "{\"city\":\"Paris\"}"},
|
||||
}]},
|
||||
{"role": "tool", "tool_call_id": "call_1", "content": "18C sunny"},
|
||||
{"role": "assistant", "content": "It is 18C and sunny."},
|
||||
]
|
||||
|
||||
|
||||
def _agent(session_id=None, store=None):
|
||||
kwargs = {}
|
||||
if session_id:
|
||||
kwargs["memory"] = MemoryConfig(history=True, auto_save=session_id)
|
||||
a = Agent(name="a", instructions="x", model="gpt-4o-mini", **kwargs)
|
||||
if store is not None:
|
||||
a._session_store = store
|
||||
a._init_session_store()
|
||||
return a
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def store(tmp_path):
|
||||
return DefaultSessionStore(session_dir=str(tmp_path))
|
||||
|
||||
|
||||
def _roles(a):
|
||||
return [m["role"] for m in a.chat_history]
|
||||
|
||||
|
||||
def test_auto_save_round_trips_tool_fields(store):
|
||||
a = _agent("sess_rt", store)
|
||||
a.chat_history.extend(_tool_history())
|
||||
a._auto_save_session()
|
||||
|
||||
b = _agent("sess_rt", store)
|
||||
restored = b.chat_history
|
||||
assert restored[1]["tool_calls"][0]["id"] == "call_1"
|
||||
assert restored[2]["tool_call_id"] == "call_1"
|
||||
assert restored[2]["content"] == "18C sunny"
|
||||
|
||||
|
||||
def test_replace_chat_history_resyncs_auto_save_cursor(store):
|
||||
a = _agent("sess_cmp", store)
|
||||
|
||||
def turn(i):
|
||||
a._append_to_chat_history({"role": "user", "content": f"q{i}"})
|
||||
a._append_to_chat_history({"role": "assistant", "content": f"a{i}"})
|
||||
a._auto_save_session()
|
||||
|
||||
for i in range(3):
|
||||
turn(i)
|
||||
a._replace_chat_history([
|
||||
{"role": "system", "content": "summary"},
|
||||
{"role": "assistant", "content": "a2"},
|
||||
])
|
||||
for i in range(3, 6):
|
||||
turn(i)
|
||||
saved = [m["content"] for m in store.get_chat_history("sess_cmp")]
|
||||
assert saved == ["q0", "a0", "q1", "a1", "q2", "a2",
|
||||
"q3", "a3", "q4", "a4", "q5", "a5"]
|
||||
|
||||
|
||||
def test_replace_chat_history_keeps_unsaved_survivors(store):
|
||||
a = _agent("sess_unsaved", store)
|
||||
a._append_to_chat_history({"role": "user", "content": "q0"})
|
||||
a._auto_save_session()
|
||||
pending = {"role": "user", "content": "q1"}
|
||||
a._append_to_chat_history(pending)
|
||||
# Compaction mid-turn keeps the not-yet-saved message in its tail.
|
||||
a._replace_chat_history([{"role": "system", "content": "summary"}, pending])
|
||||
a._auto_save_session()
|
||||
saved = [m["content"] for m in store.get_chat_history("sess_unsaved")]
|
||||
assert saved == ["q0", "q1"]
|
||||
|
||||
|
||||
def test_replace_chat_history_saves_appended_messages(store):
|
||||
# Durable resume appends replayed tool pairs to a fully-saved history.
|
||||
a = _agent("sess_replay", store)
|
||||
a._append_to_chat_history({"role": "user", "content": "q0"})
|
||||
a._auto_save_session()
|
||||
replay = _tool_history()[1:3]
|
||||
a._replace_chat_history(list(a.chat_history) + replay)
|
||||
a._auto_save_session()
|
||||
saved = store.get_chat_history("sess_replay")
|
||||
assert [m["role"] for m in saved] == ["user", "assistant", "tool"]
|
||||
assert saved[2]["tool_call_id"] == "call_1"
|
||||
|
||||
|
||||
def test_prune_history_does_not_orphan_tool_result():
|
||||
a = _agent()
|
||||
a.chat_history.extend(_tool_history())
|
||||
a.prune_history(keep_last=2)
|
||||
assert _roles(a) == ["assistant"]
|
||||
|
||||
|
||||
def test_delete_history_removes_whole_exchange():
|
||||
for idx in (1, 2):
|
||||
a = _agent()
|
||||
a.chat_history.extend(_tool_history())
|
||||
assert a.delete_history(idx) is True
|
||||
assert _roles(a) == ["user", "assistant"]
|
||||
a = _agent()
|
||||
a.chat_history.extend(_tool_history())
|
||||
assert a.delete_history(10) is False
|
||||
assert a.delete_history(-1) is True
|
||||
assert len(a.chat_history) == 3
|
||||
|
||||
|
||||
def test_truncate_does_not_leave_dangling_tool_calls():
|
||||
a = _agent()
|
||||
a.chat_history.extend(_tool_history())
|
||||
a._truncate_chat_history(2)
|
||||
assert _roles(a) == ["user"]
|
||||
a = _agent()
|
||||
a.chat_history.extend(_tool_history())
|
||||
a._truncate_chat_history(3)
|
||||
assert len(a.chat_history) == 3
|
||||
|
||||
|
||||
def test_delete_history_matching_tolerates_none_content():
|
||||
a = _agent()
|
||||
a.chat_history.extend(_tool_history())
|
||||
assert a.delete_history_matching("image") == 0
|
||||
# A hit on the tool result drops its whole exchange.
|
||||
assert a.delete_history_matching("18c sunny") == 2
|
||||
assert _roles(a) == ["user", "assistant"]
|
||||
@@ -0,0 +1,113 @@
|
||||
"""The custom-LLM (``provider/model``) path sends each user turn once.
|
||||
|
||||
``chat()`` stores the user turn in ``chat_history`` before the model call, and
|
||||
``LLM._build_messages`` appends ``prompt`` after ``chat_history`` itself, so
|
||||
the stored copy must not be forwarded as history too. It was, on every
|
||||
litellm route (``ollama/``, ``anthropic/``, ``openai/``, ``base_url=``), which
|
||||
billed the prompt twice. ``litellm.completion`` is stubbed; nothing leaves the
|
||||
machine.
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
pytest.importorskip("litellm")
|
||||
|
||||
from praisonaiagents import Agent # noqa: E402
|
||||
|
||||
MODEL = "anthropic/claude-3-5-sonnet-20240620"
|
||||
|
||||
|
||||
def _litellm():
|
||||
import litellm
|
||||
return litellm
|
||||
|
||||
|
||||
def _reply(model, text):
|
||||
return _litellm().ModelResponse(model=model, usage={
|
||||
"prompt_tokens": 10, "completion_tokens": 1, "total_tokens": 11},
|
||||
choices=[{"index": 0, "finish_reason": "stop",
|
||||
"message": {"role": "assistant", "content": text}}])
|
||||
|
||||
|
||||
def _chunks(model, text):
|
||||
litellm = _litellm()
|
||||
yield litellm.ModelResponseStream(model=model, choices=[{
|
||||
"index": 0, "delta": {"role": "assistant", "content": text}}])
|
||||
yield litellm.ModelResponseStream(model=model, choices=[{
|
||||
"index": 0, "delta": {}, "finish_reason": "stop"}])
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sent(monkeypatch):
|
||||
"""Record the ``messages`` of every model call; answer "A1", "A2", ..."""
|
||||
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test")
|
||||
monkeypatch.setattr(Agent, "_default_model", None)
|
||||
monkeypatch.setattr(Agent, "_default_model_checked", False)
|
||||
requests = []
|
||||
|
||||
def fake(*a, **k):
|
||||
requests.append(k.get("messages"))
|
||||
text = f"A{len(requests)}"
|
||||
if k.get("stream"):
|
||||
return _chunks(k.get("model"), text)
|
||||
return _reply(k.get("model"), text)
|
||||
|
||||
async def afake(*a, **k):
|
||||
return fake(*a, **k)
|
||||
|
||||
litellm = _litellm()
|
||||
monkeypatch.setattr(litellm, "completion", fake)
|
||||
monkeypatch.setattr(litellm, "acompletion", afake)
|
||||
return requests
|
||||
|
||||
|
||||
def _turns(messages):
|
||||
return [(m["role"], m["content"]) for m in messages if m["role"] != "system"]
|
||||
|
||||
|
||||
def _agent():
|
||||
return Agent(name="p", instructions="Be brief.", llm=MODEL, output="silent")
|
||||
|
||||
|
||||
def test_chat_sends_the_prompt_once(sent):
|
||||
agent = _agent()
|
||||
agent.chat("Q1")
|
||||
agent.chat("Q2")
|
||||
assert _turns(sent[0]) == [("user", "Q1")]
|
||||
assert _turns(sent[1]) == [("user", "Q1"), ("assistant", "A1"), ("user", "Q2")]
|
||||
|
||||
|
||||
def test_achat_sends_the_prompt_once(sent):
|
||||
agent = _agent()
|
||||
|
||||
async def run():
|
||||
await agent.achat("Q1")
|
||||
await agent.achat("Q2")
|
||||
|
||||
asyncio.run(run())
|
||||
assert _turns(sent[0]) == [("user", "Q1")]
|
||||
assert _turns(sent[1]) == [("user", "Q1"), ("assistant", "A1"), ("user", "Q2")]
|
||||
|
||||
|
||||
def test_streamed_turn_sends_the_prompt_once(sent):
|
||||
agent = _agent()
|
||||
assert "".join(agent.start("Q1", stream=True)) == "A1"
|
||||
assert _turns(sent[0]) == [("user", "Q1")]
|
||||
|
||||
|
||||
def test_history_still_records_each_turn_once(sent):
|
||||
agent = _agent()
|
||||
agent.chat("Q1")
|
||||
agent.chat("Q2")
|
||||
assert [(m["role"], m["content"]) for m in agent.chat_history] == [
|
||||
("user", "Q1"), ("assistant", "A1"), ("user", "Q2"), ("assistant", "A2")]
|
||||
|
||||
|
||||
def test_repeating_the_last_question_is_still_one_turn(sent):
|
||||
# A user who asks the same thing twice in a row sends it once per request,
|
||||
# with the earlier exchange kept as history.
|
||||
agent = _agent()
|
||||
agent.chat("Q1")
|
||||
agent.chat("Q1")
|
||||
assert _turns(sent[1]) == [("user", "Q1"), ("assistant", "A1"), ("user", "Q1")]
|
||||
@@ -38,7 +38,7 @@ def test_cache_hit_refreshes_current_default_tools(agent, monkeypatch):
|
||||
builds = []
|
||||
monkeypatch.setattr(
|
||||
agent, '_resolve_harness_base_prompt',
|
||||
lambda: builds.append('build') or 'stable prefix',
|
||||
lambda tools=None: builds.append('build') or 'stable prefix',
|
||||
)
|
||||
first = agent._build_system_prompt(tools=None)
|
||||
assert 'sample_tool' in first
|
||||
@@ -73,7 +73,7 @@ def test_cache_hit_refreshes_session_context(agent, monkeypatch):
|
||||
def test_cache_hit_retains_prefix_reuse_and_appends_suffix_once(agent, monkeypatch):
|
||||
"""Reuse the stable prefix while adding the suffix exactly once."""
|
||||
calls = []
|
||||
agent._resolve_harness_base_prompt = lambda: calls.append('build') or 'stable prefix'
|
||||
agent._resolve_harness_base_prompt = lambda tools=None: calls.append('build') or 'stable prefix'
|
||||
monkeypatch.setenv('PRAISONAI_APPEND_SYSTEM_PROMPT', 'per-turn suffix')
|
||||
first = agent._build_system_prompt()
|
||||
second = agent._build_system_prompt()
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
"""The system prompt carries only guidance that applies to the agent.
|
||||
|
||||
* The model-family file-editing hint ("use edit_file ..." / "use
|
||||
apply_patch ...") is for agents that can edit files; every GPT or Claude
|
||||
agent got it, tools or not.
|
||||
* ``Agent(instructions=...)`` sets both backstory and goal to the
|
||||
instructions, so the prompt repeated them under "Your Goal:".
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from praisonaiagents import Agent
|
||||
from praisonaiagents.tools.edit_tools import apply_patch, edit_file
|
||||
|
||||
OPENAI_HINT = "use edit_file to modify existing files precisely"
|
||||
ANTHROPIC_HINT = "use apply_patch"
|
||||
|
||||
|
||||
def lookup(query: str) -> str:
|
||||
"""Look something up."""
|
||||
return query
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def make(monkeypatch):
|
||||
monkeypatch.setenv("OPENAI_API_KEY", "sk-test-not-used")
|
||||
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-not-used")
|
||||
made = []
|
||||
|
||||
def _make(**kw):
|
||||
kw.setdefault("instructions", "Be brief.")
|
||||
agent = Agent(context=False, **kw)
|
||||
made.append(agent)
|
||||
return agent
|
||||
|
||||
yield _make
|
||||
for agent in made:
|
||||
agent.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("tools", [None, [lookup]])
|
||||
def test_agent_without_edit_tools_gets_no_edit_hint(make, tools):
|
||||
agent = make(llm="gpt-4o-mini", tools=tools)
|
||||
assert OPENAI_HINT not in agent._build_system_prompt()
|
||||
|
||||
|
||||
def test_agent_with_edit_file_gets_the_openai_hint(make):
|
||||
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
|
||||
assert OPENAI_HINT in agent._build_system_prompt()
|
||||
|
||||
|
||||
def test_agent_with_apply_patch_gets_the_anthropic_hint(make):
|
||||
agent = make(llm="anthropic/claude-sonnet-4-5", tools=[apply_patch])
|
||||
assert ANTHROPIC_HINT in agent._build_system_prompt()
|
||||
|
||||
|
||||
def test_edit_tool_named_by_string_counts(make):
|
||||
agent = make(llm="gpt-4o-mini")
|
||||
assert OPENAI_HINT in agent._build_system_prompt(tools=["edit_file"])
|
||||
|
||||
|
||||
def test_per_call_tools_decide_the_hint(make):
|
||||
agent = make(llm="gpt-4o-mini", tools=[edit_file])
|
||||
assert OPENAI_HINT not in agent._build_system_prompt(tools=[lookup])
|
||||
assert OPENAI_HINT in agent._build_system_prompt(tools=[edit_file])
|
||||
|
||||
|
||||
def test_cached_prompt_follows_a_change_to_the_agents_tools(make):
|
||||
agent = make(llm="gpt-4o-mini", tools=[lookup])
|
||||
assert OPENAI_HINT not in agent._build_system_prompt()
|
||||
agent.tools = [edit_file]
|
||||
assert OPENAI_HINT in agent._build_system_prompt()
|
||||
agent.tools = [lookup]
|
||||
assert OPENAI_HINT not in agent._build_system_prompt()
|
||||
|
||||
|
||||
def test_explicit_harness_override_is_always_used(make):
|
||||
agent = make(llm="gpt-4o-mini")
|
||||
agent.harness_base_prompt = "House rules."
|
||||
assert agent._build_system_prompt().startswith("House rules.")
|
||||
|
||||
|
||||
def test_instructions_appear_once(make):
|
||||
agent = make(llm="gpt-4o-mini", instructions="Answer in French.")
|
||||
prompt = agent._build_system_prompt()
|
||||
assert prompt.count("Answer in French.") == 1
|
||||
assert "Your Goal:" not in prompt
|
||||
|
||||
|
||||
def test_a_distinct_goal_is_still_stated(make):
|
||||
agent = make(llm="gpt-4o-mini", instructions=None, role="Editor",
|
||||
goal="Tighten the copy", backstory="A veteran sub-editor")
|
||||
prompt = agent._build_system_prompt()
|
||||
assert "Your Goal: Tighten the copy" in prompt
|
||||
assert "A veteran sub-editor" in prompt
|
||||
assert "Your Role: Editor" in prompt
|
||||
|
||||
|
||||
def test_a_tool_renamed_to_edit_file_counts(make):
|
||||
from praisonaiagents import tool
|
||||
|
||||
@tool(name="edit_file")
|
||||
def my_editor(filepath: str, old_string: str, new_string: str) -> str:
|
||||
"""Edit a file."""
|
||||
return "ok"
|
||||
|
||||
agent = make(llm="gpt-4o-mini", tools=[my_editor])
|
||||
assert OPENAI_HINT in agent._build_system_prompt()
|
||||
|
||||
|
||||
def test_nested_tool_list_offers_the_edit_hint(make):
|
||||
# _format_tools_for_completion flattens nested lists of tool dicts.
|
||||
agent = make(llm="gpt-4o-mini", tools=[[EDIT_SCHEMA]])
|
||||
assert OPENAI_HINT in agent._build_system_prompt()
|
||||
|
||||
|
||||
def test_nested_names_the_formatter_drops_get_no_hint(make):
|
||||
# A nested string is not sent to the model, so neither is the hint.
|
||||
agent = make(llm="gpt-4o-mini", tools=[["edit_file"]])
|
||||
assert OPENAI_HINT not in agent._build_system_prompt()
|
||||
|
||||
|
||||
def test_nested_bare_name_dict_gets_no_hint(make):
|
||||
# A nested dict without a function schema is dropped by the formatter.
|
||||
agent = make(llm="gpt-4o-mini", tools=[[{"name": "edit_file"}]])
|
||||
assert agent._format_tools_for_completion() == []
|
||||
assert OPENAI_HINT not in agent._build_system_prompt()
|
||||
|
||||
|
||||
EDIT_SCHEMA = {"type": "function", "function": {
|
||||
"name": "edit_file", "description": "Edit a file.",
|
||||
"parameters": {"type": "object", "properties": {}}}}
|
||||
|
||||
|
||||
def _system_sent(agent, monkeypatch, run):
|
||||
"""Run ``run(agent)`` on the native path; return the system prompt and tool names sent."""
|
||||
from types import SimpleNamespace
|
||||
sent = []
|
||||
sent_tools = []
|
||||
|
||||
def fake(messages=None, temperature=None, tools=None, *a, **k):
|
||||
sent.append(messages if messages is not None else k.get("messages"))
|
||||
# Record the payload the real request builds (inherit, flatten, prune).
|
||||
formatted = agent._format_tools_for_completion(tools)
|
||||
sent_tools.append([t.get("function", {}).get("name") for t in formatted])
|
||||
message = SimpleNamespace(content="ok", tool_calls=None, role="assistant", reasoning_content=None)
|
||||
return SimpleNamespace(choices=[SimpleNamespace(message=message, finish_reason="stop", index=0)],
|
||||
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=1, total_tokens=2),
|
||||
model="gpt-4o-mini")
|
||||
|
||||
async def afake(*a, **k):
|
||||
return fake(*a, **k)
|
||||
|
||||
monkeypatch.setattr(agent, "_chat_completion", fake)
|
||||
monkeypatch.setattr(agent, "_achat_completion_with_retry", afake)
|
||||
run(agent)
|
||||
return next(m["content"] for m in sent[-1] if m["role"] == "system"), sent_tools[-1]
|
||||
|
||||
|
||||
def test_native_chat_follows_per_call_edit_tools(make, monkeypatch):
|
||||
agent = make(llm="gpt-4o-mini")
|
||||
system, tool_names = _system_sent(agent, monkeypatch, lambda a: a.chat("hi", tools=[EDIT_SCHEMA]))
|
||||
assert OPENAI_HINT in system
|
||||
assert "edit_file" in tool_names
|
||||
|
||||
|
||||
def test_nested_edit_schema_is_sent_with_the_hint(make, monkeypatch):
|
||||
agent = make(llm="gpt-4o-mini", tools=[[EDIT_SCHEMA]])
|
||||
system, tool_names = _system_sent(agent, monkeypatch, lambda a: a.chat("hi"))
|
||||
assert OPENAI_HINT in system
|
||||
assert "edit_file" in tool_names
|
||||
|
||||
|
||||
def test_permission_denied_edit_tool_gets_no_hint(make, monkeypatch):
|
||||
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
|
||||
agent._perm_deny = frozenset({"edit_file"})
|
||||
system, tool_names = _system_sent(agent, monkeypatch, lambda a: a.chat("hi"))
|
||||
assert "edit_file" not in tool_names
|
||||
assert OPENAI_HINT not in system
|
||||
|
||||
|
||||
def test_deny_added_after_caching_prunes_cached_tools(make):
|
||||
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
|
||||
assert "edit_file" in [t["function"]["name"] for t in agent._format_tools_for_completion()]
|
||||
agent._perm_deny = frozenset({"edit_file"})
|
||||
assert "edit_file" not in [t["function"]["name"] for t in agent._format_tools_for_completion()]
|
||||
assert OPENAI_HINT not in agent._build_system_prompt()
|
||||
# Lifting the denial restores the tool from the same cache entry.
|
||||
agent._perm_deny = frozenset()
|
||||
assert "edit_file" in [t["function"]["name"] for t in agent._format_tools_for_completion()]
|
||||
|
||||
|
||||
def test_deny_before_first_format_restores_after_lift(make):
|
||||
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
|
||||
agent._perm_deny = frozenset({"edit_file"})
|
||||
assert "edit_file" not in [t["function"]["name"] for t in agent._format_tools_for_completion()]
|
||||
agent._perm_deny = frozenset()
|
||||
assert "edit_file" in [t["function"]["name"] for t in agent._format_tools_for_completion()]
|
||||
|
||||
|
||||
def test_bridge_tool_search_hides_denied_tools(make):
|
||||
agent = make(llm="gpt-4o-mini")
|
||||
agent._tool_search_config = None
|
||||
agent._tool_search_metadata = {"bridge_mode": True, "deferrable_tools": [EDIT_SCHEMA]}
|
||||
agent._perm_deny = frozenset({"edit_file"})
|
||||
out = agent._handle_bridge_tool_call("tool_describe", {"tool_name": "edit_file"})
|
||||
assert "not found" in out
|
||||
agent._perm_deny = frozenset()
|
||||
assert "not found" not in agent._handle_bridge_tool_call("tool_describe", {"tool_name": "edit_file"})
|
||||
|
||||
|
||||
def test_native_achat_follows_per_call_edit_tools(make, monkeypatch):
|
||||
import asyncio
|
||||
agent = make(llm="gpt-4o-mini")
|
||||
system, tool_names = _system_sent(agent, monkeypatch, lambda a: asyncio.run(a.achat("hi", tools=[EDIT_SCHEMA])))
|
||||
assert OPENAI_HINT in system
|
||||
assert "edit_file" in tool_names
|
||||
|
||||
|
||||
def test_system_prompt_attribute_states_instructions_once(make):
|
||||
agent = make(llm="gpt-4o-mini", instructions="Answer in French.")
|
||||
assert agent.system_prompt.count("Answer in French.") == 1
|
||||
|
||||
|
||||
def test_cached_prompt_follows_a_backstory_change(make):
|
||||
agent = make(llm="gpt-4o-mini", instructions="Answer in French.")
|
||||
assert "Your Goal:" not in agent._build_system_prompt()
|
||||
agent.backstory = "A translator"
|
||||
prompt = agent._build_system_prompt()
|
||||
assert "A translator" in prompt
|
||||
assert "Your Goal: Answer in French." in prompt
|
||||
+5
-1
@@ -32,6 +32,7 @@ def _agent_with_history(messages: int, words: int):
|
||||
|
||||
def fake_stream(**kwargs):
|
||||
sent["history"] = list(kwargs.get("chat_history") or [])
|
||||
sent["prompt"] = kwargs.get("prompt")
|
||||
yield "ok"
|
||||
|
||||
agent._using_custom_llm = True
|
||||
@@ -60,7 +61,10 @@ class TestCustomLlmStreamCompaction:
|
||||
"""Compaction must not fire when it is not needed."""
|
||||
agent, sent = _agent_with_history(6, 5)
|
||||
history = _drain(agent, sent)
|
||||
assert len(history) == 7, "a short history was compacted unnecessarily"
|
||||
# The six earlier turns go as history; this turn goes once, as the prompt.
|
||||
assert len(history) == 6, "a short history was compacted unnecessarily"
|
||||
assert sent["prompt"] == "hello"
|
||||
assert all(m["content"] != "hello" for m in history)
|
||||
|
||||
def test_the_context_manager_is_actually_configured_here(self):
|
||||
"""Guards the premise: tools present auto-enable context management."""
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Automatic promotion must honor the same capacity as explicit long-term writes."""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from praisonaiagents.memory.file_memory import FileMemory
|
||||
|
||||
|
||||
@pytest.mark.parametrize("limit", [1, 2, 3])
|
||||
def test_promotion_enforces_persisted_long_term_limit(tmp_path, limit):
|
||||
memory = FileMemory(
|
||||
base_path=str(tmp_path),
|
||||
config={"short_term_limit": 2, "long_term_limit": limit},
|
||||
)
|
||||
for index in range(6):
|
||||
memory.add_short_term(f"Important fact {index}", importance=0.8)
|
||||
assert len(memory.get_long_term()) == limit
|
||||
assert len(FileMemory(base_path=str(tmp_path)).get_long_term()) == limit
|
||||
assert len(memory.get_short_term()) == 2
|
||||
|
||||
|
||||
def test_promotion_keeps_highest_importance_and_newest_ties(tmp_path):
|
||||
memory = FileMemory(
|
||||
base_path=str(tmp_path),
|
||||
config={"short_term_limit": 3, "long_term_limit": 2},
|
||||
)
|
||||
memory.add_long_term("Keep strongest", importance=1.0)
|
||||
memory.add_long_term("Older tie", importance=0.8)
|
||||
memory.add_short_term("Lower priority", importance=0.7)
|
||||
memory.add_short_term("Newer tie", importance=0.8, metadata={"source": "fixture"})
|
||||
memory.add_short_term("Below promotion threshold", importance=0.2)
|
||||
for path, timestamp in [(memory.long_term_file, 100), (memory.short_term_file, 200)]:
|
||||
records = json.loads(path.read_text())
|
||||
for record in records:
|
||||
record["created_at"] = timestamp
|
||||
path.write_text(json.dumps(records))
|
||||
promoted_before = next(
|
||||
item for item in json.loads(memory.short_term_file.read_text())
|
||||
if item["content"] == "Newer tie"
|
||||
)
|
||||
memory.add_short_term("Trigger overflow", importance=0.2)
|
||||
contents = {item.content for item in memory.get_long_term()}
|
||||
assert contents == {"Keep strongest", "Newer tie"}
|
||||
promoted_after = next(item for item in memory.get_long_term() if item.content == "Newer tie")
|
||||
assert promoted_after.to_dict() == promoted_before
|
||||
assert {item.content for item in memory.get_short_term()} == {
|
||||
"Newer tie", "Below promotion threshold", "Trigger overflow",
|
||||
}
|
||||
assert {item.content for item in FileMemory(base_path=str(tmp_path)).get_long_term()} == contents
|
||||
|
||||
|
||||
def test_disabled_promotion_preserves_existing_long_term(tmp_path):
|
||||
memory = FileMemory(
|
||||
base_path=str(tmp_path),
|
||||
config={"short_term_limit": 1, "long_term_limit": 1, "auto_promote": False},
|
||||
)
|
||||
memory.add_long_term("Existing fact")
|
||||
for index in range(3):
|
||||
memory.add_short_term(f"Unpromoted {index}", importance=1.0)
|
||||
assert [item.content for item in memory.get_long_term()] == ["Existing fact"]
|
||||
|
||||
|
||||
def test_promotion_zero_capacity_matches_explicit_long_term(tmp_path):
|
||||
memory = FileMemory(
|
||||
base_path=str(tmp_path),
|
||||
config={"short_term_limit": 1, "long_term_limit": 0},
|
||||
)
|
||||
memory.add_long_term("Explicit fact", importance=1.0)
|
||||
assert memory.get_long_term() == []
|
||||
memory.add_short_term("Promoted one", importance=1.0)
|
||||
memory.add_short_term("Promoted two", importance=1.0)
|
||||
assert memory.get_long_term() == []
|
||||
assert FileMemory(base_path=str(tmp_path)).get_long_term() == []
|
||||
@@ -34,11 +34,11 @@ def _restore_approval_state():
|
||||
from praisonaiagents.approval import get_approval_registry, clear_approval_context
|
||||
|
||||
registry = get_approval_registry()
|
||||
previous = registry.get_backend()
|
||||
previous = registry._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
registry.set_backend(previous)
|
||||
registry._global_backend = previous
|
||||
clear_approval_context()
|
||||
|
||||
|
||||
@@ -154,7 +154,7 @@ def test_code_mode_changes_what_the_tool_can_do():
|
||||
|
||||
_probe_tool_name = "code_mode_probe"
|
||||
get_registry().register(code_mode_probe)
|
||||
previous = get_approval_registry().get_backend()
|
||||
previous = get_approval_registry()._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
|
||||
get_approval_registry().set_backend(AutoApproveBackend())
|
||||
try:
|
||||
safe = Agent(
|
||||
@@ -182,7 +182,7 @@ def test_code_mode_changes_what_the_tool_can_do():
|
||||
safe_result = safe_exec(code)
|
||||
unsafe_result = unsafe_exec(code)
|
||||
finally:
|
||||
get_approval_registry().set_backend(previous)
|
||||
get_approval_registry()._global_backend = previous
|
||||
clear_approval_context()
|
||||
# The tool registry is process-global. Leaving these probe tools in it
|
||||
# changed what later tests saw when they enumerated or dispatched
|
||||
@@ -221,7 +221,7 @@ def test_unsafe_code_mode_cannot_reach_tools_the_agent_was_not_granted():
|
||||
|
||||
_probe_tool_name = "ungranted_global_tool"
|
||||
get_registry().register(ungranted_global_tool)
|
||||
previous = get_approval_registry().get_backend()
|
||||
previous = get_approval_registry()._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
|
||||
get_approval_registry().set_backend(AutoApproveBackend())
|
||||
try:
|
||||
agent = Agent(
|
||||
@@ -240,7 +240,7 @@ def test_unsafe_code_mode_cannot_reach_tools_the_agent_was_not_granted():
|
||||
)
|
||||
result = execute("print(ungranted_global_tool())")
|
||||
finally:
|
||||
get_approval_registry().set_backend(previous)
|
||||
get_approval_registry()._global_backend = previous
|
||||
clear_approval_context()
|
||||
# The tool registry is process-global. Leaving these probe tools in it
|
||||
# changed what later tests saw when they enumerated or dispatched
|
||||
@@ -279,13 +279,13 @@ def test_code_execution_runs_a_trailing_expression_exactly_once():
|
||||
from praisonaiagents.approval import clear_approval_context
|
||||
|
||||
registry = get_approval_registry()
|
||||
previous = registry.get_backend()
|
||||
previous = registry._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
|
||||
registry.set_backend(AutoApproveBackend())
|
||||
try:
|
||||
assert execute_code("print(7)")["stdout"] == "7\n"
|
||||
assert execute_code_with_tools("print(7)")["stdout"] == "7\n"
|
||||
finally:
|
||||
registry.set_backend(previous)
|
||||
registry._global_backend = previous
|
||||
clear_approval_context()
|
||||
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ def test_no_handrolled_async_bridges_outside_helper():
|
||||
for path in root.rglob("*.py"):
|
||||
if path in allowed:
|
||||
continue
|
||||
tree = ast.parse(path.read_text(), filename=str(path))
|
||||
tree = ast.parse(path.read_bytes(), filename=str(path))
|
||||
for func in ast.walk(tree):
|
||||
if not isinstance(func, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
continue
|
||||
@@ -93,6 +93,15 @@ def test_no_handrolled_async_bridges_outside_helper():
|
||||
)
|
||||
|
||||
|
||||
def test_scan_accepts_python_source_encoding_cookie(tmp_path, monkeypatch):
|
||||
"""Scan the same Latin-1 source that Python's parser accepts."""
|
||||
(tmp_path / "example.py").write_bytes(
|
||||
b"# coding: latin-1; caf\xe9\nmessage = 'caf\xe9'\n"
|
||||
)
|
||||
monkeypatch.setitem(globals(), "_package_root", lambda: tmp_path)
|
||||
test_no_handrolled_async_bridges_outside_helper()
|
||||
|
||||
|
||||
def test_tool_timeout_uses_daemon_worker_and_does_not_leak():
|
||||
before = {t.ident for t in threading.enumerate()}
|
||||
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Counting rules of benchmarks/execution_benchmark.py, with fake runners.
|
||||
|
||||
No framework is imported and nothing leaves the machine: each runner is a
|
||||
``build()`` that returns the call to time, as the script's real runners do.
|
||||
"""
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parents[2] / "benchmarks" / "execution_benchmark.py"
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def bench():
|
||||
spec = importlib.util.spec_from_file_location("execution_benchmark", SCRIPT)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def runner(answers, log, name):
|
||||
"""A build() whose calls return ``answers`` in order (an Exception is raised)."""
|
||||
answers = iter(answers)
|
||||
|
||||
def build():
|
||||
log.append(("build", name))
|
||||
|
||||
def call():
|
||||
log.append(("call", name))
|
||||
answer = next(answers)
|
||||
if isinstance(answer, Exception):
|
||||
raise answer
|
||||
return answer
|
||||
return call
|
||||
return build
|
||||
|
||||
|
||||
def test_warm_up_calls_are_not_timed_and_rounds_interleave(bench):
|
||||
log = []
|
||||
runners = {"A": runner(["w", "a1", "a2"], log, "A"), "B": runner(["w", "b1", "b2"], log, "B")}
|
||||
results = bench.run_benchmark(runners, iterations=2, warmup=1)
|
||||
assert [len(r["times"]) for r in results.values()] == [2, 2]
|
||||
calls = [name for kind, name in log if kind == "call"]
|
||||
assert calls == ["A", "B", "A", "B", "B", "A"] # warm-ups, round 1, round 2 rotated
|
||||
|
||||
|
||||
def test_errors_and_empty_answers_are_failures_not_timings(bench):
|
||||
log = []
|
||||
runners = {"A": runner(["w", RuntimeError("401"), None, "None", " ", "4"], log, "A")}
|
||||
results = bench.run_benchmark(runners, iterations=5, warmup=1)
|
||||
assert len(results["A"]["times"]) == 1
|
||||
assert len(results["A"]["failures"]) == 4
|
||||
assert "RuntimeError: 401" in results["A"]["failures"][0]
|
||||
|
||||
|
||||
def test_a_failed_warm_up_does_not_drop_the_framework(bench):
|
||||
log = []
|
||||
runners = {"A": runner([RuntimeError("timeout"), "a1", "a2"], log, "A")}
|
||||
results = bench.run_benchmark(runners, iterations=2, warmup=1)
|
||||
assert len(results["A"]["times"]) == 2
|
||||
|
||||
|
||||
def test_summary_ranks_by_median_and_skips_frameworks_without_a_success(bench):
|
||||
results = {
|
||||
"slow": {"times": [1.0, 1.1, 9.0], "failures": []},
|
||||
"fast": {"times": [0.5, 0.6, 0.7], "failures": ["x"]},
|
||||
"broken": {"times": [], "failures": ["401"]},
|
||||
}
|
||||
rows = bench.summarise(results)
|
||||
assert [r[0] for r in rows] == ["fast", "slow"]
|
||||
name, median, mean, ok, failed = rows[1]
|
||||
assert (median, ok, failed) == (1.1, 3, 0)
|
||||
assert mean == pytest.approx(11.1 / 3)
|
||||
|
||||
|
||||
def test_only_the_call_is_timed(bench, monkeypatch):
|
||||
clock = iter([10.0, 10.5]) # start / end of the one timed call; build() reads no clock
|
||||
|
||||
def build():
|
||||
return lambda: "4"
|
||||
|
||||
monkeypatch.setattr(bench.time, "perf_counter", lambda: next(clock))
|
||||
results = bench.run_benchmark({"A": build}, iterations=1, warmup=0)
|
||||
assert results["A"]["times"] == [0.5]
|
||||
@@ -328,7 +328,7 @@ class TestMultiChannelGatewayConfig:
|
||||
from praisonaiagents.gateway.config import ChannelRouteConfig, MultiChannelGatewayConfig
|
||||
module = sys.modules["praisonaiagents.gateway.config"]
|
||||
source_file = module.__file__
|
||||
with open(source_file, "r") as f:
|
||||
with open(source_file, "r", encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
for dep in ["chromadb", "fastapi", "uvicorn", "pydantic", "litellm"]:
|
||||
assert f"import {dep}" not in content, f"Found heavy dep '{dep}' in config.py"
|
||||
@@ -376,7 +376,7 @@ class TestChatCommandInfo:
|
||||
from praisonaiagents.bots.protocols import BotProtocol
|
||||
module = sys.modules["praisonaiagents.bots.protocols"]
|
||||
source_file = module.__file__
|
||||
with open(source_file, "r") as f:
|
||||
with open(source_file, "r", encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
for dep in ["chromadb", "fastapi", "uvicorn", "litellm"]:
|
||||
assert f"import {dep}" not in content, f"Found heavy dep '{dep}' in protocols.py"
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
"""Regression tests for Issue #6238.
|
||||
|
||||
* Caller-supplied names must never escape the storage directory.
|
||||
* Sync entry points must work from inside a running event loop.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from praisonaiagents.planning.storage import PlanStorage
|
||||
from praisonaiagents.planning.todo import TodoList
|
||||
from praisonaiagents.utils.atomic_io import safe_join
|
||||
|
||||
|
||||
def test_safe_join_keeps_legacy_names_and_confines_others(tmp_path):
|
||||
base = str(tmp_path)
|
||||
assert safe_join(base, "current", ".json") == os.path.join(base, "current.json")
|
||||
for name in ("../../x", "/etc/passwd", "..", "", "a/b", "a\\b", ".hidden"):
|
||||
path = safe_join(base, name, ".json")
|
||||
assert os.path.dirname(path) == base, name
|
||||
# Distinct inputs never collide on one file.
|
||||
assert safe_join(base, "a/b") != safe_join(base, "a_b")
|
||||
|
||||
|
||||
def test_plan_storage_todo_name_cannot_traverse(tmp_path):
|
||||
storage = PlanStorage(base_path=str(tmp_path / "store"))
|
||||
path = storage.save_todo(TodoList(), name="../../escaped")
|
||||
assert os.path.dirname(path) == storage.todos_dir
|
||||
assert not (tmp_path / "escaped.json").exists()
|
||||
assert storage.load_todo("../../escaped") is not None
|
||||
assert storage.delete_todo("../../escaped") is True
|
||||
|
||||
|
||||
def test_generate_title_works_inside_running_loop(monkeypatch):
|
||||
from praisonaiagents.session import title
|
||||
|
||||
async def fake(*args, **kwargs):
|
||||
return "LLM Title"
|
||||
|
||||
monkeypatch.setattr(title, "generate_title_async", fake)
|
||||
|
||||
async def main():
|
||||
return title.generate_title("hello", "world")
|
||||
|
||||
assert asyncio.run(main()) == "LLM Title"
|
||||
|
||||
|
||||
def test_listed_todo_names_round_trip(tmp_path):
|
||||
storage = PlanStorage(base_path=str(tmp_path / "store"))
|
||||
storage.save_todo(TodoList(), name="my tasks")
|
||||
(listed,) = storage.list_todos()
|
||||
assert storage.load_todo(listed) is not None
|
||||
assert storage.load_todo("my tasks") is not None
|
||||
assert storage.delete_todo(listed) is True
|
||||
assert storage.list_todos() == []
|
||||
|
||||
|
||||
def test_legacy_todo_file_still_found(tmp_path):
|
||||
storage = PlanStorage(base_path=str(tmp_path / "store"))
|
||||
legacy = os.path.join(storage.todos_dir, "my tasks.json")
|
||||
with open(legacy, "w", encoding="utf-8") as f:
|
||||
f.write(TodoList().to_json())
|
||||
assert storage.load_todo("my tasks") is not None
|
||||
assert storage.save_todo(TodoList(), name="my tasks") == legacy
|
||||
assert storage.list_todos() == ["my tasks"]
|
||||
assert storage.delete_todo("my tasks") is True
|
||||
|
||||
|
||||
def test_custom_plan_filename_keeps_md_suffix(tmp_path):
|
||||
from praisonaiagents.planning.plan import Plan
|
||||
|
||||
storage = PlanStorage(base_path=str(tmp_path / "store"))
|
||||
plan = Plan(name="demo")
|
||||
path = storage.save_plan(plan, filename="my plan.md")
|
||||
assert path.endswith(".md") and os.path.dirname(path) == storage.plans_dir
|
||||
assert len(storage.list_plans()) == 1
|
||||
|
||||
|
||||
def test_encoded_looking_names_do_not_share_a_file(tmp_path):
|
||||
storage = PlanStorage(base_path=str(tmp_path / "store"))
|
||||
names = ["a/b", "a_b~c14cddc033f6", "a%2Fb", "a b", "a%20b", "~x", ".."]
|
||||
paths = {storage.save_todo(TodoList(name=n), name=n) for n in names}
|
||||
assert len(paths) == len(names)
|
||||
assert all(os.path.dirname(p) == storage.todos_dir for p in paths)
|
||||
assert sorted(storage.list_todos()) == sorted(names)
|
||||
for n in names:
|
||||
assert storage.load_todo(n).name == n
|
||||
assert storage.delete_todo("a/b") is True
|
||||
assert storage.load_todo("a%2Fb") is not None
|
||||
|
||||
|
||||
def test_long_unicode_names_are_bounded_and_round_trip(tmp_path):
|
||||
storage = PlanStorage(base_path=str(tmp_path / "store"))
|
||||
long_a, long_b = "我的任务" * 40, "我的任务" * 40 + "x"
|
||||
pa = storage.save_todo(TodoList(name=long_a), name=long_a)
|
||||
pb = storage.save_todo(TodoList(name=long_b), name=long_b)
|
||||
assert pa != pb
|
||||
for p in (pa, pb):
|
||||
assert os.path.dirname(p) == storage.todos_dir
|
||||
assert len(os.path.basename(p).encode("utf-8")) <= 200
|
||||
# Short non-ASCII names keep their plain (pre-#6238) filename.
|
||||
assert storage.save_todo(TodoList(), name="我的任务").endswith("我的任务.json")
|
||||
for listed in storage.list_todos():
|
||||
assert storage.load_todo(listed) is not None
|
||||
assert storage.delete_todo(listed) is True
|
||||
assert storage.list_todos() == []
|
||||
assert storage.load_todo(long_a) is None
|
||||
@@ -64,6 +64,35 @@ def test_grep_glob_filters_by_pattern(sample_tree):
|
||||
assert "src" not in out
|
||||
|
||||
|
||||
@pytest.mark.parametrize("pattern", ["**/*.py", "src/**/*.py", "src/**/**/test_*.py"])
|
||||
def test_grep_recursive_glob_matches_zero_directories(sample_tree, pattern):
|
||||
from praisonaiagents.tools import grep
|
||||
|
||||
direct = sample_tree / "src" / "test_direct.py"
|
||||
direct.write_text("needle\n")
|
||||
nested = sample_tree / "src" / "nested"
|
||||
nested.mkdir()
|
||||
(nested / "test_nested.py").write_text("needle\n")
|
||||
with patch("shutil.which", return_value=None):
|
||||
out = grep("needle", glob=pattern)
|
||||
assert "src/test_direct.py:1:" in out.replace(os.sep, "/")
|
||||
assert "src/nested/test_nested.py:1:" in out.replace(os.sep, "/")
|
||||
if pattern == "**/*.py":
|
||||
assert "a.py:2:" in out
|
||||
else:
|
||||
assert "a.py:2:" not in out
|
||||
if "test_" in pattern:
|
||||
assert "src/d.py:1:" not in out.replace(os.sep, "/")
|
||||
|
||||
|
||||
def test_grep_recursive_glob_single_file(sample_tree):
|
||||
from praisonaiagents.tools import grep
|
||||
|
||||
with patch("shutil.which", return_value=None):
|
||||
out = grep("needle", path="a.py", glob="**/*.py")
|
||||
assert "a.py:2:" in out
|
||||
|
||||
|
||||
def test_grep_case_insensitive(sample_tree):
|
||||
from praisonaiagents.tools import grep
|
||||
with patch("shutil.which", return_value=None):
|
||||
|
||||
@@ -491,7 +491,7 @@ class TestTeamTasks:
|
||||
return ApprovalDecision(approved=True, reason="looks good", approver="ana")
|
||||
|
||||
registry = get_approval_registry()
|
||||
previous = registry.get_backend()
|
||||
previous = registry._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
|
||||
registry.set_backend(Reviewer())
|
||||
try:
|
||||
writer = Agent(name="Writer", instructions="write")
|
||||
@@ -500,7 +500,7 @@ class TestTeamTasks:
|
||||
out = TaskOutput(description="Draft", raw="api_key=sk-secret123456 text", agent="Writer")
|
||||
team._apply_human_review(task, 0, out)
|
||||
finally:
|
||||
registry.set_backend(previous)
|
||||
registry._global_backend = previous
|
||||
requested, decided = _of(recorder, "approval_requested"), _of(recorder, "approval_decided")
|
||||
assert len(requested) == len(decided) == 1
|
||||
assert requested[0].data["kind"] == "task_output_review"
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
"""An agent's YAML `execution:` block reaches ExecutionConfig (Issue #6206)."""
|
||||
from praisonaiagents.workflows.yaml_parser import YAMLWorkflowParser
|
||||
|
||||
|
||||
def test_execution_block_passes_budget_and_durable_keys():
|
||||
agent = YAMLWorkflowParser()._create_agent("worker", {
|
||||
"role": "Worker",
|
||||
"execution": {
|
||||
"max_budget": 1.5,
|
||||
"on_budget_exceeded": "warn",
|
||||
"durable": True,
|
||||
"resume_run_id": "run-123",
|
||||
},
|
||||
"max_iter": 7,
|
||||
})
|
||||
assert agent._max_budget == 1.5
|
||||
assert agent._on_budget_exceeded == "warn"
|
||||
assert agent.execution.durable is True
|
||||
assert agent.execution.resume_run_id == "run-123"
|
||||
assert agent.execution.max_iter == 7
|
||||
|
||||
|
||||
def test_flat_max_iter_overrides_execution_block():
|
||||
agent = YAMLWorkflowParser()._create_agent("worker", {
|
||||
"execution": {"max_iter": 3},
|
||||
"max_iter": 9,
|
||||
})
|
||||
assert agent.execution.max_iter == 9
|
||||
|
||||
|
||||
def test_no_execution_block_keeps_defaults():
|
||||
agent = YAMLWorkflowParser()._create_agent("worker", {"role": "Worker"})
|
||||
assert not agent._max_budget
|
||||
|
||||
|
||||
def test_roles_format_keeps_execution_block():
|
||||
"""The agents.yaml `roles:` format must not drop `execution:` in conversion."""
|
||||
parser = YAMLWorkflowParser()
|
||||
parser.parse_string(
|
||||
"framework: praisonai\n"
|
||||
"topic: t\n"
|
||||
"roles:\n"
|
||||
" worker:\n"
|
||||
" role: Worker\n"
|
||||
" goal: g\n"
|
||||
" backstory: b\n"
|
||||
" execution:\n"
|
||||
" max_budget: 2.0\n"
|
||||
" on_budget_exceeded: warn\n"
|
||||
" tasks:\n"
|
||||
" t1:\n"
|
||||
" description: do it\n"
|
||||
" expected_output: done\n"
|
||||
)
|
||||
agent = parser._agents["worker"]
|
||||
assert agent._max_budget == 2.0
|
||||
assert agent._on_budget_exceeded == "warn"
|
||||
@@ -28,7 +28,9 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import itertools
|
||||
import logging
|
||||
import math
|
||||
import sys
|
||||
import time
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Dict, Optional
|
||||
|
||||
@@ -106,9 +108,14 @@ class AdmissionRejected(Exception):
|
||||
busy acknowledgement to the user instead of an opaque error.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str = "Busy right now — please resend in a moment."):
|
||||
def __init__(
|
||||
self,
|
||||
message: str = "Busy right now — please resend in a moment.",
|
||||
retry_after: Optional[float] = None,
|
||||
):
|
||||
super().__init__(message)
|
||||
self.message = message
|
||||
self.retry_after = retry_after
|
||||
|
||||
|
||||
class _TurnSlot:
|
||||
@@ -249,8 +256,16 @@ class AdmissionGate:
|
||||
*,
|
||||
resource_policy: Optional[object] = None,
|
||||
resource_sampler: Optional[object] = None,
|
||||
rate_limit_policy: Optional[object] = None,
|
||||
):
|
||||
self._policy = policy
|
||||
# Issue #6215: optional per-identity request-rate ceiling (a core
|
||||
# ``RateLimitPolicyProtocol``, e.g. ``SlidingWindowRateLimitPolicy``)
|
||||
# checked before a turn is admitted, keyed by the caller's rate key
|
||||
# (defaults to the session id).
|
||||
self._rate_limit_policy = rate_limit_policy
|
||||
self.rate_limited = 0
|
||||
self._rate_prune_after = 0.0
|
||||
# Optional memory/resource-pressure policy (Issue #3445). When set, a
|
||||
# lightweight RSS sample is folded into each admit decision so the gate
|
||||
# queues under soft pressure and sheds under hard pressure before the
|
||||
@@ -303,6 +318,16 @@ class AdmissionGate:
|
||||
# otherwise assume active (it was explicitly supplied).
|
||||
return bool(getattr(pol, "enabled", True))
|
||||
|
||||
def set_rate_limit_policy(self, policy: Optional[object]) -> None:
|
||||
"""Swap the per-identity rate budget in place (config hot-reload)."""
|
||||
self._rate_limit_policy = policy
|
||||
self._rate_prune_after = 0.0
|
||||
|
||||
@property
|
||||
def _rate_limit_enabled(self) -> bool:
|
||||
pol = self._rate_limit_policy
|
||||
return pol is not None and bool(getattr(pol, "enabled", True))
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
"""Whether the gate is active.
|
||||
@@ -313,7 +338,7 @@ class AdmissionGate:
|
||||
"""
|
||||
if self._policy is not None and self._max > 0:
|
||||
return True
|
||||
return self._resource_enabled
|
||||
return self._resource_enabled or self._rate_limit_enabled
|
||||
|
||||
@property
|
||||
def in_flight(self) -> int:
|
||||
@@ -337,6 +362,10 @@ class AdmissionGate:
|
||||
"admitted": self.admitted,
|
||||
"rejected": self.rejected,
|
||||
"shed": self.shed,
|
||||
"per_user_per_minute": int(
|
||||
getattr(self._rate_limit_policy, "max_requests", 0) or 0
|
||||
),
|
||||
"rate_limited": self.rate_limited,
|
||||
"max_rss_mb": float(
|
||||
getattr(self._resource_policy, "hard_rss_mb", 0.0) or 0.0
|
||||
),
|
||||
@@ -362,9 +391,14 @@ class AdmissionGate:
|
||||
return self._sem
|
||||
|
||||
@asynccontextmanager
|
||||
async def admit(self, *, session_id: str = ""):
|
||||
async def admit(self, *, session_id: str = "", rate_key: Optional[str] = None):
|
||||
"""Admit (or shed) an inbound turn, holding a slot for its duration.
|
||||
|
||||
``rate_key`` is the identity charged against the per-user request
|
||||
budget. It defaults to ``session_id``; surfaces where the session id is
|
||||
client-chosen (API headers, WebSocket sessions) pass a stable caller
|
||||
identity instead so a new conversation cannot reset the budget.
|
||||
|
||||
Yields when capacity is available (immediately or after waiting in the
|
||||
bounded queue). Raises :class:`AdmissionRejected` when the policy sheds
|
||||
the turn. A no-op pass-through when the gate is disabled.
|
||||
@@ -374,6 +408,8 @@ class AdmissionGate:
|
||||
yield
|
||||
return
|
||||
|
||||
self._check_rate_limit(rate_key or session_id)
|
||||
|
||||
# Issue #5168: the scope key is the caller-supplied session_id (the
|
||||
# per-user/per-tenant storage key at the call sites). Only tracked when
|
||||
# a per-scope sub-limit is active, so the unscoped path is unchanged.
|
||||
@@ -473,6 +509,54 @@ class AdmissionGate:
|
||||
pass
|
||||
slot.close()
|
||||
|
||||
def _check_rate_limit(self, session_id: str) -> None:
|
||||
"""Shed the turn if *session_id* is over its request-rate budget (#6215)."""
|
||||
if not self._rate_limit_enabled:
|
||||
return
|
||||
pol = self._rate_limit_policy
|
||||
now = time.monotonic()
|
||||
try:
|
||||
decision = pol.check(identity=session_id or "-", scope="ingress", now=now)
|
||||
except Exception as e: # pragma: no cover — never block on policy error
|
||||
logger.warning("AdmissionGate: rate-limit policy failed (%s); admitting", e)
|
||||
return
|
||||
finally:
|
||||
# Bounded cadence: a burst of distinct callers must not trigger a
|
||||
# full scan on every request while their windows are still live.
|
||||
if now >= self._rate_prune_after:
|
||||
self._rate_prune_after = now + _RATE_PRUNE_INTERVAL
|
||||
self._prune_rate_limit_state(pol, now)
|
||||
if decision.allowed:
|
||||
return
|
||||
self.rate_limited += 1
|
||||
retry = decision.retry_after_seconds
|
||||
wait = max(1, math.ceil(retry)) if retry is not None else None
|
||||
logger.info("AdmissionGate: rate-limited session=%s retry_after=%s", session_id or "-", wait)
|
||||
if wait is None:
|
||||
raise AdmissionRejected("Rate limit reached — please try again later.")
|
||||
raise AdmissionRejected(
|
||||
f"Rate limit reached — please try again in {wait}s.", retry_after=float(wait)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _prune_rate_limit_state(pol: object, now: float, cap: int = 4096) -> None:
|
||||
"""Drop expired per-identity entries so an open bot cannot grow them forever.
|
||||
|
||||
``SlidingWindowRateLimitPolicy`` reclaims a key only when that key is
|
||||
checked again and leaves reclamation of an unbounded identity space to
|
||||
the wrapper; inbound users are such a space.
|
||||
"""
|
||||
windows = getattr(pol, "_windows", None)
|
||||
lockouts = getattr(pol, "_lockouts", None)
|
||||
if not isinstance(windows, dict) or len(windows) + len(lockouts or {}) <= cap:
|
||||
return
|
||||
span = float(getattr(pol, "window_seconds", 0) or 0)
|
||||
for key in [k for k, (start, _) in windows.items() if now - start >= span]:
|
||||
windows.pop(key, None)
|
||||
if isinstance(lockouts, dict):
|
||||
for key in [k for k, until in lockouts.items() if now >= until]:
|
||||
lockouts.pop(key, None)
|
||||
|
||||
def _take_slot(self, scope: str) -> None:
|
||||
self._in_flight += 1
|
||||
if scope:
|
||||
@@ -596,6 +680,46 @@ def build_memory_pressure_policy(
|
||||
return MemoryPressurePolicy(soft_rss_mb=soft, hard_rss_mb=hard)
|
||||
|
||||
|
||||
_RATE_PRUNE_INTERVAL = 5.0
|
||||
|
||||
|
||||
def build_rate_limit_policy(per_user_per_minute: object) -> Optional[object]:
|
||||
"""Return a per-user request budget policy, or ``None`` when unlimited.
|
||||
|
||||
Accepts a non-negative whole number (``0``/``None`` = unlimited). Fractional,
|
||||
boolean, negative or non-numeric values raise ``ValueError`` so a typo never
|
||||
silently disables the budget the operator asked for.
|
||||
"""
|
||||
value = per_user_per_minute
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float, str)):
|
||||
raise ValueError(
|
||||
f"per_user_per_minute must be a non-negative integer, got {value!r}"
|
||||
)
|
||||
try:
|
||||
if isinstance(value, float):
|
||||
if not value.is_integer():
|
||||
raise ValueError
|
||||
value = int(value)
|
||||
else:
|
||||
value = int(value)
|
||||
except (TypeError, ValueError) as err:
|
||||
raise ValueError(
|
||||
f"per_user_per_minute must be a non-negative integer, "
|
||||
f"got {per_user_per_minute!r}"
|
||||
) from err
|
||||
if value < 0:
|
||||
raise ValueError(
|
||||
f"per_user_per_minute must be >= 0, got {per_user_per_minute!r}"
|
||||
)
|
||||
if value == 0:
|
||||
return None
|
||||
from praisonaiagents.gateway import SlidingWindowRateLimitPolicy
|
||||
|
||||
return SlidingWindowRateLimitPolicy(max_requests=value, window_seconds=60.0)
|
||||
|
||||
|
||||
def build_admission_gate(
|
||||
max_concurrent_runs: int = 0,
|
||||
queue_depth: int = 0,
|
||||
@@ -603,6 +727,7 @@ def build_admission_gate(
|
||||
policy: Optional[object] = None,
|
||||
resource_policy: Optional[object] = None,
|
||||
max_concurrent_runs_per_scope: int = 0,
|
||||
per_user_per_minute: int = 0,
|
||||
) -> Optional[AdmissionGate]:
|
||||
"""Construct an :class:`AdmissionGate` from config, or return ``None``.
|
||||
|
||||
@@ -620,7 +745,13 @@ def build_admission_gate(
|
||||
gate so it queues under soft RSS pressure and sheds under hard pressure. A
|
||||
gate is built when *either* a concurrency ceiling *or* a resource policy is
|
||||
configured, so a memory-only deployment still gets backpressure.
|
||||
|
||||
``per_user_per_minute`` (Issue #6215) caps how many turns one identity may
|
||||
start per minute; ``0`` (the default) leaves ingress unlimited. Over the cap
|
||||
the turn is refused with a ``retry_after`` hint before any work starts.
|
||||
"""
|
||||
# Raises ValueError on a negative / non-integer value (fail fast).
|
||||
rate_limit_policy = build_rate_limit_policy(per_user_per_minute)
|
||||
if policy is None:
|
||||
# Only an explicit ``0`` (or ``None``) disables admission control. A
|
||||
# *negative* ceiling is a misconfiguration, not "disabled": fall through
|
||||
@@ -634,8 +765,12 @@ def build_admission_gate(
|
||||
if no_ceiling:
|
||||
# No concurrency ceiling: build a resource-only gate when a
|
||||
# resource policy is supplied, else no gate (legacy behaviour).
|
||||
if resource_policy is not None:
|
||||
return AdmissionGate(None, resource_policy=resource_policy)
|
||||
if resource_policy is not None or rate_limit_policy is not None:
|
||||
return AdmissionGate(
|
||||
None,
|
||||
resource_policy=resource_policy,
|
||||
rate_limit_policy=rate_limit_policy,
|
||||
)
|
||||
return None
|
||||
try:
|
||||
from praisonaiagents.gateway import ConcurrencyLimitPolicy
|
||||
@@ -647,4 +782,6 @@ def build_admission_gate(
|
||||
overflow_policy=overflow_policy,
|
||||
max_concurrent_runs_per_scope=max_concurrent_runs_per_scope,
|
||||
)
|
||||
return AdmissionGate(policy, resource_policy=resource_policy)
|
||||
return AdmissionGate(
|
||||
policy, resource_policy=resource_policy, rate_limit_policy=rate_limit_policy
|
||||
)
|
||||
@@ -912,6 +912,8 @@ class GatewayServerSchema(BaseModel):
|
||||
max_concurrent_runs: Optional[int] = Field(None, ge=0)
|
||||
queue_depth: Optional[int] = Field(None, ge=0)
|
||||
overflow_policy: Optional[str] = None
|
||||
# Per-user inbound request budget (#6215). 0 = unlimited (default).
|
||||
per_user_per_minute: Optional[int] = Field(None, ge=0)
|
||||
preauth_max_connections_per_ip: Optional[int] = Field(None, ge=0)
|
||||
max_unauthorized_frames: Optional[int] = Field(None, ge=0)
|
||||
# Graceful-drain windows (#2375 / #2533)
|
||||
|
||||
@@ -106,6 +106,49 @@ def _coerce_drain_timeout(value: Optional[Any]) -> Optional[float]:
|
||||
return timeout
|
||||
|
||||
|
||||
# Issue #6210: keys ``BotOS.from_config`` actually reads. Anything else in a
|
||||
# BotOS YAML used to be dropped silently.
|
||||
_BOTOS_TOP_KEYS = frozenset({
|
||||
"name", "agent", "platforms", "health", "supervision", "gateway",
|
||||
"drain_timeout", "max_concurrent_runs", "queue_depth", "overflow_policy",
|
||||
"max_concurrent_runs_per_scope", "max_rss_mb", "reliability",
|
||||
"per_user_per_minute",
|
||||
})
|
||||
_BOTOS_GATEWAY_KEYS = frozenset({
|
||||
"drain_timeout", "max_concurrent_runs", "queue_depth", "overflow_policy",
|
||||
"max_concurrent_runs_per_scope", "max_rss_mb", "reliability",
|
||||
"per_user_per_minute",
|
||||
})
|
||||
# Sections that only the gateway runtime (``praisonai gateway start``) honours.
|
||||
_GATEWAY_ONLY_KEYS = frozenset({
|
||||
"channels", "agents", "routing", "schedules", "rooms", "hooks",
|
||||
"access_groups", "mcp_servers", "identity",
|
||||
})
|
||||
|
||||
|
||||
def _check_botos_config_shape(raw: Dict[str, Any], path: str) -> None:
|
||||
"""Fail loudly on gateway-only YAML and warn on keys BotOS will not read."""
|
||||
gateway_only = sorted(k for k in raw if k in _GATEWAY_ONLY_KEYS)
|
||||
if gateway_only:
|
||||
raise ValueError(
|
||||
f"{path} is a gateway config (sections: {', '.join(gateway_only)}) "
|
||||
"which BotOS.from_config() cannot honour. Run it the same way as "
|
||||
"`praisonai gateway start --config`: "
|
||||
"asyncio.run(WebSocketGateway().start_with_config(path)) "
|
||||
"(from praisonai_bot import WebSocketGateway)."
|
||||
)
|
||||
ignored = [str(k) for k in raw if k not in _BOTOS_TOP_KEYS]
|
||||
gateway_cfg = raw.get("gateway")
|
||||
if isinstance(gateway_cfg, dict):
|
||||
ignored += [f"gateway.{k}" for k in gateway_cfg if k not in _BOTOS_GATEWAY_KEYS]
|
||||
if ignored:
|
||||
logger.warning(
|
||||
"BotOS.from_config(%s): ignoring unsupported keys %s "
|
||||
"(use WebSocketGateway().start_with_config() for full gateway configs)",
|
||||
path, ", ".join(sorted(ignored)),
|
||||
)
|
||||
|
||||
|
||||
class BotOS:
|
||||
"""Multi-platform bot orchestrator.
|
||||
|
||||
@@ -150,6 +193,7 @@ class BotOS:
|
||||
admission_policy: Optional[Any] = None,
|
||||
max_rss_mb: float = 0.0,
|
||||
reliability: Optional[str] = None,
|
||||
per_user_per_minute: int = 0,
|
||||
turn_lock: Optional[Any] = None,
|
||||
redis_config: Optional[Any] = None,
|
||||
degraded_registry: Optional[Any] = None,
|
||||
@@ -224,6 +268,8 @@ class BotOS:
|
||||
max_concurrent_runs_per_scope=max_concurrent_runs_per_scope,
|
||||
policy=admission_policy,
|
||||
resource_policy=build_memory_pressure_policy(max_rss_mb),
|
||||
# Issue #6215: per-user inbound request budget (0 = unlimited).
|
||||
per_user_per_minute=per_user_per_minute,
|
||||
)
|
||||
self._on_quiesce = None # optional callable(): host-suspend driver
|
||||
# W1: shared identity resolver applied to every managed bot —
|
||||
@@ -1600,11 +1646,22 @@ class BotOS:
|
||||
startup_grace: 60
|
||||
max_restarts_per_hour: 10
|
||||
|
||||
This is the lightweight BotOS dialect. A full ``gateway.yaml`` (with
|
||||
``channels:``/``agents:``/``routing:``/``schedules:`` …) is *not*
|
||||
accepted here — it raises ``ValueError`` instead of being silently
|
||||
degraded. Run such a file exactly like ``praisonai gateway start``::
|
||||
|
||||
from praisonai_bot import WebSocketGateway
|
||||
asyncio.run(WebSocketGateway().start_with_config("gateway.yaml"))
|
||||
|
||||
Args:
|
||||
path: Path to YAML config file.
|
||||
|
||||
Returns:
|
||||
Configured BotOS instance.
|
||||
|
||||
Raises:
|
||||
ValueError: If the file uses gateway-only sections.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
@@ -1620,6 +1677,8 @@ class BotOS:
|
||||
if not raw or not isinstance(raw, dict):
|
||||
raise ValueError(f"Invalid BotOS config: {path}")
|
||||
|
||||
_check_botos_config_shape(raw, path)
|
||||
|
||||
def _resolve_env(val):
|
||||
if isinstance(val, str):
|
||||
return re.sub(
|
||||
@@ -1788,6 +1847,11 @@ class BotOS:
|
||||
# ``reliability`` or ``gateway.reliability``; the preset composes drain
|
||||
# + admission while any explicit field above still overrides it.
|
||||
reliability = raw.get("reliability", gateway_cfg.get("reliability"))
|
||||
# Issue #6215: per-user inbound request budget. Default 0 = unlimited.
|
||||
# Validated (not truncated) by ``build_rate_limit_policy``.
|
||||
per_user_per_minute = raw.get(
|
||||
"per_user_per_minute", gateway_cfg.get("per_user_per_minute", 0)
|
||||
)
|
||||
|
||||
return cls(
|
||||
bots=bots,
|
||||
@@ -1800,6 +1864,7 @@ class BotOS:
|
||||
max_concurrent_runs_per_scope=max_concurrent_runs_per_scope,
|
||||
max_rss_mb=max_rss_mb,
|
||||
reliability=reliability,
|
||||
per_user_per_minute=per_user_per_minute,
|
||||
)
|
||||
|
||||
async def probe_all(self, timeout: float = 15.0) -> Dict[str, Any]:
|
||||
|
||||
@@ -1739,6 +1739,19 @@
|
||||
"default": null,
|
||||
"title": "Overflow Policy"
|
||||
},
|
||||
"per_user_per_minute": {
|
||||
"anyOf": [
|
||||
{
|
||||
"minimum": 0,
|
||||
"type": "integer"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"title": "Per User Per Minute"
|
||||
},
|
||||
"preauth_max_connections_per_ip": {
|
||||
"anyOf": [
|
||||
{
|
||||
|
||||
@@ -16,6 +16,8 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextvars
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import time
|
||||
import uuid
|
||||
@@ -373,7 +375,13 @@ class GatewayApiEndpoints:
|
||||
if gate is not None and getattr(gate, "enabled", False):
|
||||
from ..bots._admission import AdmissionRejected
|
||||
try:
|
||||
async with gate.admit(session_id=session.session_id):
|
||||
# Issue #6215: charge a stable caller identity, only when a
|
||||
# budget is set (keeps duck-typed gates' signature intact).
|
||||
rk = (
|
||||
{"rate_key": self._rate_key_var.get() or "ip:unknown"}
|
||||
if getattr(gate, "_rate_limit_enabled", False) else {}
|
||||
)
|
||||
async with gate.admit(session_id=session.session_id, **rk):
|
||||
result = await self._gw._dispatch_agent_turn(
|
||||
agent, content, on_complete=_capture, **extra
|
||||
)
|
||||
@@ -554,6 +562,41 @@ class GatewayApiEndpoints:
|
||||
session_id = f"api:{key}:{agent_id}"
|
||||
return self._gw.create_session(agent_id=agent_id, session_id=session_id)
|
||||
|
||||
# Per-request stable identity for the per-user request budget (#6215);
|
||||
# inherited by the turn's task/stream context like ``_stream_turn_var``.
|
||||
_rate_key_var: "contextvars.ContextVar[Optional[str]]" = (
|
||||
contextvars.ContextVar("praisonai_gw_rate_key", default=None)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _rate_key(request: Any, auth_token: Optional[str] = None) -> str:
|
||||
"""Stable caller identity that a client-chosen value cannot reset.
|
||||
|
||||
A bearer token is used only when it *is* the gateway's configured
|
||||
``auth_token`` (so it was validated); any other bearer value is
|
||||
attacker-chosen and would mint a fresh budget per request, so such
|
||||
callers are charged to their peer address. Never returns ``None``, so the
|
||||
budget never falls back to the client-chosen session id.
|
||||
"""
|
||||
auth = request.headers.get("authorization", "")
|
||||
bearer = auth[7:] if auth.startswith("Bearer ") else ""
|
||||
if bearer and auth_token and hmac.compare_digest(
|
||||
bearer.encode("utf-8"), str(auth_token).encode("utf-8")
|
||||
):
|
||||
return "token:" + hashlib.sha256(bearer.encode("utf-8")).hexdigest()[:16]
|
||||
client = getattr(request, "client", None)
|
||||
host = getattr(client, "host", None)
|
||||
return f"ip:{host}" if host else "ip:unknown"
|
||||
|
||||
def _bind_rate_key(self, request: Any) -> None:
|
||||
"""Bind this request's per-user budget key for its turn (#6215).
|
||||
|
||||
Called by the auth-guarded route wrapper after auth has passed.
|
||||
"""
|
||||
cfg = getattr(self._gw, "config", None)
|
||||
token = getattr(cfg, "auth_token", None)
|
||||
self._rate_key_var.set(self._rate_key(request, token))
|
||||
|
||||
@staticmethod
|
||||
def _caller_key(request: Any) -> str:
|
||||
"""Derive a stable per-caller session key from the request.
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
import os
|
||||
import socket
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Optional, Tuple
|
||||
|
||||
@@ -86,19 +87,21 @@ def _process_create_time(pid: int) -> Optional[float]:
|
||||
class GatewayPIDLock:
|
||||
"""Manages PID lock file for single-instance gateway enforcement."""
|
||||
|
||||
def __init__(self, lock_dir: Optional[Path] = None, host: str = "127.0.0.1", port: int = 8765):
|
||||
def __init__(self, lock_dir: Optional[Path] = None, host: str = "127.0.0.1", port: int = 8765,
|
||||
lock_name: Optional[str] = None):
|
||||
"""Initialize PID lock manager.
|
||||
|
||||
Args:
|
||||
lock_dir: Directory for lock file (default: ~/.praisonai/)
|
||||
host: Gateway host
|
||||
port: Gateway port
|
||||
lock_name: Override the lock file name (used by the state lock)
|
||||
"""
|
||||
self.lock_dir = lock_dir or Path.home() / ".praisonai"
|
||||
# Make lock file specific to host and port to allow multiple instances
|
||||
safe_host = host.replace(":", "_").replace(".", "_")
|
||||
self.lock_file = self.lock_dir / f"gateway-{safe_host}-{port}.pid"
|
||||
self.lock_dir.mkdir(exist_ok=True)
|
||||
self.lock_file = self.lock_dir / (lock_name or f"gateway-{safe_host}-{port}.pid")
|
||||
self.lock_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def acquire_lock(self, host: str, port: int) -> bool:
|
||||
"""Acquire the PID lock for the gateway.
|
||||
@@ -110,6 +113,27 @@ class GatewayPIDLock:
|
||||
Returns:
|
||||
True if lock acquired successfully, False if conflict
|
||||
"""
|
||||
# Serialise check-then-claim across processes so two simultaneous
|
||||
# starts can never both observe "no live holder" and both win.
|
||||
with self._claim_guard():
|
||||
return self._acquire_locked(host, port)
|
||||
|
||||
@contextmanager
|
||||
def _claim_guard(self):
|
||||
"""Hold an OS advisory lock on a sidecar file for the claim (POSIX)."""
|
||||
try:
|
||||
import fcntl
|
||||
except ImportError: # Windows: exclusive link() below still applies
|
||||
yield
|
||||
return
|
||||
guard = open(self.lock_file.with_name(self.lock_file.name + ".guard"), "a")
|
||||
try:
|
||||
fcntl.flock(guard, fcntl.LOCK_EX)
|
||||
yield
|
||||
finally:
|
||||
guard.close() # closing drops the flock
|
||||
|
||||
def _acquire_locked(self, host: str, port: int) -> bool:
|
||||
current_pid = os.getpid()
|
||||
|
||||
# Check if lock file exists
|
||||
@@ -120,6 +144,9 @@ class GatewayPIDLock:
|
||||
lines = lock_content.split('\n')
|
||||
if lines:
|
||||
existing_pid = int(lines[0])
|
||||
if existing_pid == current_pid:
|
||||
# Re-acquire by the same process (in-process restart).
|
||||
return True
|
||||
existing_create_time = None
|
||||
if len(lines) >= 5 and lines[4]:
|
||||
try:
|
||||
@@ -145,14 +172,29 @@ class GatewayPIDLock:
|
||||
create_time = _process_create_time(current_pid)
|
||||
create_time_line = "" if create_time is None else repr(create_time)
|
||||
lock_content = f"{current_pid}\n{host}\n{port}\n{int(time.time())}\n{create_time_line}\n"
|
||||
# Exclusive claim: write a private temp file, then hard-link it into
|
||||
# place. ``link`` fails if the lock already exists, so a racing claimant
|
||||
# can never overwrite a live holder (unlike rename/replace).
|
||||
temp_lock_file = self.lock_file.with_name(f"{self.lock_file.name}.{current_pid}.tmp")
|
||||
try:
|
||||
# Use a temporary file and atomic rename to prevent race conditions
|
||||
temp_lock_file = self.lock_file.with_suffix(".tmp")
|
||||
temp_lock_file.write_text(lock_content)
|
||||
temp_lock_file.replace(self.lock_file)
|
||||
try:
|
||||
os.link(temp_lock_file, self.lock_file)
|
||||
except FileExistsError:
|
||||
return False
|
||||
except OSError:
|
||||
# Filesystem without hard links: fall back to O_EXCL create.
|
||||
fd = os.open(self.lock_file, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644)
|
||||
with os.fdopen(fd, "w") as f:
|
||||
f.write(lock_content)
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
finally:
|
||||
try:
|
||||
temp_lock_file.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def release_lock(self) -> None:
|
||||
"""Release the PID lock."""
|
||||
@@ -253,6 +295,43 @@ class GatewayPIDLock:
|
||||
pass
|
||||
|
||||
|
||||
def resolve_state_dir() -> Path:
|
||||
"""Return the gateway's shared mutable state root (``~/.praisonai``).
|
||||
|
||||
The durable outbox, DLQ, pairing and hook stores are hard-wired under
|
||||
``~/.praisonai`` (they do not honour ``PRAISONAI_HOME``), so the home root -
|
||||
not the TCP port or ``PRAISONAI_HOME`` - is what identifies a gateway.
|
||||
"""
|
||||
return Path.home() / ".praisonai"
|
||||
|
||||
|
||||
def state_lock(state_dir: Optional[Path] = None, host: str = "127.0.0.1",
|
||||
port: int = 8765) -> GatewayPIDLock:
|
||||
"""Single-instance lock keyed on the shared state directory, regardless of port.
|
||||
|
||||
Two gateways sharing one state root would poll the same bot tokens and write
|
||||
the same SQLite stores concurrently (split-brain), so only one may own it.
|
||||
"""
|
||||
return GatewayPIDLock(lock_dir=state_dir or resolve_state_dir(), host=host,
|
||||
port=port, lock_name="gateway.state.pid")
|
||||
|
||||
|
||||
def format_state_collision_error(state_dir: Path, lock_info: Optional[dict] = None) -> str:
|
||||
"""Error shown when another live gateway already owns the shared state."""
|
||||
holder = ""
|
||||
if lock_info:
|
||||
holder = f" (PID {lock_info['pid']} on {lock_info['host']}:{lock_info['port']})"
|
||||
return "\n".join([
|
||||
f"Error: another gateway already owns the state at {state_dir}{holder}.",
|
||||
"",
|
||||
" Two gateways sharing state poll the same bot tokens and write the same",
|
||||
" queues, causing 409 Conflict errors and duplicate delivery.",
|
||||
" Stop it: praisonai gateway stop",
|
||||
" Or run an isolated gateway with its own state and bot tokens:",
|
||||
" HOME=/srv/gateway-b praisonai gateway start",
|
||||
])
|
||||
|
||||
|
||||
def format_collision_error(host: str, port: int, lock_info: Optional[dict] = None) -> str:
|
||||
"""Format a user-friendly error message for port collisions.
|
||||
|
||||
@@ -275,10 +354,11 @@ def format_collision_error(host: str, port: int, lock_info: Optional[dict] = Non
|
||||
pid = lock_info['pid']
|
||||
lines.extend([
|
||||
f" Another gateway may be running (PID {pid}).",
|
||||
f" Stop it: praisonai gateway stop",
|
||||
f" Or use a different port: GATEWAY_PORT={port + 1} praisonai gateway start",
|
||||
" Stop it: praisonai gateway stop",
|
||||
"",
|
||||
" Only ONE gateway process should poll each Telegram bot token.",
|
||||
" Only ONE gateway process may poll each bot token. To run a second,",
|
||||
" isolated gateway give it its own state, bot tokens and port:",
|
||||
f" HOME=/srv/gateway-b GATEWAY_PORT={port + 1} praisonai gateway start",
|
||||
])
|
||||
else:
|
||||
lines.extend([
|
||||
|
||||
@@ -1412,6 +1412,7 @@ class WebSocketGateway:
|
||||
|
||||
# PID lock for single-instance enforcement
|
||||
self._pid_lock: Optional[Any] = None
|
||||
self._state_lock: Optional[Any] = None
|
||||
|
||||
# Channel supervisor for resilient bot management. Share the gateway's
|
||||
# degraded-capability registry so the fleet crash-loop breaker (Issue
|
||||
@@ -1544,9 +1545,46 @@ class WebSocketGateway:
|
||||
logger.error("Gateway startup failed - another instance is running")
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Store PID lock for cleanup
|
||||
self._pid_lock = pid_lock
|
||||
|
||||
# When ``start_with_config`` already claimed the state root, its channel
|
||||
# bots are still polling if we fail here; only it may release that lock
|
||||
# (after channels stop), or a second gateway could claim the state.
|
||||
owns_state_lock = self._state_lock is None
|
||||
try:
|
||||
self._acquire_state_lock()
|
||||
await self._start_locked()
|
||||
except BaseException:
|
||||
# Any failure after claiming (bad config, bind error, cancel) must
|
||||
# not leave a live process holding the locks and blocking restarts.
|
||||
self._release_locks(state=owns_state_lock)
|
||||
raise
|
||||
|
||||
def _acquire_state_lock(self) -> None:
|
||||
"""Claim the shared state root (idempotent).
|
||||
|
||||
The port lock alone admits a second gateway on another port sharing the
|
||||
same state + bot tokens (split-brain), so the state root is locked too.
|
||||
"""
|
||||
if self._state_lock is not None:
|
||||
return
|
||||
from .port_utils import state_lock, resolve_state_dir, format_state_collision_error
|
||||
state_dir = resolve_state_dir()
|
||||
st_lock = state_lock(state_dir, host=self._host, port=self._port)
|
||||
if not st_lock.acquire_lock(self._host, self._port):
|
||||
logger.error("Gateway startup failed - another instance owns the shared state")
|
||||
raise RuntimeError(format_state_collision_error(state_dir, st_lock.get_lock_info()))
|
||||
self._state_lock = st_lock
|
||||
|
||||
def _release_locks(self, state: bool = True) -> None:
|
||||
"""Release the port lock and (if ``state``) the state lock (idempotent)."""
|
||||
for attr in ("_pid_lock", "_state_lock") if state else ("_pid_lock",):
|
||||
lock = getattr(self, attr, None)
|
||||
if lock:
|
||||
lock.release_lock()
|
||||
setattr(self, attr, None)
|
||||
|
||||
async def _start_locked(self) -> None:
|
||||
"""Body of :meth:`start` once the single-instance locks are held."""
|
||||
# Validate bind-aware auth configuration before starting
|
||||
from .auth import assert_external_bind_safe
|
||||
self.config.host = self._host
|
||||
@@ -2631,6 +2669,8 @@ class WebSocketGateway:
|
||||
auth_err = _check_auth(request)
|
||||
if auth_err:
|
||||
return auth_err
|
||||
# Issue #6215: bind the per-user budget key post-auth.
|
||||
self._api_endpoints._bind_rate_key(request)
|
||||
return await handler(request)
|
||||
return _wrapped
|
||||
|
||||
@@ -2768,14 +2808,8 @@ class WebSocketGateway:
|
||||
self._ledger_heartbeat_loop(), name="gateway-lifecycle-ledger"
|
||||
)
|
||||
try:
|
||||
# Lock cleanup on failure is handled by ``start()``.
|
||||
await self._server.serve()
|
||||
except Exception as e:
|
||||
# Clean up PID lock on any startup failure
|
||||
if hasattr(self, '_pid_lock') and self._pid_lock:
|
||||
self._pid_lock.release_lock()
|
||||
self._pid_lock = None
|
||||
# Re-raise the original exception
|
||||
raise
|
||||
finally:
|
||||
self._disarm_watchdog()
|
||||
await self._cancel_liveness_task()
|
||||
@@ -3993,9 +4027,7 @@ class WebSocketGateway:
|
||||
logger.debug("Failed to shut down pooled MCP client")
|
||||
self._mcp_pool.clear()
|
||||
|
||||
# Release PID lock
|
||||
if hasattr(self, '_pid_lock') and self._pid_lock:
|
||||
self._pid_lock.release_lock()
|
||||
self._release_locks()
|
||||
|
||||
logger.info("Gateway stopped")
|
||||
|
||||
@@ -4614,7 +4646,7 @@ class WebSocketGateway:
|
||||
|
||||
elif msg_type == "server_reply":
|
||||
# Issue #5351: a client's correlated answer to a server→client
|
||||
# interactive request (approval / choice / input).
|
||||
# interactive request (approval / choice / input / secret).
|
||||
request_id = data.get("request_id")
|
||||
value = data.get("value")
|
||||
if not request_id or not isinstance(value, str):
|
||||
@@ -4648,7 +4680,7 @@ class WebSocketGateway:
|
||||
return True
|
||||
# Answering an approval on behalf of the turn is an approval decision
|
||||
# and requires the APPROVALS scope (parity with approvals.resolve);
|
||||
# choice/input answers require WRITE like sending a message.
|
||||
# choice/input/secret answers require WRITE like sending a message.
|
||||
required_scope = (
|
||||
OperatorScope.APPROVALS if req_kind == "approval" else OperatorScope.WRITE
|
||||
)
|
||||
@@ -5089,7 +5121,14 @@ class WebSocketGateway:
|
||||
# so enforce the shared gate here too.
|
||||
from ..bots._admission import AdmissionRejected
|
||||
try:
|
||||
async with gate.admit(session_id=session.session_id):
|
||||
# Issue #6215: stable per-user budget key (only
|
||||
# when a budget is set; duck-typed gates unchanged).
|
||||
_rk = (
|
||||
{"rate_key": self._client_rate_key(client_id)}
|
||||
if getattr(gate, "_rate_limit_enabled", False)
|
||||
else {}
|
||||
)
|
||||
async with gate.admit(session_id=session.session_id, **_rk):
|
||||
if getattr(session, "_stop_epoch", 0) != dequeued_stop_epoch:
|
||||
response = _stop_sentinel
|
||||
else:
|
||||
@@ -5812,7 +5851,9 @@ class WebSocketGateway:
|
||||
|
||||
* ``approval`` → ``"allow"`` or ``"deny"``.
|
||||
* ``choice`` → one of the offered ``options`` (any value if none given).
|
||||
* ``input`` → any string.
|
||||
* ``input`` / ``secret`` → any string. A ``secret`` value is handed
|
||||
only to the waiting caller — never logged, echoed, or journalled
|
||||
(Issue #6223).
|
||||
"""
|
||||
kind = getattr(req, "kind", "input")
|
||||
if kind == "approval":
|
||||
@@ -5894,6 +5935,18 @@ class WebSocketGateway:
|
||||
|
||||
# ── Client Management (Public API) ────────────────────────────────
|
||||
|
||||
def _client_rate_key(self, client_id: Optional[str]) -> str:
|
||||
"""Stable per-user budget key for a WebSocket turn (#6215).
|
||||
|
||||
Session ids are client-chosen, so charge the peer address instead.
|
||||
With no known peer, callers share one ``ip:unknown`` budget rather
|
||||
than falling back to the client-chosen session id.
|
||||
"""
|
||||
ws = self._clients.get(client_id) if client_id else None
|
||||
peer = getattr(ws, "client", None)
|
||||
host = getattr(peer, "host", None)
|
||||
return f"ip:{host}" if host else "ip:unknown"
|
||||
|
||||
def add_client(self, client_id: str, websocket: Any) -> None:
|
||||
"""Register a client WebSocket connection.
|
||||
|
||||
@@ -11281,6 +11334,29 @@ class WebSocketGateway:
|
||||
self._shell_routed_agents.clear()
|
||||
self._shell_routed_agents.update(candidate.get("shell", {}))
|
||||
|
||||
@staticmethod
|
||||
def _rate_policy_from_config(cfg: Any) -> Any:
|
||||
"""Build the ``gateway.per_user_per_minute`` policy (#6215); may raise."""
|
||||
from ..bots._admission import build_rate_limit_policy
|
||||
|
||||
gw_cfg = (cfg or {}).get("gateway") or {}
|
||||
return build_rate_limit_policy(gw_cfg.get("per_user_per_minute", 0))
|
||||
|
||||
def _apply_rate_policy(self, policy: Any) -> None:
|
||||
"""Install a reloaded per-user budget on the live admission gate (#6215).
|
||||
|
||||
An existing gate is updated in place (bots/API/WS already share it); a
|
||||
budget added to a gateway that had no gate builds a rate-only gate,
|
||||
which channels started afterwards pick up via ``_stamp_admission_gate``.
|
||||
"""
|
||||
gate = getattr(self, "_admission_gate", None)
|
||||
if gate is not None:
|
||||
gate.set_rate_limit_policy(policy)
|
||||
elif policy is not None:
|
||||
from ..bots._admission import AdmissionGate
|
||||
|
||||
self._admission_gate = AdmissionGate(None, rate_limit_policy=policy)
|
||||
|
||||
async def _restore_previous_config(
|
||||
self, error: str, changed_paths: Set[str]
|
||||
) -> None:
|
||||
@@ -11302,6 +11378,10 @@ class WebSocketGateway:
|
||||
)
|
||||
return
|
||||
try:
|
||||
try:
|
||||
self._apply_rate_policy(self._rate_policy_from_config(prev_cfg))
|
||||
except ValueError:
|
||||
pass # previous config was validated when it went live
|
||||
prev_agents_cfg = prev_cfg.get("agents", {})
|
||||
if prev_agents_cfg:
|
||||
provider_cfg = prev_cfg.get("provider", {})
|
||||
@@ -11431,6 +11511,13 @@ class WebSocketGateway:
|
||||
# drain window keep resolving the previous agents until the cutover
|
||||
# commits.
|
||||
report = self._validate_candidate(new_cfg)
|
||||
# Issue #6215: validate the per-user budget with the candidate so a
|
||||
# bad value rejects the reload instead of failing mid-cutover.
|
||||
try:
|
||||
rate_policy = self._rate_policy_from_config(new_cfg)
|
||||
except ValueError as e:
|
||||
report.ok = False
|
||||
report.failures.append(f"gateway.per_user_per_minute: {e}")
|
||||
if not report.ok:
|
||||
error = "; ".join(report.failures) or "candidate validation failed"
|
||||
logger.error(f"Reload rejected — {error}; keeping previous config")
|
||||
@@ -11453,6 +11540,7 @@ class WebSocketGateway:
|
||||
# channel is accepting turns (greptile #1 — never expose the
|
||||
# candidate before the cutover commits).
|
||||
self._activate_candidate(report.candidate)
|
||||
self._apply_rate_policy(rate_policy)
|
||||
|
||||
channels_cfg = new_cfg.get("channels", {})
|
||||
try:
|
||||
@@ -12181,6 +12269,8 @@ class WebSocketGateway:
|
||||
queue_depth=_queue_depth,
|
||||
overflow_policy=_overflow,
|
||||
max_concurrent_runs_per_scope=_max_runs_per_scope,
|
||||
# Issue #6215: per-user inbound request budget (0 = unlimited).
|
||||
per_user_per_minute=gw_cfg.get("per_user_per_minute", 0),
|
||||
)
|
||||
if self._admission_gate is not None:
|
||||
logger.info(
|
||||
@@ -12436,6 +12526,9 @@ class WebSocketGateway:
|
||||
except (NotImplementedError, OSError, ValueError):
|
||||
pass
|
||||
|
||||
# Claim the state root BEFORE channels start: channel setup replays the
|
||||
# inbound journal, which a rejected second gateway must never touch.
|
||||
self._acquire_state_lock()
|
||||
try:
|
||||
await _run_all()
|
||||
finally:
|
||||
@@ -12486,3 +12579,4 @@ class WebSocketGateway:
|
||||
logger.warning("Error draining websocket sessions: %s", e)
|
||||
finally:
|
||||
self._cancel_shutdown_deadline()
|
||||
self._release_locks()
|
||||
@@ -67,6 +67,37 @@ def test_server_reply_as_dict():
|
||||
}
|
||||
|
||||
|
||||
def test_server_reply_repr_hides_value():
|
||||
reply = GatewayServerReply(request_id="r1", value="hunter2")
|
||||
assert "hunter2" not in repr(reply)
|
||||
assert reply.value == "hunter2"
|
||||
|
||||
|
||||
def test_secret_request_resolves_only_to_caller():
|
||||
"""Issue #6223: a secret reply reaches the waiting caller and nowhere else."""
|
||||
async def go():
|
||||
gw = WebSocketGateway()
|
||||
sent = _connect(gw, "c1", "sess-1")
|
||||
req = GatewayServerRequest(
|
||||
request_id="req-s", kind="secret", prompt="sudo password?",
|
||||
session_id="sess-1",
|
||||
)
|
||||
task = asyncio.create_task(gw.request_channel.request(req, timeout_s=5))
|
||||
await asyncio.sleep(0)
|
||||
assert sent[0]["request"]["kind"] == "secret"
|
||||
|
||||
await gw._handle_client_message(
|
||||
"c1", {"type": "server_reply", "request_id": "req-s", "value": "hunter2"}
|
||||
)
|
||||
reply = await asyncio.wait_for(task, timeout=1)
|
||||
assert reply.value == "hunter2"
|
||||
# The value is never echoed back over the wire (ack carries no value).
|
||||
assert all("hunter2" not in repr(frame) for frame in sent)
|
||||
assert gw.request_channel.open_requests("sess-1") == []
|
||||
|
||||
asyncio.run(go())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Test harness: a gateway with a connected, session-bound client that records
|
||||
# every frame the server sends it.
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""A gateway's single-instance identity is its shared state, not its port (#6208)."""
|
||||
|
||||
import asyncio
|
||||
import multiprocessing
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from praisonai_bot.gateway.port_utils import (
|
||||
format_collision_error,
|
||||
format_state_collision_error,
|
||||
resolve_state_dir,
|
||||
state_lock,
|
||||
)
|
||||
|
||||
|
||||
def _claim(d, start, done, results):
|
||||
start.wait() # release every worker at once so the claims genuinely race
|
||||
results.put(state_lock(Path(d)).acquire_lock("127.0.0.1", 8765))
|
||||
done.wait() # stay alive until all claimed, so no lock is seen as stale
|
||||
|
||||
|
||||
def _seed_foreign_holder(lock, pid):
|
||||
lock.lock_file.write_text(f"{pid}\n127.0.0.1\n8765\n0\n\n")
|
||||
|
||||
|
||||
def test_second_gateway_on_another_port_is_refused(tmp_path, monkeypatch):
|
||||
first = state_lock(tmp_path, port=8765)
|
||||
_seed_foreign_holder(first, os.getppid()) # a live process other than us
|
||||
second = state_lock(tmp_path, port=8766)
|
||||
assert second.lock_file == first.lock_file
|
||||
assert second.acquire_lock("127.0.0.1", 8766) is False
|
||||
msg = format_state_collision_error(tmp_path, second.get_lock_info())
|
||||
assert str(tmp_path) in msg and "HOME=" in msg and "PRAISONAI_HOME" not in msg
|
||||
|
||||
|
||||
def test_distinct_state_dirs_are_independent(tmp_path):
|
||||
a = state_lock(tmp_path / "a")
|
||||
_seed_foreign_holder(a, os.getppid())
|
||||
assert state_lock(tmp_path / "b").acquire_lock("127.0.0.1", 8766) is True
|
||||
|
||||
|
||||
def test_stale_state_lock_is_reclaimed(tmp_path, monkeypatch):
|
||||
lock = state_lock(tmp_path)
|
||||
_seed_foreign_holder(lock, 999999)
|
||||
monkeypatch.setattr(lock, "_is_process_running", lambda pid: False)
|
||||
assert lock.acquire_lock("127.0.0.1", 8765) is True
|
||||
|
||||
|
||||
def test_state_dir_ignores_praisonai_home(tmp_path, monkeypatch):
|
||||
# The stores live under ~/.praisonai regardless of PRAISONAI_HOME, so a
|
||||
# different PRAISONAI_HOME must not escape the lock.
|
||||
monkeypatch.setenv("PRAISONAI_HOME", str(tmp_path))
|
||||
assert resolve_state_dir() == Path.home() / ".praisonai"
|
||||
|
||||
|
||||
def test_concurrent_claims_have_exactly_one_winner(tmp_path):
|
||||
ctx = multiprocessing.get_context("spawn")
|
||||
n = 8
|
||||
start, done, results = ctx.Barrier(n), ctx.Barrier(n), ctx.Queue()
|
||||
procs = [ctx.Process(target=_claim, args=(str(tmp_path), start, done, results)) for _ in range(n)]
|
||||
for p in procs:
|
||||
p.start()
|
||||
outcomes = [results.get(timeout=60) for _ in range(n)]
|
||||
for p in procs:
|
||||
p.join(timeout=60)
|
||||
assert sum(outcomes) == 1
|
||||
|
||||
|
||||
def test_claim_never_overwrites_existing_lock(tmp_path):
|
||||
lock = state_lock(tmp_path)
|
||||
_seed_foreign_holder(lock, os.getppid())
|
||||
assert lock.acquire_lock("127.0.0.1", 8766) is False
|
||||
assert lock.lock_file.read_text().startswith(str(os.getppid()))
|
||||
|
||||
|
||||
def test_failed_startup_releases_both_locks(tmp_path, monkeypatch):
|
||||
from praisonai_bot.gateway import server as srv
|
||||
|
||||
monkeypatch.setenv("HOME", str(tmp_path)) # isolate both lock files
|
||||
gw = srv.WebSocketGateway.__new__(srv.WebSocketGateway)
|
||||
gw._is_running, gw._host, gw._port = False, "127.0.0.1", 0
|
||||
gw._pid_lock = gw._state_lock = None
|
||||
|
||||
async def boom():
|
||||
raise RuntimeError("bad config")
|
||||
|
||||
gw._start_locked = boom
|
||||
monkeypatch.setattr("praisonai_bot.gateway.port_utils.check_port_available", lambda h, p: (True, None))
|
||||
with pytest.raises(RuntimeError, match="bad config"):
|
||||
asyncio.run(gw.start())
|
||||
assert gw._pid_lock is None and gw._state_lock is None
|
||||
assert not list((tmp_path / ".praisonai").glob("*.pid"))
|
||||
|
||||
|
||||
def test_start_failure_keeps_state_lock_owned_by_start_with_config(tmp_path, monkeypatch):
|
||||
# start_with_config claims the state lock before channels start; if start()
|
||||
# then fails, channels are still polling, so start() must not drop it.
|
||||
from praisonai_bot.gateway import server as srv
|
||||
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
gw = srv.WebSocketGateway.__new__(srv.WebSocketGateway)
|
||||
gw._is_running, gw._host, gw._port = False, "127.0.0.1", 0
|
||||
gw._pid_lock = gw._state_lock = None
|
||||
gw._acquire_state_lock()
|
||||
held = gw._state_lock
|
||||
|
||||
async def boom():
|
||||
raise RuntimeError("bind error")
|
||||
|
||||
gw._start_locked = boom
|
||||
monkeypatch.setattr("praisonai_bot.gateway.port_utils.check_port_available", lambda h, p: (True, None))
|
||||
with pytest.raises(RuntimeError, match="bind error"):
|
||||
asyncio.run(gw.start())
|
||||
assert gw._pid_lock is None
|
||||
assert gw._state_lock is held and held.lock_file.exists()
|
||||
gw._release_locks() # start_with_config's finally, after channels stop
|
||||
assert gw._state_lock is None and not held.lock_file.exists()
|
||||
|
||||
|
||||
def test_collision_error_no_longer_suggests_bare_port_change():
|
||||
msg = format_collision_error("127.0.0.1", 8765, {"pid": 1, "is_running": True})
|
||||
assert "Or use a different port" not in msg
|
||||
assert "HOME=" in msg and "PRAISONAI_HOME" not in msg
|
||||
@@ -0,0 +1,240 @@
|
||||
"""Per-user inbound request budget in the admission gate (Issue #6215)."""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from praisonai_bot.bots._admission import (
|
||||
AdmissionGate,
|
||||
AdmissionRejected,
|
||||
build_admission_gate,
|
||||
)
|
||||
|
||||
|
||||
async def _turn(gate, user):
|
||||
async with gate.admit(session_id=user):
|
||||
return "ran"
|
||||
|
||||
|
||||
def test_unconfigured_means_no_gate():
|
||||
assert build_admission_gate(per_user_per_minute=0) is None
|
||||
|
||||
|
||||
def test_negative_budget_fails_fast():
|
||||
with pytest.raises(ValueError):
|
||||
build_admission_gate(per_user_per_minute=-1)
|
||||
|
||||
|
||||
def test_over_budget_user_refused_with_retry_after_others_unaffected():
|
||||
gate = build_admission_gate(per_user_per_minute=2)
|
||||
assert gate is not None and gate.enabled
|
||||
|
||||
async def run():
|
||||
assert await _turn(gate, "alice") == "ran"
|
||||
assert await _turn(gate, "alice") == "ran"
|
||||
with pytest.raises(AdmissionRejected) as exc:
|
||||
await _turn(gate, "alice")
|
||||
# A different user still has their own budget.
|
||||
assert await _turn(gate, "bob") == "ran"
|
||||
return exc.value
|
||||
|
||||
rej = asyncio.run(run())
|
||||
assert rej.retry_after is not None and 1 <= rej.retry_after <= 60
|
||||
assert "try again in" in rej.message
|
||||
stats = gate.stats()
|
||||
assert stats["rate_limited"] == 1
|
||||
assert stats["per_user_per_minute"] == 2
|
||||
assert stats["in_flight"] == 0
|
||||
|
||||
|
||||
def test_rate_limit_composes_with_concurrency_gate():
|
||||
gate = build_admission_gate(max_concurrent_runs=4, per_user_per_minute=1)
|
||||
assert gate.stats()["max_concurrent_runs"] == 4
|
||||
|
||||
async def run():
|
||||
await _turn(gate, "u")
|
||||
with pytest.raises(AdmissionRejected):
|
||||
await _turn(gate, "u")
|
||||
|
||||
asyncio.run(run())
|
||||
assert gate.stats()["in_flight"] == 0
|
||||
|
||||
|
||||
def test_stale_identities_are_pruned():
|
||||
from praisonaiagents.gateway import SlidingWindowRateLimitPolicy
|
||||
|
||||
pol = SlidingWindowRateLimitPolicy(max_requests=1, window_seconds=60)
|
||||
for i in range(10):
|
||||
pol.check(identity=f"u{i}", scope="ingress", now=0.0)
|
||||
AdmissionGate._prune_rate_limit_state(pol, now=120.0, cap=5)
|
||||
assert pol._windows == {}
|
||||
|
||||
|
||||
def test_botos_kwarg_and_yaml(tmp_path):
|
||||
try:
|
||||
from praisonai_bot.bots.botos import BotOS
|
||||
except ModuleNotFoundError: # pragma: no cover - optional deps
|
||||
pytest.skip("BotOS optional dependency not installed")
|
||||
assert BotOS(per_user_per_minute=5).admission_stats["per_user_per_minute"] == 5
|
||||
pytest.importorskip("yaml")
|
||||
cfg = tmp_path / "botos.yaml"
|
||||
cfg.write_text("gateway:\n per_user_per_minute: 7\n")
|
||||
assert BotOS.from_config(str(cfg)).admission_stats["per_user_per_minute"] == 7
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", [0.5, -0.5, 1.5, "0.5", "abc", True, -1])
|
||||
def test_fractional_or_invalid_budget_rejected(bad):
|
||||
with pytest.raises(ValueError):
|
||||
build_admission_gate(per_user_per_minute=bad)
|
||||
|
||||
|
||||
def test_botos_yaml_fractional_budget_rejected(tmp_path):
|
||||
try:
|
||||
from praisonai_bot.bots.botos import BotOS
|
||||
except ModuleNotFoundError: # pragma: no cover - optional deps
|
||||
pytest.skip("BotOS optional dependency not installed")
|
||||
pytest.importorskip("yaml")
|
||||
cfg = tmp_path / "botos.yaml"
|
||||
cfg.write_text("gateway:\n per_user_per_minute: 0.5\n")
|
||||
with pytest.raises(ValueError):
|
||||
BotOS.from_config(str(cfg))
|
||||
with pytest.raises(ValueError):
|
||||
BotOS(per_user_per_minute=0.5)
|
||||
|
||||
|
||||
def test_rate_key_shared_across_new_sessions():
|
||||
"""A caller cannot reset the budget by switching conversation ids."""
|
||||
gate = build_admission_gate(per_user_per_minute=1)
|
||||
|
||||
async def run():
|
||||
async with gate.admit(session_id="conv-1", rate_key="ip:1.2.3.4"):
|
||||
pass
|
||||
with pytest.raises(AdmissionRejected):
|
||||
async with gate.admit(session_id="conv-2", rate_key="ip:1.2.3.4"):
|
||||
pass
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_prune_runs_at_bounded_interval(monkeypatch):
|
||||
gate = build_admission_gate(per_user_per_minute=5)
|
||||
calls = []
|
||||
monkeypatch.setattr(
|
||||
AdmissionGate, "_prune_rate_limit_state",
|
||||
staticmethod(lambda pol, now, cap=4096: calls.append(now)),
|
||||
)
|
||||
for i in range(50):
|
||||
gate._check_rate_limit(f"user-{i}")
|
||||
assert len(calls) == 1
|
||||
|
||||
|
||||
def test_set_rate_limit_policy_swaps_budget_in_place():
|
||||
from praisonai_bot.bots._admission import build_rate_limit_policy
|
||||
|
||||
gate = build_admission_gate(max_concurrent_runs=4)
|
||||
gate.set_rate_limit_policy(build_rate_limit_policy(1))
|
||||
assert gate.stats()["per_user_per_minute"] == 1
|
||||
|
||||
async def run():
|
||||
await _turn(gate, "u")
|
||||
with pytest.raises(AdmissionRejected):
|
||||
await _turn(gate, "u")
|
||||
|
||||
asyncio.run(run())
|
||||
gate.set_rate_limit_policy(build_rate_limit_policy(0))
|
||||
assert gate.stats()["per_user_per_minute"] == 0
|
||||
assert asyncio.run(_turn(gate, "u")) == "ran"
|
||||
|
||||
|
||||
def test_gateway_reload_applies_budget():
|
||||
try:
|
||||
from praisonai_bot.gateway.server import WebSocketGateway
|
||||
except ModuleNotFoundError: # pragma: no cover - optional deps
|
||||
pytest.skip("gateway optional dependency not installed")
|
||||
gw = WebSocketGateway.__new__(WebSocketGateway)
|
||||
gw._admission_gate = None
|
||||
gw._apply_rate_policy(
|
||||
gw._rate_policy_from_config({"gateway": {"per_user_per_minute": 3}})
|
||||
)
|
||||
assert gw._admission_gate.stats()["per_user_per_minute"] == 3
|
||||
gw._apply_rate_policy(gw._rate_policy_from_config({"gateway": {}}))
|
||||
assert gw._admission_gate.stats()["per_user_per_minute"] == 0
|
||||
with pytest.raises(ValueError):
|
||||
gw._rate_policy_from_config({"gateway": {"per_user_per_minute": 0.5}})
|
||||
|
||||
|
||||
def test_api_rate_key_ignores_session_header():
|
||||
try:
|
||||
from praisonai_bot.gateway.api_endpoints import GatewayApiEndpoints
|
||||
except ModuleNotFoundError: # pragma: no cover - optional deps
|
||||
pytest.skip("gateway optional dependency not installed")
|
||||
|
||||
class _Req:
|
||||
def __init__(self, headers, host="10.0.0.1"):
|
||||
self.headers = headers
|
||||
self.client = type("C", (), {"host": host})()
|
||||
|
||||
a = GatewayApiEndpoints._rate_key(_Req({"x-session-id": "one"}))
|
||||
b = GatewayApiEndpoints._rate_key(_Req({"x-session-id": "two"}))
|
||||
assert a == b == "ip:10.0.0.1"
|
||||
tok = GatewayApiEndpoints._rate_key(
|
||||
_Req({"authorization": "Bearer secret-token"}), "secret-token"
|
||||
)
|
||||
assert tok.startswith("token:") and "secret" not in tok
|
||||
|
||||
|
||||
def test_api_rate_key_rotating_unvalidated_bearer_shares_ip_budget():
|
||||
"""An unvalidated bearer (auth open, or wrong token) cannot mint budgets."""
|
||||
try:
|
||||
from praisonai_bot.gateway.api_endpoints import GatewayApiEndpoints
|
||||
except ModuleNotFoundError: # pragma: no cover - optional deps
|
||||
pytest.skip("gateway optional dependency not installed")
|
||||
|
||||
class _Req:
|
||||
def __init__(self, headers, host="10.0.0.1"):
|
||||
self.headers = headers
|
||||
self.client = type("C", (), {"host": host})()
|
||||
|
||||
for configured in (None, "", "real-token"):
|
||||
keys = {
|
||||
GatewayApiEndpoints._rate_key(
|
||||
_Req({"authorization": f"Bearer rotate-{i}"}), configured
|
||||
)
|
||||
for i in range(5)
|
||||
}
|
||||
assert keys == {"ip:10.0.0.1"}
|
||||
|
||||
|
||||
def test_api_rate_key_missing_peer_never_falls_back_to_session():
|
||||
try:
|
||||
from praisonai_bot.gateway.api_endpoints import GatewayApiEndpoints
|
||||
except ModuleNotFoundError: # pragma: no cover - optional deps
|
||||
pytest.skip("gateway optional dependency not installed")
|
||||
|
||||
class _Req:
|
||||
headers = {"x-session-id": "chosen"}
|
||||
client = None
|
||||
|
||||
assert GatewayApiEndpoints._rate_key(_Req()) == "ip:unknown"
|
||||
|
||||
gate = build_admission_gate(per_user_per_minute=1)
|
||||
|
||||
async def run():
|
||||
async with gate.admit(session_id="conv-1", rate_key="ip:unknown"):
|
||||
pass
|
||||
with pytest.raises(AdmissionRejected):
|
||||
async with gate.admit(session_id="conv-2", rate_key="ip:unknown"):
|
||||
pass
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
|
||||
def test_ws_rate_key_missing_peer_is_shared():
|
||||
try:
|
||||
from praisonai_bot.gateway.server import WebSocketGateway
|
||||
except ModuleNotFoundError: # pragma: no cover - optional deps
|
||||
pytest.skip("gateway optional dependency not installed")
|
||||
gw = WebSocketGateway.__new__(WebSocketGateway)
|
||||
gw._clients = {}
|
||||
assert gw._client_rate_key(None) == "ip:unknown"
|
||||
assert gw._client_rate_key("gone") == "ip:unknown"
|
||||
@@ -239,3 +239,60 @@ class TestBotOSFromConfigEnhanced:
|
||||
assert botos.list_bots() == []
|
||||
finally:
|
||||
os.unlink(path)
|
||||
|
||||
|
||||
class TestBotOSFromConfigGatewayDrift:
|
||||
"""Issue #6210: gateway.yaml must not be silently degraded by BotOS."""
|
||||
|
||||
def _write_yaml(self, data):
|
||||
f = tempfile.NamedTemporaryFile(mode="w", suffix=".yaml", delete=False)
|
||||
yaml.dump(data, f)
|
||||
f.close()
|
||||
return f.name
|
||||
|
||||
@pytest.mark.parametrize("section", ["channels", "agents", "routing", "schedules"])
|
||||
def test_gateway_sections_raise(self, section):
|
||||
from praisonai_bot.bots.botos import BotOS
|
||||
|
||||
path = self._write_yaml({
|
||||
"platforms": {"telegram": {"token": "fake"}},
|
||||
section: {"x": {}},
|
||||
})
|
||||
try:
|
||||
with pytest.raises(ValueError, match="start_with_config"):
|
||||
BotOS.from_config(path)
|
||||
finally:
|
||||
os.unlink(path)
|
||||
|
||||
def test_unknown_keys_warn(self, caplog):
|
||||
from praisonai_bot.bots.botos import BotOS
|
||||
|
||||
path = self._write_yaml({
|
||||
"platforms": {"telegram": {"token": "fake"}},
|
||||
"helth": {"interval": 5},
|
||||
"gateway": {"port": 9999, "drain_timeout": 5},
|
||||
})
|
||||
try:
|
||||
with caplog.at_level("WARNING"):
|
||||
botos = BotOS.from_config(path)
|
||||
assert "telegram" in botos.list_bots()
|
||||
assert "helth" in caplog.text and "gateway.port" in caplog.text
|
||||
assert "gateway.drain_timeout" not in caplog.text
|
||||
finally:
|
||||
os.unlink(path)
|
||||
|
||||
def test_non_string_unknown_key_warns_not_crashes(self, caplog):
|
||||
from praisonai_bot.bots.botos import BotOS
|
||||
|
||||
path = self._write_yaml({
|
||||
"platforms": {"telegram": {"token": "fake"}},
|
||||
123: "ignored",
|
||||
"zzz": 1,
|
||||
})
|
||||
try:
|
||||
with caplog.at_level("WARNING"):
|
||||
botos = BotOS.from_config(path)
|
||||
assert "telegram" in botos.list_bots()
|
||||
assert "123" in caplog.text and "zzz" in caplog.text
|
||||
finally:
|
||||
os.unlink(path)
|
||||
@@ -7,6 +7,7 @@ Provides CLI interface for running agents in autonomous loops with:
|
||||
- Iteration limits and timeouts
|
||||
"""
|
||||
|
||||
import math
|
||||
from typing import Optional
|
||||
|
||||
import typer
|
||||
@@ -31,6 +32,8 @@ def loop_main(
|
||||
observe: bool = typer.Option(False, "--observe", "-o", help="Enable observability event logging"),
|
||||
recovery: bool = typer.Option(True, "--recovery/--no-recovery", help="Enable graduated doom loop recovery (default: on)"),
|
||||
level: Optional[str] = typer.Option(None, "--level", "-l", help="Autonomy level: suggest, auto_edit, full_auto"),
|
||||
max_budget: Optional[float] = typer.Option(None, "--max-budget", help="Hard USD spend cap for the whole run"),
|
||||
on_budget_exceeded: str = typer.Option("stop", "--on-budget-exceeded", help="When --max-budget is hit: stop or warn"),
|
||||
):
|
||||
"""Run an agent in an autonomous loop.
|
||||
|
||||
@@ -66,6 +69,8 @@ def loop_main(
|
||||
observe=observe,
|
||||
recovery=recovery,
|
||||
level=level,
|
||||
max_budget=max_budget,
|
||||
on_budget_exceeded=on_budget_exceeded,
|
||||
)
|
||||
|
||||
|
||||
@@ -80,8 +85,17 @@ def _run_loop(
|
||||
observe: bool = False,
|
||||
recovery: bool = True,
|
||||
level: Optional[str] = None,
|
||||
max_budget: Optional[float] = None,
|
||||
on_budget_exceeded: str = "stop",
|
||||
):
|
||||
"""Execute the autonomous loop."""
|
||||
if on_budget_exceeded not in ("stop", "warn"):
|
||||
typer.echo("Error: --on-budget-exceeded must be 'stop' or 'warn'", err=True)
|
||||
raise typer.Exit(2)
|
||||
# 0 would mean "no cap" downstream; negatives and NaN/inf are never a real cap.
|
||||
if max_budget is not None and not (math.isfinite(max_budget) and max_budget > 0):
|
||||
typer.echo("Error: --max-budget must be a positive number", err=True)
|
||||
raise typer.Exit(2)
|
||||
try:
|
||||
from praisonaiagents import Agent
|
||||
except ImportError:
|
||||
@@ -117,6 +131,13 @@ def _run_loop(
|
||||
|
||||
if not verbose:
|
||||
agent_kwargs["output"] = "silent"
|
||||
|
||||
# Only forward a budget when the user set one (None = no cap, zero overhead).
|
||||
if max_budget is not None:
|
||||
agent_kwargs["execution"] = {
|
||||
"max_budget": max_budget,
|
||||
"on_budget_exceeded": on_budget_exceeded,
|
||||
}
|
||||
|
||||
# Add completion promise instructions if provided
|
||||
if completion_promise:
|
||||
@@ -138,6 +159,8 @@ def _run_loop(
|
||||
typer.echo(" Context clearing: enabled")
|
||||
if timeout:
|
||||
typer.echo(f" Timeout: {timeout}s")
|
||||
if max_budget is not None:
|
||||
typer.echo(f" Max budget: ${max_budget} ({on_budget_exceeded})")
|
||||
typer.echo("")
|
||||
|
||||
# Run autonomous loop using unified API
|
||||
@@ -200,6 +223,8 @@ OPTIONS:
|
||||
-p, --completion-promise STR Promise text to signal completion
|
||||
-c, --clear-context Clear chat history between iterations
|
||||
-t, --timeout FLOAT Timeout in seconds
|
||||
--max-budget FLOAT Hard USD spend cap for the whole run
|
||||
--on-budget-exceeded STR stop (default) or warn when the cap is hit
|
||||
-m, --model STR LLM model to use
|
||||
-v, --verbose Show verbose output
|
||||
|
||||
|
||||
@@ -1712,6 +1712,7 @@ def run_main(
|
||||
# execution path does not discover and execute the same definitions twice.
|
||||
preloaded_agent_config = None
|
||||
custom_agent_model = None
|
||||
custom_agent_base_url = None
|
||||
if target or agent or command: # Any execution surface may need model/budget resolution
|
||||
import sys
|
||||
from praisonai_code.llm.credentials import ensure_configured_or_onboard
|
||||
@@ -1731,6 +1732,10 @@ def run_main(
|
||||
_preview_llm = (_preview_config or {}).get("llm")
|
||||
if isinstance(_preview_llm, dict):
|
||||
custom_agent_model = _preview_llm.get("model")
|
||||
# Gate an Ollama agent on the server it will really call.
|
||||
_base = _preview_llm.get("base_url")
|
||||
if isinstance(_base, str) and _base.strip():
|
||||
custom_agent_base_url = _base.strip()
|
||||
else:
|
||||
custom_agent_model = _preview_llm
|
||||
if isinstance(custom_agent_model, str):
|
||||
@@ -1747,7 +1752,9 @@ def run_main(
|
||||
# endpoint, credentials, sampling, and retry options.
|
||||
credential_model = model or custom_agent_model
|
||||
resolved_model = ensure_configured_or_onboard(
|
||||
model=credential_model, interactive=not _headless
|
||||
model=credential_model,
|
||||
interactive=not _headless,
|
||||
base_url=None if model else custom_agent_base_url,
|
||||
)
|
||||
if not (agent and model is None and custom_agent_model):
|
||||
model = resolved_model
|
||||
|
||||
@@ -123,6 +123,43 @@ def detect_local_endpoint():
|
||||
return None
|
||||
|
||||
|
||||
_OLLAMA_PREFIXES = ("ollama/", "ollama_chat/")
|
||||
|
||||
|
||||
def _ollama_host_for_model(
|
||||
model: Optional[str], base_url: Optional[str] = None
|
||||
) -> Optional[str]:
|
||||
"""Return the Ollama host to probe for an ``ollama/`` model, else ``None``.
|
||||
|
||||
Ollama needs no key, so an explicit Ollama model is gated on the server
|
||||
being reachable rather than on a credential. The endpoint the caller will
|
||||
actually use (e.g. an agent's ``llm.base_url``) wins, then
|
||||
``OLLAMA_API_BASE`` (what the runtime honours), then ``OLLAMA_HOST``, then
|
||||
the default port.
|
||||
"""
|
||||
import os
|
||||
|
||||
if not model or not model.lower().startswith(_OLLAMA_PREFIXES):
|
||||
return None
|
||||
from praisonai_code.llm.local_detect import _DEFAULT_OLLAMA_HOST
|
||||
|
||||
return (
|
||||
base_url
|
||||
or os.environ.get("OLLAMA_API_BASE")
|
||||
or os.environ.get("OLLAMA_HOST")
|
||||
or _DEFAULT_OLLAMA_HOST
|
||||
)
|
||||
|
||||
|
||||
def _ollama_reachable(host: str) -> bool:
|
||||
try:
|
||||
from praisonai_code.llm.local_detect import ollama_reachable
|
||||
|
||||
return ollama_reachable(host)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _provider_key_vars_for_model(model: str) -> tuple[str, ...]:
|
||||
"""Map a model id to the environment variable(s) for its provider.
|
||||
|
||||
@@ -258,6 +295,10 @@ def is_configured(model: Optional[str] = None) -> bool:
|
||||
if model is None:
|
||||
model = default_model_for_available_provider()
|
||||
|
||||
ollama_host = _ollama_host_for_model(model) if explicit_model else None
|
||||
if ollama_host is not None:
|
||||
return _ollama_reachable(ollama_host)
|
||||
|
||||
required_vars = _provider_key_vars_for_model(model)
|
||||
|
||||
if required_vars:
|
||||
@@ -303,6 +344,7 @@ def ensure_configured_or_onboard(
|
||||
*,
|
||||
model: Optional[str] = None,
|
||||
interactive: bool = True,
|
||||
base_url: Optional[str] = None,
|
||||
) -> Optional[str]:
|
||||
"""Shared first-run credential gate for LLM-bound interactive commands.
|
||||
|
||||
@@ -331,6 +373,9 @@ def ensure_configured_or_onboard(
|
||||
keyless local-first fallback (its own provider gate applies).
|
||||
interactive: Whether a wizard prompt may be shown. Callers pass ``False``
|
||||
for headless (``-p``, piped, ``--output json``) paths.
|
||||
base_url: The endpoint the caller will actually use (e.g. a custom
|
||||
agent's ``llm.base_url``). For an explicit Ollama model this is the
|
||||
server probed, instead of ``OLLAMA_API_BASE``/``OLLAMA_HOST``.
|
||||
|
||||
Returns:
|
||||
The model id to proceed with. May be a detected local model id when the
|
||||
@@ -343,6 +388,21 @@ def ensure_configured_or_onboard(
|
||||
import typer
|
||||
|
||||
inject_credentials_into_env()
|
||||
|
||||
# An explicit Ollama model needs a running server, not a key: probe the
|
||||
# endpoint the caller will really use and, if it is down, say so instead
|
||||
# of sending the user to an API-key wizard.
|
||||
ollama_host = _ollama_host_for_model(model, base_url)
|
||||
if ollama_host is not None:
|
||||
if _ollama_reachable(ollama_host):
|
||||
return model
|
||||
typer.echo(
|
||||
f"Error: Ollama isn't reachable at {ollama_host} — start it with "
|
||||
"`ollama serve` (or set OLLAMA_API_BASE).",
|
||||
err=True,
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
if is_configured(model):
|
||||
return model
|
||||
|
||||
@@ -360,11 +420,16 @@ def ensure_configured_or_onboard(
|
||||
return local.model
|
||||
|
||||
if not interactive:
|
||||
# Only mention auto-detection when it actually ran and found nothing.
|
||||
hint = f" for model {model}" if model else ""
|
||||
tail = (
|
||||
"" if model else
|
||||
"\n(no running local endpoint such as Ollama was found; one "
|
||||
"would be used automatically)"
|
||||
)
|
||||
typer.echo(
|
||||
"Error: No API key configured. Run: praisonai setup\n"
|
||||
"or set environment variables like OPENAI_API_KEY\n"
|
||||
"(a running local endpoint such as Ollama would be used "
|
||||
"automatically)",
|
||||
f"Error: No API key configured{hint}. Run: praisonai setup\n"
|
||||
f"or set environment variables like OPENAI_API_KEY{tail}",
|
||||
err=True,
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
@@ -19,9 +19,10 @@ from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
# Total time budget for a probe so the credential/first-run hot path is never
|
||||
# stalled when nothing is listening. Kept small deliberately.
|
||||
_PROBE_TIMEOUT_S = 0.15
|
||||
# Per-request probe timeout. Only paid on the keyless path; a closed port is
|
||||
# refused instantly, while a running Ollama can take ~0.35 s on its first call,
|
||||
# so a tighter budget made detection flaky (issue #6228).
|
||||
_PROBE_TIMEOUT_S = 1.0
|
||||
|
||||
# How long a negative probe is remembered so repeated invocations in the same
|
||||
# process (e.g. bare TUI then run) don't re-pay the connection latency.
|
||||
@@ -187,6 +188,11 @@ def _probe_ollama_tags(host: str) -> Optional[str]:
|
||||
return None
|
||||
|
||||
|
||||
def ollama_reachable(host: str) -> bool:
|
||||
"""Return ``True`` if an Ollama server answers ``/api/tags`` at ``host``."""
|
||||
return _get_json(_root_host(_with_scheme(host)) + "/api/tags") is not None
|
||||
|
||||
|
||||
def detect_local_model(*, use_cache: bool = True) -> Optional[LocalModel]:
|
||||
"""Detect a reachable local OpenAI-compatible endpoint.
|
||||
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
"""`praisonai loop --max-budget` reaches the agent's ExecutionConfig (Issue #6206)."""
|
||||
import pytest
|
||||
|
||||
pytest.importorskip("typer")
|
||||
|
||||
import typer
|
||||
|
||||
from praisonai_code.cli.commands import loop as loop_module
|
||||
|
||||
|
||||
class _Result:
|
||||
success = True
|
||||
completion_reason = "promise"
|
||||
iterations = 1
|
||||
duration_seconds = 0.1
|
||||
started_at = None
|
||||
output = "done"
|
||||
error = None
|
||||
|
||||
|
||||
def _capture(monkeypatch):
|
||||
captured = {}
|
||||
|
||||
class FakeAgent:
|
||||
def __init__(self, **kwargs):
|
||||
captured["agent_kwargs"] = kwargs
|
||||
|
||||
def start(self, prompt, **kwargs):
|
||||
return _Result()
|
||||
|
||||
import praisonaiagents
|
||||
monkeypatch.setattr(praisonaiagents, "Agent", FakeAgent)
|
||||
return captured
|
||||
|
||||
|
||||
def _run(**extra):
|
||||
loop_module._run_loop(
|
||||
prompt="build it", max_iterations=2, completion_promise=None,
|
||||
clear_context=False, timeout=None, model=None, verbose=False, **extra,
|
||||
)
|
||||
|
||||
|
||||
def test_max_budget_forwarded_to_execution(monkeypatch):
|
||||
captured = _capture(monkeypatch)
|
||||
_run(max_budget=0.5, on_budget_exceeded="warn")
|
||||
assert captured["agent_kwargs"]["execution"] == {
|
||||
"max_budget": 0.5, "on_budget_exceeded": "warn",
|
||||
}
|
||||
|
||||
|
||||
def test_no_budget_leaves_execution_unset(monkeypatch):
|
||||
captured = _capture(monkeypatch)
|
||||
_run()
|
||||
assert "execution" not in captured["agent_kwargs"]
|
||||
|
||||
|
||||
def test_invalid_on_budget_exceeded_rejected(monkeypatch):
|
||||
_capture(monkeypatch)
|
||||
with pytest.raises(typer.Exit):
|
||||
_run(max_budget=1.0, on_budget_exceeded="explode")
|
||||
|
||||
|
||||
def test_budget_dict_builds_real_execution_config():
|
||||
pytest.importorskip("praisonaiagents")
|
||||
from praisonaiagents import Agent
|
||||
|
||||
agent = Agent(name="a", execution={"max_budget": 0.25, "on_budget_exceeded": "stop"})
|
||||
assert agent._max_budget == 0.25
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", [0.0, -1.0, float("nan"), float("inf")])
|
||||
def test_invalid_max_budget_rejected(monkeypatch, bad):
|
||||
captured = _capture(monkeypatch)
|
||||
with pytest.raises(typer.Exit):
|
||||
_run(max_budget=bad)
|
||||
assert "agent_kwargs" not in captured
|
||||
@@ -0,0 +1,144 @@
|
||||
"""Tests for issue #6228 — explicit ``ollama/`` models need no API key.
|
||||
|
||||
An explicit Ollama model is gated on the server being reachable, not on
|
||||
``OLLAMA_HOST`` being set; the probe tolerates a slow first response; and the
|
||||
error text for an explicit model does not claim auto-detection would kick in.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
|
||||
import pytest
|
||||
import typer
|
||||
|
||||
from praisonai_code.llm import credentials, local_detect
|
||||
|
||||
|
||||
_ENV = (
|
||||
"OPENAI_API_KEY", "ANTHROPIC_API_KEY", "GEMINI_API_KEY", "GOOGLE_API_KEY",
|
||||
"GROQ_API_KEY", "COHERE_API_KEY", "OPENROUTER_API_KEY", "OLLAMA_HOST",
|
||||
"OLLAMA_API_BASE", "OPENAI_BASE_URL", "MODEL_NAME", "OPENAI_MODEL_NAME",
|
||||
"PRAISONAI_LOCAL_ENDPOINTS",
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clean_env(monkeypatch):
|
||||
for var in _ENV:
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
monkeypatch.setattr(credentials, "inject_credentials_into_env", lambda: False)
|
||||
local_detect.reset_cache()
|
||||
yield monkeypatch
|
||||
local_detect.reset_cache()
|
||||
|
||||
|
||||
class _Resp(io.BytesIO):
|
||||
status = 200
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *a):
|
||||
return False
|
||||
|
||||
|
||||
def _fake_urlopen(seen, delay=0.0):
|
||||
def _urlopen(url, timeout):
|
||||
seen.append((url, timeout))
|
||||
if delay > timeout:
|
||||
raise TimeoutError("probe timed out")
|
||||
return _Resp(json.dumps({"models": [{"name": "llama3.2:latest"}]}).encode())
|
||||
return _urlopen
|
||||
|
||||
|
||||
def _refuse(url, timeout):
|
||||
raise ConnectionRefusedError()
|
||||
|
||||
|
||||
def test_explicit_ollama_configured_when_reachable(clean_env):
|
||||
seen = []
|
||||
clean_env.setattr(local_detect.urllib.request, "urlopen", _fake_urlopen(seen))
|
||||
assert credentials.is_configured("ollama/llama3.2") is True
|
||||
assert seen[0][0] == "http://127.0.0.1:11434/api/tags"
|
||||
|
||||
|
||||
def test_explicit_ollama_honours_ollama_api_base(clean_env):
|
||||
seen = []
|
||||
clean_env.setenv("OLLAMA_API_BASE", "http://gpu-box:11434")
|
||||
clean_env.setattr(local_detect.urllib.request, "urlopen", _fake_urlopen(seen))
|
||||
assert credentials.is_configured("ollama_chat/llama3.2") is True
|
||||
assert seen[0][0] == "http://gpu-box:11434/api/tags"
|
||||
|
||||
|
||||
def test_explicit_ollama_not_configured_when_unreachable(clean_env):
|
||||
clean_env.setattr(local_detect.urllib.request, "urlopen", _refuse)
|
||||
assert credentials.is_configured("ollama/llama3.2") is False
|
||||
|
||||
|
||||
def test_unreachable_ollama_error_is_actionable(clean_env, capsys):
|
||||
clean_env.setattr(local_detect.urllib.request, "urlopen", _refuse)
|
||||
with pytest.raises(typer.Exit):
|
||||
credentials.ensure_configured_or_onboard(
|
||||
model="ollama/llama3.2", interactive=False
|
||||
)
|
||||
err = capsys.readouterr().err
|
||||
assert "Ollama isn't reachable at http://127.0.0.1:11434" in err
|
||||
assert "ollama serve" in err
|
||||
assert "API key" not in err
|
||||
|
||||
|
||||
def test_explicit_ollama_passes_gate_when_reachable(clean_env):
|
||||
clean_env.setattr(local_detect.urllib.request, "urlopen", _fake_urlopen([]))
|
||||
assert credentials.ensure_configured_or_onboard(
|
||||
model="ollama/llama3.2", interactive=False
|
||||
) == "ollama/llama3.2"
|
||||
|
||||
|
||||
def test_probe_tolerates_slow_first_response(clean_env):
|
||||
"""A ~0.4 s first /api/tags response must still be detected."""
|
||||
clean_env.setattr(
|
||||
local_detect.urllib.request, "urlopen", _fake_urlopen([], delay=0.4)
|
||||
)
|
||||
result = local_detect.detect_local_model(use_cache=False)
|
||||
assert result is not None
|
||||
assert result.model == "ollama/llama3.2:latest"
|
||||
|
||||
|
||||
def test_explicit_cloud_model_error_omits_auto_detect_hint(clean_env, capsys):
|
||||
clean_env.setattr(credentials, "detect_local_endpoint", lambda: None)
|
||||
clean_env.setattr(credentials, "is_configured", lambda model=None: False)
|
||||
with pytest.raises(typer.Exit):
|
||||
credentials.ensure_configured_or_onboard(
|
||||
model="gpt-4o-mini", interactive=False
|
||||
)
|
||||
err = capsys.readouterr().err
|
||||
assert "for model gpt-4o-mini" in err
|
||||
assert "automatically" not in err
|
||||
|
||||
|
||||
def test_keyless_no_model_error_mentions_auto_detect(clean_env, capsys):
|
||||
clean_env.setattr(credentials, "detect_local_endpoint", lambda: None)
|
||||
clean_env.setattr(credentials, "is_configured", lambda model=None: False)
|
||||
with pytest.raises(typer.Exit):
|
||||
credentials.ensure_configured_or_onboard(model=None, interactive=False)
|
||||
assert "automatically" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_gate_probes_agent_base_url_not_env_host(clean_env):
|
||||
"""A custom agent's ``llm.base_url`` is the server gated, not OLLAMA_HOST."""
|
||||
seen = []
|
||||
clean_env.setenv("OLLAMA_HOST", "http://down-box:11434")
|
||||
|
||||
def _urlopen(url, timeout):
|
||||
seen.append(url)
|
||||
if "down-box" in url:
|
||||
raise ConnectionRefusedError()
|
||||
return _Resp(json.dumps({"models": []}).encode())
|
||||
|
||||
clean_env.setattr(local_detect.urllib.request, "urlopen", _urlopen)
|
||||
assert credentials.ensure_configured_or_onboard(
|
||||
model="ollama/llama3.2",
|
||||
interactive=False,
|
||||
base_url="http://gpu-box:11434/v1",
|
||||
) == "ollama/llama3.2"
|
||||
assert seen == ["http://gpu-box:11434/api/tags"]
|
||||
@@ -717,6 +717,9 @@ def train_export(
|
||||
mtp_draft: bool = typer.Option(
|
||||
False, "--mtp-draft/--no-mtp-draft",
|
||||
help="Also download the stock MTP drafter (Gemma-4 only) for fast inference"),
|
||||
push: Optional[bool] = typer.Option(
|
||||
None, "--push/--no-push",
|
||||
help="ollama: push to ollama.com after `ollama create` (--no-push keeps it local)"),
|
||||
):
|
||||
"""
|
||||
Export / publish an ALREADY-trained model without re-running training.
|
||||
@@ -726,6 +729,7 @@ def train_export(
|
||||
praisonai-train export gguf --model-dir lora_model --quant q4_k_m # local .gguf
|
||||
praisonai-train export gguf --model-dir lora_model --hf me/my-model # local + push
|
||||
praisonai-train export ollama --model-dir lora_model --ollama me/my-model --quant q4_k_m
|
||||
praisonai-train export ollama --model-dir lora_model --ollama my-model --no-push # local only
|
||||
"""
|
||||
from ..output.console import get_output_controller
|
||||
|
||||
@@ -765,6 +769,9 @@ def train_export(
|
||||
cfg["hf_model_name"] = hf
|
||||
if ollama:
|
||||
cfg["ollama_model"] = ollama
|
||||
# Either explicit flag overrides the config; unset keeps config/default (push).
|
||||
if isinstance(push, bool):
|
||||
cfg["ollama_push"] = push
|
||||
|
||||
# Recover the model name (drives chat-template selection) from --base-model or
|
||||
# the trained model's config.json (_name_or_path), falling back to the dir name.
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
ollama_save: "true"
|
||||
ollama_push: "true" # set "false" to keep the Ollama model local (no upload to ollama.com)
|
||||
huggingface_save: "true"
|
||||
train: "true"
|
||||
|
||||
|
||||
@@ -220,8 +220,9 @@ def create_and_push_ollama_model(
|
||||
model_parameters: str,
|
||||
modelfile_content: str,
|
||||
quantization: Optional[str] = None,
|
||||
push: bool = True,
|
||||
) -> None:
|
||||
"""Create and push an Ollama model with proper daemon management.
|
||||
"""Create (and by default push) an Ollama model with proper daemon management.
|
||||
|
||||
Args:
|
||||
ollama_model: Name of the Ollama model
|
||||
@@ -229,6 +230,8 @@ def create_and_push_ollama_model(
|
||||
modelfile_content: Content for the Modelfile
|
||||
quantization: Optional quantization (e.g. "q4_k_m") passed to
|
||||
``ollama create --quantize`` so the model isn't stored as huge f16.
|
||||
push: When False, stop after a successful ``ollama create`` and keep the
|
||||
model local-only (nothing is uploaded to ollama.com).
|
||||
|
||||
Raises:
|
||||
RuntimeError: If ollama create/push operations fail (with actionable guidance)
|
||||
@@ -259,6 +262,10 @@ def create_and_push_ollama_model(
|
||||
f"{(create.stderr or create.stdout or '').strip()}"
|
||||
)
|
||||
|
||||
if not push:
|
||||
print(f"Created local Ollama model `{tag}` (not pushed). Run it with: ollama run {tag}")
|
||||
return
|
||||
|
||||
# Push with captured output so a 401/403 becomes an actionable message rather
|
||||
# than a raw non-zero exit.
|
||||
result = subprocess.run(["ollama", "push", tag], capture_output=True, text=True)
|
||||
|
||||
@@ -545,6 +545,7 @@ class TrainModel:
|
||||
"optim", "weight_decay", "lr_scheduler_type", "seed", "output_dir",
|
||||
"assistant_only_loss", "train_on_responses_only", "save_steps",
|
||||
"train", "huggingface_save", "huggingface_save_gguf", "ollama_save",
|
||||
"ollama_push",
|
||||
"method", "beta", "max_prompt_length", "desirable_weight", "undesirable_weight",
|
||||
"reward_funcs", "num_generations", "max_completion_length",
|
||||
"embedding_learning_rate",
|
||||
@@ -951,6 +952,10 @@ class TrainModel:
|
||||
if self._flag(dataset_info.get("shuffle"), default=False):
|
||||
dataset = dataset.shuffle(
|
||||
seed=int(dataset_info.get("seed", self.config.get("seed", 3407))))
|
||||
# OpenAI-style chat rows ({"messages": [...]}, as written by
|
||||
# praisonaiagents.eval.export_sft) are the ShareGPT shape under another name.
|
||||
if "messages" in dataset.column_names and "conversations" not in dataset.column_names:
|
||||
dataset = dataset.rename_column("messages", "conversations")
|
||||
if "conversations" in dataset.column_names:
|
||||
print("DEBUG: Standardizing dataset (ShareGPT style)...")
|
||||
dataset = standardize_sharegpt(dataset)
|
||||
@@ -970,8 +975,8 @@ class TrainModel:
|
||||
if len(dataset) == 0:
|
||||
raise ValueError(
|
||||
"All examples formatted to empty text. The dataset columns are "
|
||||
f"{columns_before}; this trainer understands ShareGPT "
|
||||
"('conversations'), Alpaca ('instruction'/'input'/'output') and "
|
||||
f"{columns_before}; this trainer understands chat "
|
||||
"('conversations' or 'messages'), Alpaca ('instruction'/'input'/'output') and "
|
||||
"raw text ('text'). Rename your columns to one of those shapes, "
|
||||
"or set chat_template to match your data.")
|
||||
return dataset
|
||||
@@ -1784,6 +1789,7 @@ class TrainModel:
|
||||
self.config.get('model_parameters', 'latest'),
|
||||
modelfile_content,
|
||||
quantization=self.config.get('quantization_method'),
|
||||
push=self._flag(self.config.get('ollama_push'), default=True),
|
||||
)
|
||||
|
||||
def run(self):
|
||||
|
||||
@@ -313,7 +313,8 @@ PARAMETER top_p 0.9
|
||||
create_and_push_ollama_model(
|
||||
self.config['ollama_model'],
|
||||
self.config['model_parameters'],
|
||||
modelfile_content
|
||||
modelfile_content,
|
||||
push=self._flag(self.config.get('ollama_push'), default=True),
|
||||
)
|
||||
|
||||
def run(self):
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
"""An OpenAI-style `messages` chat dataset formatted to empty text.
|
||||
|
||||
`praisonai-train validate` and `praisonaiagents.eval.export_sft` both use the
|
||||
`{"messages": [...]}` shape, but `process_dataset` only recognised
|
||||
`conversations`, so every row fell into the Alpaca branch, formatted to "",
|
||||
and the run died after the base model had already been loaded.
|
||||
"""
|
||||
|
||||
from praisonai_train.train.llm import trainer as trainer_mod
|
||||
|
||||
|
||||
class _FakeDataset:
|
||||
def __init__(self, rows):
|
||||
self.rows = rows
|
||||
|
||||
@property
|
||||
def column_names(self):
|
||||
return list(self.rows[0].keys()) if self.rows else []
|
||||
|
||||
def __len__(self):
|
||||
return len(self.rows)
|
||||
|
||||
def rename_column(self, src, dst):
|
||||
return _FakeDataset([{(dst if k == src else k): v for k, v in r.items()}
|
||||
for r in self.rows])
|
||||
|
||||
def map(self, func, batched, remove_columns):
|
||||
batch = {c: [r[c] for r in self.rows] for c in self.column_names}
|
||||
out = func(batch)
|
||||
return _FakeDataset([{"text": t} for t in out["text"]])
|
||||
|
||||
def filter(self, pred):
|
||||
return _FakeDataset([r for r in self.rows if pred(r)])
|
||||
|
||||
|
||||
class _Tokenizer:
|
||||
def apply_chat_template(self, convo, tokenize, add_generation_prompt):
|
||||
return "".join(f"<{m['role']}>{m['content']}" for m in convo)
|
||||
|
||||
|
||||
def test_messages_column_is_treated_as_conversations(monkeypatch):
|
||||
rows = [{"messages": [
|
||||
{"role": "system", "content": "s"},
|
||||
{"role": "user", "content": f"q{i}"},
|
||||
{"role": "assistant", "content": f"a{i}"},
|
||||
]} for i in range(3)]
|
||||
monkeypatch.setattr(trainer_mod, "load_dataset",
|
||||
lambda *a, **k: _FakeDataset(rows), raising=False)
|
||||
monkeypatch.setattr(trainer_mod, "standardize_sharegpt", lambda d: d,
|
||||
raising=False)
|
||||
|
||||
tm = trainer_mod.TrainModel.__new__(trainer_mod.TrainModel)
|
||||
tm.config = {"method": "sft"}
|
||||
tm.chat_tokenizer = _Tokenizer()
|
||||
|
||||
out = tm.process_dataset({"name": "hub/some-dataset"})
|
||||
assert len(out) == 3
|
||||
assert out.rows[0]["text"] == "<system>s<user>q0<assistant>a0"
|
||||
@@ -412,3 +412,88 @@ def test_export_command_bad_target_exits(monkeypatch, tmp_path):
|
||||
quant=None,
|
||||
base_model=None,
|
||||
)
|
||||
|
||||
|
||||
def test_ollama_no_push_skips_push(monkeypatch, tmp_path):
|
||||
"""push=False stops after `ollama create`; nothing is sent to ollama.com."""
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setattr(ollama_mod, "_check_ollama_disk", lambda *_a, **_k: None)
|
||||
monkeypatch.setattr(ollama_mod, "ensure_ollama_running", lambda *_a, **_k: None)
|
||||
calls = []
|
||||
|
||||
def _fake_run(cmd, *a, **k):
|
||||
calls.append(cmd)
|
||||
return types.SimpleNamespace(returncode=0, stdout="", stderr="")
|
||||
|
||||
monkeypatch.setattr(ollama_mod.subprocess, "run", _fake_run)
|
||||
ollama_mod.create_and_push_ollama_model(
|
||||
"my-model", "latest", "FROM ./model\n", push=False)
|
||||
assert [c[1] for c in calls] == ["create"]
|
||||
assert not any("push" in c for c in calls)
|
||||
|
||||
|
||||
def _ollama_export_cfg(monkeypatch, tmp_path, push, config_push=None):
|
||||
"""Run `export ollama` with a fake trainer; return the config it received."""
|
||||
import praisonai_train.cli.commands.train as cli
|
||||
|
||||
seen = {}
|
||||
|
||||
class _FakeTrainer:
|
||||
@classmethod
|
||||
def for_export(cls, cfg):
|
||||
obj = cls()
|
||||
obj.config = cfg
|
||||
obj.model = None
|
||||
obj.hf_tokenizer = None
|
||||
return obj
|
||||
|
||||
def load_model(self):
|
||||
return ("MODEL", "TOK")
|
||||
|
||||
def create_and_push_ollama_model(self):
|
||||
seen.update(self.config)
|
||||
|
||||
monkeypatch.setattr(trainer_mod, "TrainModel", _FakeTrainer)
|
||||
config = None
|
||||
if config_push is not None:
|
||||
config = tmp_path / "cfg.yaml"
|
||||
config.write_text(f"ollama_push: {config_push}\n")
|
||||
cli.train_export(
|
||||
target="ollama", model_dir=str(tmp_path), config=str(config) if config else None,
|
||||
ollama="my-model", hf=None, quant=None, base_model="unsloth/gemma-2-2b",
|
||||
mtp_draft=False, push=push,
|
||||
)
|
||||
return seen
|
||||
|
||||
|
||||
def test_export_ollama_no_push_flag_sets_config(monkeypatch, tmp_path):
|
||||
assert _ollama_export_cfg(monkeypatch, tmp_path, push=False)["ollama_push"] is False
|
||||
|
||||
|
||||
def test_export_ollama_explicit_push_overrides_config(monkeypatch, tmp_path):
|
||||
cfg = _ollama_export_cfg(monkeypatch, tmp_path, push=True, config_push="false")
|
||||
assert cfg["ollama_push"] is True
|
||||
|
||||
|
||||
def test_export_ollama_unset_push_keeps_config(monkeypatch, tmp_path):
|
||||
cfg = _ollama_export_cfg(monkeypatch, tmp_path, push=None, config_push="false")
|
||||
assert cfg["ollama_push"] is False # YAML bool preserved, not overridden
|
||||
assert "ollama_push" not in _ollama_export_cfg(monkeypatch, tmp_path, push=None)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value, expected", [
|
||||
(False, False), ("false", False), (True, True), ("true", True), (None, True),
|
||||
])
|
||||
def test_llm_trainer_forwards_ollama_push(monkeypatch, value, expected):
|
||||
"""TrainModel forwards `ollama_push` (bool or string) to the helper."""
|
||||
captured = {}
|
||||
monkeypatch.setattr(
|
||||
ollama_mod, "create_and_push_ollama_model",
|
||||
lambda *a, **k: captured.update(k))
|
||||
fake = types.SimpleNamespace(
|
||||
config={"ollama_model": "m", "ollama_push": value},
|
||||
prepare_modelfile_content=lambda: "FROM ./model\n",
|
||||
_flag=trainer_mod.TrainModel._flag,
|
||||
)
|
||||
trainer_mod.TrainModel.create_and_push_ollama_model(fake)
|
||||
assert captured["push"] is expected
|
||||
@@ -1,6 +1,8 @@
|
||||
# PraisonAI TypeScript Node.js AI Agents Framework
|
||||
|
||||
PraisonAI is a production-ready Multi AI Agents framework, designed to create AI Agents to automate and solve problems ranging from simple tasks to complex challenges. It provides a low-code solution to streamline the building and management of multi-agent LLM systems, emphasising simplicity, customisation, and effective human-agent collaboration.
|
||||
PraisonAI runs AI agents on your own computers, even fully offline, so private data never has to leave the building. It is built for highly regulated teams, and free and open source for everyone.
|
||||
|
||||
This package is the TypeScript and JavaScript SDK of the open-source PraisonAI agent framework. Build single agents, multi-agent workflows, tools and memory in Node.js, on hosted models or on local models such as Ollama. Data stays on your own machines only when the model runs locally too; with a hosted model such as the default OpenAI one, prompts are sent to that provider. The Python framework and the full docs are at [github.com/MervinPraison/PraisonAI](https://github.com/MervinPraison/PraisonAI) and [praison.ai/docs](https://praison.ai/docs).
|
||||
|
||||
## Installation
|
||||
|
||||
|
||||
@@ -219,7 +219,7 @@ TS-only members: `previousResult`?, `onToken`?, `signal`?, `onEvent`?, `options`
|
||||
|
||||
### `Agent.chat`
|
||||
|
||||
- Python: `src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3657`
|
||||
- Python: `src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3714`
|
||||
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:3592`
|
||||
- Counts: 17 python params: 7 exact, 9 camelCase, 1 alias, 0 flattened, 0 missing; 0 mismatches; 0 waived; 2 TS-only of 19
|
||||
|
||||
@@ -316,7 +316,7 @@ TS-only members: `config`
|
||||
### `Session.__init__`
|
||||
|
||||
- Python: `src/praisonai-agents/praisonaiagents/session/api.py:67`
|
||||
- TypeScript: `src/praisonai-ts/src/session/session.ts:29` (ctor `src/praisonai-ts/src/session/session.ts:132`)
|
||||
- TypeScript: `src/praisonai-ts/src/session/session.ts:29` (ctor `src/praisonai-ts/src/session/session.ts:133`)
|
||||
- Counts: 7 python params: 1 exact, 6 camelCase, 0 alias, 0 flattened, 0 missing; 1 mismatches; 1 waived; 5 TS-only of 12
|
||||
|
||||
| Python param | Kind | Py default | Py type | Match | TS name | TS default | TS type | Status |
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "praisonai",
|
||||
"version": "1.7.4",
|
||||
"description": "PraisonAI TypeScript AI Agents Framework - Node.js, npm, and Javascript AI Agents Framework",
|
||||
"description": "Open-source TypeScript and JavaScript AI agent framework for Node.js, on hosted or local models. The JavaScript SDK of PraisonAI.",
|
||||
"main": "dist/index.js",
|
||||
"module": "dist/esm/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
|
||||
@@ -1797,7 +1797,7 @@
|
||||
}
|
||||
],
|
||||
"python": {
|
||||
"location": "src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3657",
|
||||
"location": "src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3714",
|
||||
"resolved_class": "ChatMixin"
|
||||
},
|
||||
"ts_only": [
|
||||
@@ -5142,7 +5142,7 @@
|
||||
],
|
||||
"ts_only_required": [],
|
||||
"typescript": {
|
||||
"ctor_location": "src/praisonai-ts/src/session/session.ts:132",
|
||||
"ctor_location": "src/praisonai-ts/src/session/session.ts:133",
|
||||
"location": "src/praisonai-ts/src/session/session.ts:29",
|
||||
"options_parameters": [
|
||||
"config"
|
||||
|
||||
Vendored
+20
-5
@@ -72,7 +72,7 @@ export class MemoryCache extends BaseCache {
|
||||
if (this.maxSize && this.cache.size >= this.maxSize && !this.cache.has(key)) {
|
||||
// Remove oldest entry
|
||||
const firstKey = this.cache.keys().next().value;
|
||||
if (firstKey) this.cache.delete(firstKey);
|
||||
if (firstKey !== undefined) this.cache.delete(firstKey);
|
||||
}
|
||||
|
||||
const effectiveTtl = ttl ?? this.ttl;
|
||||
@@ -137,12 +137,15 @@ export class MemoryCache extends BaseCache {
|
||||
*/
|
||||
export class FileCache extends BaseCache {
|
||||
private cacheDir: string;
|
||||
private ttl?: number;
|
||||
private fs: any;
|
||||
private path: any;
|
||||
private listQueues: Map<string, Promise<void>> = new Map();
|
||||
|
||||
constructor(config: CacheConfig & { cacheDir?: string } = {}) {
|
||||
super(config);
|
||||
this.cacheDir = config.cacheDir || '.cache';
|
||||
this.ttl = config.ttl;
|
||||
}
|
||||
|
||||
private async ensureDir(): Promise<void> {
|
||||
@@ -177,10 +180,11 @@ export class FileCache extends BaseCache {
|
||||
|
||||
async set<T = any>(key: string, value: T, ttl?: number): Promise<void> {
|
||||
await this.ensureDir();
|
||||
const effectiveTtl = ttl ?? this.ttl;
|
||||
const entry: CacheEntry<T> = {
|
||||
value,
|
||||
createdAt: Date.now(),
|
||||
expiresAt: ttl ? Date.now() + ttl : undefined
|
||||
expiresAt: effectiveTtl ? Date.now() + effectiveTtl : undefined
|
||||
};
|
||||
await this.fs.writeFile(this.getFilePath(key), JSON.stringify(entry));
|
||||
}
|
||||
@@ -227,9 +231,20 @@ export class FileCache extends BaseCache {
|
||||
}
|
||||
|
||||
async listPush<T = any>(key: string, value: T): Promise<void> {
|
||||
const list = (await this.get<T[]>(`list_${key}`)) || [];
|
||||
list.push(value);
|
||||
await this.set(`list_${key}`, list);
|
||||
// Serialize the full read/modify/write operation for this instance and key.
|
||||
// A failed append must not block subsequent appends.
|
||||
const previous = this.listQueues.get(key) ?? Promise.resolve();
|
||||
const next = previous.catch(() => {}).then(async () => {
|
||||
const list = (await this.get<T[]>(`list_${key}`)) || [];
|
||||
list.push(value);
|
||||
await this.set(`list_${key}`, list);
|
||||
});
|
||||
this.listQueues.set(key, next);
|
||||
try {
|
||||
await next;
|
||||
} finally {
|
||||
if (this.listQueues.get(key) === next) this.listQueues.delete(key);
|
||||
}
|
||||
}
|
||||
|
||||
async listLength(key: string): Promise<number> {
|
||||
|
||||
@@ -231,12 +231,17 @@ export class HooksManager {
|
||||
return promise;
|
||||
}
|
||||
|
||||
return Promise.race([
|
||||
promise,
|
||||
new Promise<T>((_, reject) =>
|
||||
setTimeout(() => reject(new Error(`Hook timeout after ${timeout}ms`)), timeout)
|
||||
),
|
||||
]);
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
try {
|
||||
return await Promise.race([
|
||||
promise,
|
||||
new Promise<T>((_, reject) => {
|
||||
timer = setTimeout(() => reject(new Error(`Hook timeout after ${timeout}ms`)), timeout);
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
if (timer !== undefined) clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -251,7 +256,7 @@ export class HooksManager {
|
||||
* Get all registered hooks for an event
|
||||
*/
|
||||
getHooks(event: HookEvent): readonly HookConfig[] {
|
||||
return Object.freeze(this.hooks.get(event) ?? []);
|
||||
return Object.freeze([...(this.hooks.get(event) ?? [])]);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -82,16 +82,20 @@ export class Chunking {
|
||||
* Chunk by sentences
|
||||
*/
|
||||
chunkBySentence(text: string): Chunk[] {
|
||||
const sentences = text.match(/[^.!?]+[.!?]+/g) || [text];
|
||||
return sentences.map((content, index) => {
|
||||
const startOffset = text.indexOf(content);
|
||||
return {
|
||||
content: content.trim(),
|
||||
index,
|
||||
const chunks: Chunk[] = [];
|
||||
// Include the unterminated tail and punctuation-only spans as well.
|
||||
for (const match of text.matchAll(/[^.!?]+[.!?]*|[.!?]+/g)) {
|
||||
const content = match[0].trim();
|
||||
if (!content) continue;
|
||||
const startOffset = match.index + match[0].indexOf(content);
|
||||
chunks.push({
|
||||
content,
|
||||
index: chunks.length,
|
||||
startOffset,
|
||||
endOffset: startOffset + content.length
|
||||
};
|
||||
});
|
||||
});
|
||||
}
|
||||
return chunks;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -60,7 +60,23 @@ export class GraphStore {
|
||||
* Add an edge to the graph
|
||||
*/
|
||||
addEdge(edge: GraphEdge): void {
|
||||
const previous = this.edges.get(edge.id);
|
||||
this.edges.set(edge.id, edge);
|
||||
|
||||
if (previous && (previous.source !== edge.source || previous.target !== edge.target)) {
|
||||
// Another edge may still connect the old endpoints.
|
||||
let hasParallelEdge = false;
|
||||
for (const candidate of this.edges.values()) {
|
||||
if (candidate.source === previous.source && candidate.target === previous.target) {
|
||||
hasParallelEdge = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!hasParallelEdge) {
|
||||
this.adjacencyList.get(previous.source)?.delete(previous.target);
|
||||
this.reverseAdjacencyList.get(previous.target)?.delete(previous.source);
|
||||
}
|
||||
}
|
||||
|
||||
// Update adjacency lists
|
||||
if (!this.adjacencyList.has(edge.source)) {
|
||||
|
||||
@@ -70,22 +70,24 @@ export class WebSocketTransport implements MCPTransport {
|
||||
}
|
||||
const WebSocket = wsModule.default;
|
||||
|
||||
this.ws = new WebSocket(this.url);
|
||||
const socket = new WebSocket(this.url);
|
||||
this.ws = socket;
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
this.ws.on('open', () => {
|
||||
socket.on('open', () => {
|
||||
this.connected = true;
|
||||
resolve();
|
||||
});
|
||||
this.ws.on('error', (err: Error) => reject(err));
|
||||
socket.on('error', (err: Error) => reject(err));
|
||||
});
|
||||
|
||||
this.ws.on('message', (data: Buffer | string) => {
|
||||
socket.on('message', (data: Buffer | string) => {
|
||||
if (this.ws !== socket) return;
|
||||
try {
|
||||
const message = JSON.parse(data.toString()) as TransportMessage;
|
||||
|
||||
// Handle response
|
||||
if (message.id && this.pendingRequests.has(String(message.id))) {
|
||||
if (message.id !== undefined && this.pendingRequests.has(String(message.id))) {
|
||||
const { resolve, reject } = this.pendingRequests.get(String(message.id))!;
|
||||
this.pendingRequests.delete(String(message.id));
|
||||
if (message.error) {
|
||||
@@ -102,16 +104,29 @@ export class WebSocketTransport implements MCPTransport {
|
||||
}
|
||||
});
|
||||
|
||||
this.ws.on('error', (err: Error) => this.errorHandler?.(err));
|
||||
this.ws.on('close', () => { this.connected = false; });
|
||||
socket.on('error', (err: Error) => {
|
||||
if (this.ws === socket) this.errorHandler?.(err);
|
||||
});
|
||||
socket.on('close', () => {
|
||||
if (this.ws !== socket) return;
|
||||
this.connected = false;
|
||||
this.ws = null;
|
||||
this.rejectPendingRequests();
|
||||
});
|
||||
}
|
||||
|
||||
async disconnect(): Promise<void> {
|
||||
if (this.ws) {
|
||||
this.ws.close();
|
||||
this.ws = null;
|
||||
this.connected = false;
|
||||
}
|
||||
const socket = this.ws;
|
||||
this.ws = null;
|
||||
this.connected = false;
|
||||
this.rejectPendingRequests();
|
||||
socket?.close();
|
||||
}
|
||||
|
||||
private rejectPendingRequests(): void {
|
||||
const error = new Error('WebSocket connection closed');
|
||||
for (const { reject } of this.pendingRequests.values()) reject(error);
|
||||
this.pendingRequests.clear();
|
||||
}
|
||||
|
||||
async send(message: TransportMessage): Promise<TransportMessage> {
|
||||
|
||||
@@ -28,6 +28,7 @@ export class FileMemory {
|
||||
private initialized: boolean = false;
|
||||
private initPromise?: Promise<void>;
|
||||
private writeQueue: Promise<void> = Promise.resolve();
|
||||
private appendNeedsSeparator: boolean = false;
|
||||
|
||||
constructor(config: FileMemoryConfig) {
|
||||
this.filePath = config.filePath;
|
||||
@@ -67,6 +68,8 @@ export class FileMemory {
|
||||
try {
|
||||
const fs = await import('fs/promises');
|
||||
const content = await fs.readFile(this.filePath, 'utf-8').catch(() => '');
|
||||
// A previous instance may have left a record without its final newline.
|
||||
this.appendNeedsSeparator = content.length > 0 && !content.endsWith('\n');
|
||||
|
||||
if (content) {
|
||||
const lines = content.trim().split('\n').filter(Boolean);
|
||||
@@ -155,6 +158,7 @@ export class FileMemory {
|
||||
*/
|
||||
async getRecent(count: number): Promise<MemoryEntry[]> {
|
||||
const all = await this.getAll();
|
||||
if (count === 0) return [];
|
||||
return all.slice(-count);
|
||||
}
|
||||
|
||||
@@ -265,7 +269,7 @@ export class FileMemory {
|
||||
* Build context string from recent memory
|
||||
*/
|
||||
async buildContext(count?: number): Promise<string> {
|
||||
const entries = count ? await this.getRecent(count) : await this.getAll();
|
||||
const entries = count !== undefined ? await this.getRecent(count) : await this.getAll();
|
||||
return entries
|
||||
.map(e => `${e.role}: ${e.content}`)
|
||||
.join('\n');
|
||||
@@ -273,11 +277,17 @@ export class FileMemory {
|
||||
|
||||
private async appendEntry(entry: FileMemoryEntry): Promise<void> {
|
||||
// Queue writes to ensure order
|
||||
this.writeQueue = this.writeQueue.then(async () => {
|
||||
const write = this.writeQueue.then(async () => {
|
||||
const fs = await import('fs/promises');
|
||||
await fs.appendFile(this.filePath, JSON.stringify(entry) + '\n');
|
||||
// A failed append may have left an unfinished JSONL line.
|
||||
const separator = this.appendNeedsSeparator ? '\n' : '';
|
||||
this.appendNeedsSeparator = true;
|
||||
await fs.appendFile(this.filePath, separator + JSON.stringify(entry) + '\n');
|
||||
this.appendNeedsSeparator = false;
|
||||
});
|
||||
await this.writeQueue;
|
||||
// Report this write's error to its caller without blocking later writes.
|
||||
this.writeQueue = write.catch(() => {});
|
||||
await write;
|
||||
}
|
||||
|
||||
private textSearch(query: string, limit: number): SearchResult[] {
|
||||
|
||||
@@ -87,6 +87,7 @@ export class Memory {
|
||||
* Get recent entries
|
||||
*/
|
||||
getRecent(count: number): MemoryEntry[] {
|
||||
if (count === 0) return [];
|
||||
return this.getAll().slice(-count);
|
||||
}
|
||||
|
||||
@@ -179,7 +180,7 @@ export class Memory {
|
||||
* Build context string from recent memory
|
||||
*/
|
||||
buildContext(count?: number): string {
|
||||
const entries = count ? this.getRecent(count) : this.getAll();
|
||||
const entries = count !== undefined ? this.getRecent(count) : this.getAll();
|
||||
return entries
|
||||
.map(e => `${e.role}: ${e.content}`)
|
||||
.join('\n');
|
||||
|
||||
@@ -123,7 +123,8 @@ export class Session {
|
||||
private readonly timeout: number;
|
||||
private readonly isRemote: boolean;
|
||||
|
||||
private state: Record<string, any> = {};
|
||||
// State keys are user data, including names inherited by ordinary objects.
|
||||
private state: Record<string, any> = Object.create(null);
|
||||
private messages: SessionMessage[] = [];
|
||||
private createdAt: number;
|
||||
private updatedAt: number;
|
||||
@@ -264,7 +265,7 @@ export class Session {
|
||||
* Set entire state object.
|
||||
*/
|
||||
setState(state: Record<string, any>): void {
|
||||
this.state = { ...state };
|
||||
this.state = Object.assign(Object.create(null), state);
|
||||
this.updatedAt = Date.now();
|
||||
}
|
||||
|
||||
@@ -272,7 +273,7 @@ export class Session {
|
||||
* Clear all state.
|
||||
*/
|
||||
clearState(): void {
|
||||
this.state = {};
|
||||
this.state = Object.create(null);
|
||||
this.updatedAt = Date.now();
|
||||
}
|
||||
|
||||
@@ -364,7 +365,7 @@ export class Session {
|
||||
const messages = await this.db.getMessages(this.id, 1);
|
||||
if (messages.length > 0) {
|
||||
const sessionState: SessionState = JSON.parse(messages[0].content || '{}');
|
||||
this.state = sessionState.data || {};
|
||||
this.setState(sessionState.data || {});
|
||||
this.messages = sessionState.messages || [];
|
||||
this.createdAt = sessionState.createdAt || Date.now();
|
||||
this.updatedAt = sessionState.updatedAt || Date.now();
|
||||
@@ -382,7 +383,7 @@ export class Session {
|
||||
return {
|
||||
id: this.id,
|
||||
userId: this.userId,
|
||||
data: this.state,
|
||||
data: this.getState(),
|
||||
messages: this.messages,
|
||||
createdAt: this.createdAt,
|
||||
updatedAt: this.updatedAt,
|
||||
@@ -399,7 +400,7 @@ export class Session {
|
||||
userId: json.userId,
|
||||
...config,
|
||||
});
|
||||
session.state = json.data || {};
|
||||
session.setState(json.data || {});
|
||||
session.messages = json.messages || [];
|
||||
session.createdAt = json.createdAt;
|
||||
session.updatedAt = json.updatedAt;
|
||||
|
||||
@@ -111,6 +111,8 @@ export class FileSessionStore implements ISessionStore {
|
||||
private autoSaveInterval?: number;
|
||||
private saveTimer?: NodeJS.Timeout;
|
||||
private dirty: boolean = false;
|
||||
private saveQueue: Promise<void> = Promise.resolve();
|
||||
private pendingSaves: number = 0;
|
||||
|
||||
constructor(filePath: string, config?: { defaultTTL?: number; autoSaveInterval?: number }) {
|
||||
this.filePath = filePath;
|
||||
@@ -120,7 +122,7 @@ export class FileSessionStore implements ISessionStore {
|
||||
|
||||
if (this.autoSaveInterval) {
|
||||
this.saveTimer = setInterval(() => {
|
||||
if (this.dirty) this.save();
|
||||
if (this.dirty && this.pendingSaves === 0) this.save();
|
||||
}, this.autoSaveInterval);
|
||||
}
|
||||
}
|
||||
@@ -146,10 +148,26 @@ export class FileSessionStore implements ISessionStore {
|
||||
}
|
||||
|
||||
async save(): Promise<void> {
|
||||
const fs = await import('fs').then(m => m.promises);
|
||||
const sessions = Array.from(this.sessions.values());
|
||||
await fs.writeFile(this.filePath, JSON.stringify(sessions, null, 2));
|
||||
this.dirty = false;
|
||||
this.pendingSaves++;
|
||||
const saving = this.saveQueue.then(async () => {
|
||||
const fs = await import('fs').then(m => m.promises);
|
||||
const data = JSON.stringify(Array.from(this.sessions.values()), null, 2);
|
||||
// Only this snapshot is clean; mutations during the write stay dirty.
|
||||
this.dirty = false;
|
||||
try {
|
||||
await fs.writeFile(this.filePath, data);
|
||||
} catch (error) {
|
||||
this.dirty = true;
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
// Keep the queue usable after a failure, while this caller still sees it.
|
||||
this.saveQueue = saving.catch(() => {});
|
||||
try {
|
||||
await saving;
|
||||
} finally {
|
||||
this.pendingSaves--;
|
||||
}
|
||||
}
|
||||
|
||||
async get(id: string): Promise<StoredSession | null> {
|
||||
@@ -205,6 +223,7 @@ export class FileSessionStore implements ISessionStore {
|
||||
if (this.saveTimer) {
|
||||
clearInterval(this.saveTimer);
|
||||
}
|
||||
await this.saveQueue;
|
||||
if (this.dirty) {
|
||||
await this.save();
|
||||
}
|
||||
|
||||
@@ -420,7 +420,21 @@ export class FunctionTool<TParams = any, TResult = any> {
|
||||
}
|
||||
const delay = retryDelayMs(policy, attempt - 1);
|
||||
if (delay > 0) {
|
||||
await new Promise<void>(resolve => setTimeout(resolve, delay));
|
||||
const signal = context?.signal;
|
||||
const completed = await new Promise<boolean>(resolve => {
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
const finish = (value: boolean) => {
|
||||
if (timer !== undefined) clearTimeout(timer);
|
||||
signal?.removeEventListener('abort', onAbort);
|
||||
resolve(value);
|
||||
};
|
||||
const onAbort = () => finish(false);
|
||||
timer = setTimeout(() => finish(true), delay);
|
||||
signal?.addEventListener('abort', onAbort, { once: true });
|
||||
if (signal?.aborted) finish(false);
|
||||
});
|
||||
// Preserve the existing last-error behavior when cancellation stops retries.
|
||||
if (!completed || signal?.aborted) throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -147,6 +147,9 @@ export function createRetryMiddleware(
|
||||
let lastError: Error | undefined;
|
||||
|
||||
for (let attempt = 0; attempt <= maxRetries; attempt++) {
|
||||
if (context.signal?.aborted) {
|
||||
throw context.signal.reason ?? new Error('The operation was aborted');
|
||||
}
|
||||
try {
|
||||
return await next();
|
||||
} catch (error) {
|
||||
@@ -212,7 +215,9 @@ export function createValidationMiddleware(): ToolMiddleware {
|
||||
const limits = context.limits;
|
||||
|
||||
if (limits?.maxPayloadBytes) {
|
||||
const size = JSON.stringify(input).length;
|
||||
const serialized = JSON.stringify(input);
|
||||
if (serialized === undefined) throw new TypeError('Input payload is not JSON serializable');
|
||||
const size = new TextEncoder().encode(serialized).byteLength;
|
||||
if (size > limits.maxPayloadBytes) {
|
||||
throw new Error(`Input payload size (${size} bytes) exceeds limit (${limits.maxPayloadBytes} bytes)`);
|
||||
}
|
||||
@@ -227,17 +232,16 @@ export function createValidationMiddleware(): ToolMiddleware {
|
||||
*/
|
||||
export function composeMiddleware(...middlewares: ToolMiddleware[]): ToolMiddleware {
|
||||
return async (input, context, next) => {
|
||||
let index = 0;
|
||||
|
||||
const composedNext = async (): Promise<unknown> => {
|
||||
// Each continuation keeps its own downstream position so retry middleware
|
||||
// can invoke next() again without skipping the remaining layers.
|
||||
const dispatch = async (index: number): Promise<unknown> => {
|
||||
if (index < middlewares.length) {
|
||||
const mw = middlewares[index++];
|
||||
return mw(input, context, composedNext);
|
||||
return middlewares[index](input, context, () => dispatch(index + 1));
|
||||
}
|
||||
return next();
|
||||
};
|
||||
|
||||
return composedNext();
|
||||
return dispatch(0);
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,8 @@
|
||||
* Supports lazy loading, middleware, and multi-agent safety.
|
||||
*/
|
||||
|
||||
import { composeMiddleware } from './middleware';
|
||||
|
||||
import type {
|
||||
ToolMetadata,
|
||||
ToolFactory,
|
||||
@@ -61,6 +63,7 @@ export class ToolsRegistry {
|
||||
*/
|
||||
register(metadata: ToolMetadata, factory: ToolFactory): this {
|
||||
this.tools.set(metadata.id, { metadata, factory });
|
||||
this.installCache.delete(metadata.id);
|
||||
return this;
|
||||
}
|
||||
|
||||
@@ -68,6 +71,7 @@ export class ToolsRegistry {
|
||||
* Unregister a tool
|
||||
*/
|
||||
unregister(id: string): boolean {
|
||||
this.installCache.delete(id);
|
||||
return this.tools.delete(id);
|
||||
}
|
||||
|
||||
@@ -158,16 +162,7 @@ export class ToolsRegistry {
|
||||
if (this.middleware.length === 0) {
|
||||
result = await executeWithMiddleware();
|
||||
} else {
|
||||
// Chain middleware
|
||||
let index = 0;
|
||||
const next = async (): Promise<unknown> => {
|
||||
if (index < this.middleware.length) {
|
||||
const mw = this.middleware[index++];
|
||||
return mw(input, ctx, next);
|
||||
}
|
||||
return executeWithMiddleware();
|
||||
};
|
||||
result = await next();
|
||||
result = await composeMiddleware(...this.middleware)(input, ctx, executeWithMiddleware);
|
||||
}
|
||||
|
||||
// Call afterToolCall hook
|
||||
@@ -222,27 +217,25 @@ export class ToolsRegistry {
|
||||
* Check if a tool's optional dependency is installed
|
||||
*/
|
||||
async checkInstalled(id: string): Promise<boolean> {
|
||||
// Check cache first
|
||||
const registered = this.tools.get(id);
|
||||
if (!registered) return false;
|
||||
|
||||
if (this.installCache.has(id)) {
|
||||
return this.installCache.get(id)!;
|
||||
}
|
||||
|
||||
const registered = this.tools.get(id);
|
||||
if (!registered) {
|
||||
return false;
|
||||
const { packageName } = registered.metadata;
|
||||
let installed: boolean;
|
||||
try {
|
||||
await import(packageName);
|
||||
installed = true;
|
||||
} catch {
|
||||
installed = false;
|
||||
}
|
||||
|
||||
const { packageName } = registered.metadata;
|
||||
|
||||
try {
|
||||
// Try to resolve the package
|
||||
await import(packageName);
|
||||
this.installCache.set(id, true);
|
||||
return true;
|
||||
} catch {
|
||||
this.installCache.set(id, false);
|
||||
return false;
|
||||
}
|
||||
// An earlier import must not repopulate the cache after replacement/removal.
|
||||
if (this.tools.get(id) === registered) this.installCache.set(id, installed);
|
||||
return installed;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
+86
@@ -3,6 +3,9 @@
|
||||
*/
|
||||
|
||||
import { MemoryCache, FileCache, createMemoryCache, createFileCache } from '../../../src/cache';
|
||||
import { mkdtemp, rm } from 'fs/promises';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
describe('MemoryCache', () => {
|
||||
let cache: MemoryCache;
|
||||
@@ -158,3 +161,86 @@ describe('Factory functions', () => {
|
||||
expect(cache).toBeInstanceOf(FileCache);
|
||||
});
|
||||
});
|
||||
|
||||
describe('FileCache default TTL', () => {
|
||||
let cacheDir: string;
|
||||
let now: number;
|
||||
|
||||
beforeEach(async () => {
|
||||
cacheDir = await mkdtemp(join(tmpdir(), 'praisonai-file-cache-ttl-'));
|
||||
now = 1000;
|
||||
jest.spyOn(Date, 'now').mockImplementation(() => now);
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
jest.restoreAllMocks();
|
||||
await rm(cacheDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it.each(['get', 'has'] as const)('honors the configured TTL through %s after reopening', async (method) => {
|
||||
const cache = createFileCache({ cacheDir, ttl: 50 });
|
||||
await cache.set('answer', 'old answer');
|
||||
expect(await cache.get('answer')).toBe('old answer');
|
||||
|
||||
const reopened = new FileCache({ cacheDir });
|
||||
now += 51;
|
||||
if (method === 'get') {
|
||||
expect(await reopened.get('answer')).toBeUndefined();
|
||||
} else {
|
||||
expect(await reopened.has('answer')).toBe(false);
|
||||
}
|
||||
expect(await reopened.keys()).not.toContain('answer');
|
||||
});
|
||||
|
||||
it('allows a per-entry TTL to override the configured TTL', async () => {
|
||||
const cache = new FileCache({ cacheDir, ttl: 50 });
|
||||
await cache.set('answer', 'value', 100);
|
||||
now += 51;
|
||||
expect(await cache.get('answer')).toBe('value');
|
||||
now += 50;
|
||||
expect(await cache.get('answer')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('allows an explicit zero TTL to disable the configured expiry', async () => {
|
||||
const cache = new FileCache({ cacheDir, ttl: 50 });
|
||||
await cache.set('answer', 'value', 0);
|
||||
now += 1000;
|
||||
expect(await cache.get('answer')).toBe('value');
|
||||
});
|
||||
|
||||
it('keeps entries without a default or explicit TTL', async () => {
|
||||
const cache = new FileCache({ cacheDir });
|
||||
await cache.set('answer', 'value');
|
||||
now += 1000;
|
||||
expect(await cache.get('answer')).toBe('value');
|
||||
});
|
||||
});
|
||||
|
||||
describe('MemoryCache capacity with empty keys', () => {
|
||||
it.each([1, 2])('keeps capacity %i bounded when the oldest key is empty', async (maxSize) => {
|
||||
const cache = new MemoryCache({ maxSize });
|
||||
await cache.set('', 'oldest');
|
||||
for (let i = 0; i <= maxSize; i++) {
|
||||
await cache.set(`key-${i}`, i);
|
||||
}
|
||||
|
||||
expect(await cache.size()).toBe(maxSize);
|
||||
expect(await cache.has('')).toBe(false);
|
||||
expect(await cache.keys()).toEqual(
|
||||
Array.from({ length: maxSize }, (_, i) => `key-${i + 1}`)
|
||||
);
|
||||
expect(await cache.get(`key-${maxSize}`)).toBe(maxSize);
|
||||
});
|
||||
|
||||
it('updates an existing empty key without evicting another entry', async () => {
|
||||
const cache = new MemoryCache({ maxSize: 2 });
|
||||
await cache.set('', 'old value');
|
||||
await cache.set('other', 'retained');
|
||||
await cache.set('', 'new value');
|
||||
|
||||
expect(await cache.size()).toBe(2);
|
||||
expect(await cache.keys()).toEqual(['', 'other']);
|
||||
expect(await cache.get('')).toBe('new value');
|
||||
expect(await cache.get('other')).toBe('retained');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,70 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
|
||||
import { mkdtemp, mkdir, rm } from 'fs/promises';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { createFileCache } from '../../../src/cache';
|
||||
|
||||
describe('FileCache concurrent list appends', () => {
|
||||
let directory: string;
|
||||
beforeEach(async () => { directory = await mkdtemp(join(tmpdir(), 'praison-list-')); });
|
||||
afterEach(async () => { await rm(directory, { recursive: true, force: true }); });
|
||||
|
||||
it('persists every successful concurrent append in call order', async () => {
|
||||
const cache = createFileCache({ cacheDir: directory });
|
||||
await cache.listPush('events', 'existing');
|
||||
const values = Array.from({ length: 25 }, (_, index) => `event-${index}`);
|
||||
await Promise.all(values.map(value => cache.listPush('events', value)));
|
||||
const reopened = createFileCache({ cacheDir: directory });
|
||||
expect(await reopened.listRange('events', 0)).toEqual(['existing', ...values]);
|
||||
expect(await reopened.listLength('events')).toBe(26);
|
||||
});
|
||||
|
||||
it('retains each list when different keys receive concurrent appends', async () => {
|
||||
const cache = createFileCache({ cacheDir: directory });
|
||||
await cache.keys();
|
||||
const values = Array.from({ length: 12 }, (_, index) => index + 1);
|
||||
await Promise.all(values.flatMap(value => [cache.listPush('left', value), cache.listPush('right', -value)]));
|
||||
const reopened = createFileCache({ cacheDir: directory });
|
||||
expect(await reopened.listRange('left', 0)).toEqual(values);
|
||||
expect(await reopened.listRange('right', 0)).toEqual(values.map(value => -value));
|
||||
});
|
||||
|
||||
it('recovers after a failed append without persisting that value', async () => {
|
||||
const cache = createFileCache({ cacheDir: directory });
|
||||
const blockedFile = join(directory, 'list_events.json');
|
||||
await mkdir(blockedFile);
|
||||
await expect(cache.listPush('events', 'failed')).rejects.toThrow();
|
||||
await rm(blockedFile, { recursive: true });
|
||||
await cache.listPush('events', 'retry');
|
||||
expect(await cache.listRange('events', 0)).toEqual(['retry']);
|
||||
});
|
||||
|
||||
it('continues an already queued append after a transient write failure', async () => {
|
||||
const cache = createFileCache({ cacheDir: directory });
|
||||
let releaseFailure!: () => void;
|
||||
let signalWrite!: () => void;
|
||||
const blocked = new Promise<void>(resolve => { releaseFailure = resolve; });
|
||||
const writing = new Promise<void>(resolve => { signalWrite = resolve; });
|
||||
// Inject one failed public write; the queued retry uses the real filesystem.
|
||||
jest.spyOn(cache, 'set').mockImplementationOnce(async () => {
|
||||
signalWrite();
|
||||
await blocked;
|
||||
throw new Error('transient write failure');
|
||||
});
|
||||
const failed = expect(cache.listPush('events', 'failed')).rejects.toThrow('transient write failure');
|
||||
const retry = cache.listPush('events', 'retry');
|
||||
await writing;
|
||||
releaseFailure();
|
||||
await failed;
|
||||
await retry;
|
||||
const reopened = createFileCache({ cacheDir: directory });
|
||||
expect(await reopened.listRange('events', 0)).toEqual(['retry']);
|
||||
});
|
||||
|
||||
it('preserves sequential appends and list slicing', async () => {
|
||||
const cache = createFileCache({ cacheDir: directory });
|
||||
for (const value of [1, 2, 3]) await cache.listPush('events', value);
|
||||
expect(await cache.listRange('events', 1, 2)).toEqual([2]);
|
||||
expect(await cache.listRange('events', -1)).toEqual([3]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,107 @@
|
||||
import { promises as fs } from 'fs';
|
||||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { FileMemory } from '../../src/memory/file-memory';
|
||||
|
||||
describe('FileMemory write recovery', () => {
|
||||
let tempDir: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'praisonai-write-recovery-'));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
jest.restoreAllMocks();
|
||||
await fs.rm(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('persists later writes after the filesystem becomes available again', async () => {
|
||||
const directory = path.join(tempDir, 'store');
|
||||
const unavailableDirectory = path.join(tempDir, 'unavailable');
|
||||
await fs.mkdir(directory);
|
||||
const filePath = path.join(directory, 'memory.jsonl');
|
||||
const memory = new FileMemory({ filePath });
|
||||
const initial = await memory.add('Before interruption', 'user');
|
||||
|
||||
await fs.rename(directory, unavailableDirectory);
|
||||
await expect(memory.add('During interruption', 'user')).rejects.toMatchObject({ code: 'ENOENT' });
|
||||
await fs.rename(unavailableDirectory, directory);
|
||||
|
||||
const recovered = await memory.add('After recovery', 'assistant');
|
||||
const reopened = new FileMemory({ filePath });
|
||||
expect(await reopened.getAll()).toEqual([initial, recovered]);
|
||||
});
|
||||
|
||||
test('preserves each write result and continues queued writes after a rejection', async () => {
|
||||
const filePath = path.join(tempDir, 'memory.jsonl');
|
||||
const memory = new FileMemory({ filePath });
|
||||
const initial = await memory.add('Before interruption', 'user');
|
||||
const appendFile = fs.appendFile;
|
||||
const failure = new Error('Temporary append failure');
|
||||
let releaseFailure!: () => void;
|
||||
let notifyAppend!: () => void;
|
||||
const blocked = new Promise<void>(resolve => { releaseFailure = resolve; });
|
||||
const started = new Promise<void>(resolve => { notifyAppend = resolve; });
|
||||
jest.spyOn(fs, 'appendFile').mockImplementationOnce(async () => {
|
||||
notifyAppend();
|
||||
await blocked;
|
||||
throw failure;
|
||||
}).mockImplementation(appendFile);
|
||||
|
||||
const failed = memory.add('Failed write', 'user');
|
||||
const queued = memory.add('Queued write', 'assistant');
|
||||
const settled = Promise.allSettled([failed, queued]);
|
||||
await started;
|
||||
releaseFailure();
|
||||
const [first, second] = await settled;
|
||||
|
||||
expect(first).toEqual({ status: 'rejected', reason: failure });
|
||||
expect(second.status).toBe('fulfilled');
|
||||
if (second.status !== 'fulfilled') throw second.reason;
|
||||
const later = await memory.add('Later write', 'user');
|
||||
const reopened = new FileMemory({ filePath });
|
||||
expect(await reopened.getAll()).toEqual([initial, second.value, later]);
|
||||
});
|
||||
|
||||
test.each([1, 2])('preserves later records after %i partial append failures', async (failures) => {
|
||||
const filePath = path.join(tempDir, 'memory.jsonl');
|
||||
const memory = new FileMemory({ filePath });
|
||||
const initial = await memory.add('Before interruption', 'user');
|
||||
const appendFile = fs.appendFile;
|
||||
const failure = Object.assign(new Error('Disk full after a partial write'), { code: 'ENOSPC' });
|
||||
let attempts = 0;
|
||||
jest.spyOn(fs, 'appendFile').mockImplementation(async (file, data, options) => {
|
||||
if (attempts++ < failures) {
|
||||
// Write an actual unterminated JSON prefix before reporting failure.
|
||||
await appendFile(file, String(data).slice(0, 20), options);
|
||||
throw failure;
|
||||
}
|
||||
await appendFile(file, data, options);
|
||||
});
|
||||
|
||||
for (let attempt = 0; attempt < failures; attempt++) {
|
||||
await expect(memory.add('Interrupted write', 'user')).rejects.toBe(failure);
|
||||
}
|
||||
const recovered = await memory.add('Recovered write', 'assistant');
|
||||
const later = await memory.add('Later write', 'user');
|
||||
const reopened = new FileMemory({ filePath });
|
||||
expect(await reopened.getAll()).toEqual([initial, recovered, later]);
|
||||
});
|
||||
|
||||
test.each([
|
||||
['incomplete JSON', false],
|
||||
['complete JSON', true],
|
||||
] as const)('preserves new records after loading an unterminated %s line', async (_, complete) => {
|
||||
const filePath = path.join(tempDir, 'memory.jsonl');
|
||||
const original = new FileMemory({ filePath });
|
||||
const initial = await original.add('Before restart', 'user');
|
||||
const last = { id: 'last-record', content: 'Last record', role: 'user', timestamp: initial.timestamp };
|
||||
const tail = complete ? JSON.stringify(last) : '{"id":"interrupted","content":';
|
||||
await fs.appendFile(filePath, tail);
|
||||
|
||||
const restarted = new FileMemory({ filePath });
|
||||
const recovered = await restarted.add('After restart', 'assistant');
|
||||
const reopened = new FileMemory({ filePath });
|
||||
expect(await reopened.getAll()).toEqual(complete ? [initial, last, recovered] : [initial, recovered]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,47 @@
|
||||
import { describe, expect, it } from '@jest/globals';
|
||||
import { HooksManager } from '../../src/hooks/manager';
|
||||
|
||||
describe('HooksManager inspection', () => {
|
||||
it('allows registering and executing hooks after inspection', async () => {
|
||||
const manager = new HooksManager();
|
||||
const called: string[] = [];
|
||||
const first = manager.register('pre_tool_call', () => { called.push('first'); });
|
||||
const snapshot = manager.getHooks('pre_tool_call');
|
||||
const second = manager.register('pre_tool_call', () => { called.push('second'); }, { priority: 1 });
|
||||
expect(snapshot.map(hook => hook.id)).toEqual([first]);
|
||||
expect(manager.getHooks('pre_tool_call').map(hook => hook.id)).toEqual([second, first]);
|
||||
expect((await manager.execute('pre_tool_call', {})).success).toBe(true);
|
||||
expect(called).toEqual(['second', 'first']);
|
||||
});
|
||||
|
||||
it('allows unregistering a hook after inspection', async () => {
|
||||
const manager = new HooksManager();
|
||||
const called: string[] = [];
|
||||
const first = manager.register('pre_tool_call', () => { called.push('first'); });
|
||||
const second = manager.register('pre_tool_call', () => { called.push('second'); });
|
||||
const snapshot = manager.getHooks('pre_tool_call');
|
||||
expect(manager.unregister(first)).toBe(true);
|
||||
expect(snapshot.map(hook => hook.id)).toEqual([first, second]);
|
||||
expect(manager.getStats().totalHooks).toBe(1);
|
||||
await manager.execute('pre_tool_call', {});
|
||||
expect(called).toEqual(['second']);
|
||||
});
|
||||
|
||||
it('keeps returned arrays frozen against caller mutations', () => {
|
||||
const manager = new HooksManager();
|
||||
const id = manager.register('pre_tool_call', () => {});
|
||||
const snapshot = manager.getHooks('pre_tool_call');
|
||||
expect(Object.isFrozen(snapshot)).toBe(true);
|
||||
expect(() => (snapshot as unknown[]).pop()).toThrow(TypeError);
|
||||
expect(manager.getHooks('pre_tool_call').map(hook => hook.id)).toEqual([id]);
|
||||
});
|
||||
|
||||
it('allows first registration after inspecting an empty event', () => {
|
||||
const manager = new HooksManager();
|
||||
const snapshot = manager.getHooks('pre_tool_call');
|
||||
const id = manager.register('pre_tool_call', () => {});
|
||||
expect(snapshot).toEqual([]);
|
||||
expect(Object.isFrozen(snapshot)).toBe(true);
|
||||
expect(manager.getHooks('pre_tool_call').map(hook => hook.id)).toEqual([id]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,59 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
|
||||
import { spawnSync } from 'child_process';
|
||||
import { resolve } from 'path';
|
||||
import { HooksManager } from '../../src/hooks/manager';
|
||||
|
||||
describe('HooksManager timeout lifecycle', () => {
|
||||
beforeEach(() => { jest.useFakeTimers(); });
|
||||
afterEach(() => { jest.clearAllTimers(); jest.useRealTimers(); });
|
||||
|
||||
it('releases the deadline after a successful asynchronous hook', async () => {
|
||||
const hooks = new HooksManager({ timeout: 30000 });
|
||||
hooks.register('pre_tool_call', async (context) => ({ ...context, checked: true }));
|
||||
const result = await hooks.execute('pre_tool_call', { input: 'test' });
|
||||
expect(result.modifiedContext).toEqual({ input: 'test', checked: true });
|
||||
expect(result.blocked).toBe(false);
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('releases the deadline after an asynchronous hook rejects', async () => {
|
||||
const hooks = new HooksManager({ timeout: 30000 });
|
||||
hooks.register('pre_tool_call', async () => { throw new Error('validation failed'); }, { blockOnFailure: true });
|
||||
const result = await hooks.execute('pre_tool_call', {});
|
||||
expect(result).toMatchObject({ success: false, blocked: true, error: 'validation failed' });
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('still times out a pending hook and preserves blocking policy', async () => {
|
||||
const hooks = new HooksManager({ timeout: 10 });
|
||||
hooks.register('pre_tool_call', () => new Promise(() => {}), { blockOnFailure: true });
|
||||
const execution = hooks.execute('pre_tool_call', {});
|
||||
expect(jest.getTimerCount()).toBe(1);
|
||||
await jest.advanceTimersByTimeAsync(10);
|
||||
expect(await execution).toMatchObject({ success: false, blocked: true, error: 'Hook timeout after 10ms' });
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('does not allocate a deadline for a synchronous hook', async () => {
|
||||
const hooks = new HooksManager();
|
||||
hooks.register('pre_tool_call', () => ({ checked: true }));
|
||||
expect((await hooks.execute('pre_tool_call', {})).modifiedContext).toEqual({ checked: true });
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('lets a real process exit naturally after its async hook settles', () => {
|
||||
const script = `
|
||||
const { HooksManager } = require(${JSON.stringify(resolve(__dirname, '../../src/hooks/manager'))});
|
||||
const hooks = new HooksManager({ timeout: 30000 });
|
||||
hooks.register('pre_tool_call', async context => context);
|
||||
hooks.execute('pre_tool_call', {}).then(() => console.log('hook settled')).catch(error => { console.error(error); process.exitCode = 1; });
|
||||
`;
|
||||
const result = spawnSync(process.execPath, ['-r', 'ts-node/register/transpile-only', '-e', script], {
|
||||
timeout: 10000,
|
||||
encoding: 'utf-8',
|
||||
});
|
||||
expect(result.stdout).toContain('hook settled');
|
||||
expect(result.error).toBeUndefined();
|
||||
expect({ status: result.status, stderr: result.stderr }).toEqual({ status: 0, stderr: '' });
|
||||
}, 15000);
|
||||
});
|
||||
@@ -30,6 +30,34 @@ describe('Chunking', () => {
|
||||
expect(chunks.length).toBe(3);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'First sentence. Final important fact without punctuation',
|
||||
'Echo.Echo.Echo.',
|
||||
' \nFirst. Second! \nTail \t',
|
||||
'... Leading punctuation. Tail',
|
||||
'No punctuation here',
|
||||
'Really?! Next.',
|
||||
'你好。完整段落',
|
||||
])('preserves sentence content and its source span: %j', async text => {
|
||||
const { Chunking } = await import('../../../src/knowledge/chunking');
|
||||
const chunks = new Chunking({ strategy: 'sentence' }).chunk(text);
|
||||
let cursor = 0;
|
||||
for (const [index, chunk] of chunks.entries()) {
|
||||
expect(chunk.index).toBe(index);
|
||||
expect(chunk.content.length).toBeGreaterThan(0);
|
||||
expect(chunk.startOffset).toBeGreaterThanOrEqual(cursor);
|
||||
expect(text.slice(cursor, chunk.startOffset).trim()).toBe('');
|
||||
expect(text.slice(chunk.startOffset, chunk.endOffset)).toBe(chunk.content);
|
||||
cursor = chunk.endOffset;
|
||||
}
|
||||
expect(text.slice(cursor).trim()).toBe('');
|
||||
});
|
||||
|
||||
it.each(['', ' \n\t '])('does not emit empty sentence chunks for %j', async text => {
|
||||
const { Chunking } = await import('../../../src/knowledge/chunking');
|
||||
expect(new Chunking({ strategy: 'sentence' }).chunk(text)).toEqual([]);
|
||||
});
|
||||
|
||||
it('should chunk by paragraphs', async () => {
|
||||
const { Chunking } = await import('../../../src/knowledge/chunking');
|
||||
const chunker = new Chunking({ strategy: 'paragraph' });
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import { describe, expect, it } from '@jest/globals';
|
||||
import { GraphStore } from '../../../src/knowledge/graph-rag';
|
||||
|
||||
function graph(): GraphStore {
|
||||
const store = new GraphStore();
|
||||
for (const id of ['a', 'b', 'c', 'd']) store.addNode({ id, type: 'document', content: id });
|
||||
return store;
|
||||
}
|
||||
|
||||
describe('GraphStore edge replacement', () => {
|
||||
it.each([
|
||||
['a', 'c'], ['c', 'b'], ['c', 'd'], ['a', 'a'],
|
||||
])('removes obsolete adjacency when an edge moves to %s -> %s', (source, target) => {
|
||||
const store = graph();
|
||||
store.addEdge({ id: 'edge', source: 'a', target: 'b', type: 'link' });
|
||||
store.addEdge({ id: 'edge', source, target, type: 'link' });
|
||||
expect(store.getNeighbors('a', 'outgoing').map(n => n.id)).not.toContain('b');
|
||||
expect(store.getNeighbors('b', 'incoming').map(n => n.id)).not.toContain('a');
|
||||
expect(store.getEdgesBetween('a', 'b')).toEqual([]);
|
||||
expect(store.findPaths('a', 'b')).toEqual([]);
|
||||
expect(store.getNeighbors(source, 'outgoing').map(n => n.id)).toContain(target);
|
||||
expect(store.getNeighbors(target, 'incoming').map(n => n.id)).toContain(source);
|
||||
expect(store.getStats().edgeCount).toBe(1);
|
||||
});
|
||||
|
||||
it('retains adjacency backed by a parallel edge', () => {
|
||||
const store = graph();
|
||||
store.addEdge({ id: 'one', source: 'a', target: 'b', type: 'one' });
|
||||
store.addEdge({ id: 'two', source: 'a', target: 'b', type: 'two' });
|
||||
store.addEdge({ id: 'one', source: 'c', target: 'd', type: 'one' });
|
||||
expect(store.getNeighbors('a', 'outgoing').map(n => n.id)).toEqual(['b']);
|
||||
expect(store.getNeighbors('b', 'incoming').map(n => n.id)).toEqual(['a']);
|
||||
expect(store.getEdgesBetween('a', 'b').map(e => e.id)).toEqual(['two']);
|
||||
store.addEdge({ id: 'two', source: 'c', target: 'd', type: 'two' });
|
||||
expect(store.getNeighbors('a', 'outgoing')).toEqual([]);
|
||||
expect(store.getNeighbors('b', 'incoming')).toEqual([]);
|
||||
expect(store.getStats().edgeCount).toBe(2);
|
||||
});
|
||||
|
||||
it('preserves neighbors when only edge metadata changes', () => {
|
||||
const store = graph();
|
||||
store.addEdge({ id: 'edge', source: 'a', target: 'b', type: 'link', weight: 1 });
|
||||
store.addEdge({ id: 'edge', source: 'a', target: 'b', type: 'link', weight: 2 });
|
||||
expect(store.getNeighbors('a', 'outgoing').map(n => n.id)).toEqual(['b']);
|
||||
expect(store.getEdgesBetween('a', 'b')[0].weight).toBe(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,93 @@
|
||||
import { EventEmitter } from 'events';
|
||||
import { WebSocketTransport } from '../../src/mcp/transports';
|
||||
|
||||
class MockWebSocket extends EventEmitter {
|
||||
static last: MockWebSocket;
|
||||
send = jest.fn();
|
||||
// Real close handshakes finish asynchronously; tests emit close explicitly.
|
||||
close = jest.fn();
|
||||
|
||||
constructor(_url: string) {
|
||||
super();
|
||||
MockWebSocket.last = this;
|
||||
queueMicrotask(() => this.emit('open'));
|
||||
}
|
||||
}
|
||||
|
||||
jest.mock('ws', () => ({ __esModule: true, default: MockWebSocket }), { virtual: true });
|
||||
|
||||
async function boundedOutcome(pending: Promise<unknown>): Promise<unknown> {
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
try {
|
||||
return await Promise.race([
|
||||
pending.then(() => 'resolved', error => error),
|
||||
new Promise(resolve => { timer = setTimeout(() => resolve('pending'), 100); }),
|
||||
]);
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
const response = (source: string) => JSON.stringify({ jsonrpc: '2.0', id: 1, result: { source } });
|
||||
|
||||
describe('WebSocketTransport connection lifecycle', () => {
|
||||
let transport: WebSocketTransport;
|
||||
|
||||
beforeEach(async () => {
|
||||
transport = new WebSocketTransport('ws://localhost');
|
||||
await transport.connect();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await transport.disconnect();
|
||||
});
|
||||
|
||||
it.each(['remote', 'local'])('rejects every pending request on %s close', async (mode) => {
|
||||
const outcomes = [1, 2].map(id => boundedOutcome(
|
||||
transport.send({ jsonrpc: '2.0', id, method: 'tools/list' })
|
||||
));
|
||||
if (mode === 'remote') MockWebSocket.last.emit('close');
|
||||
else await transport.disconnect();
|
||||
|
||||
expect(transport.isConnected()).toBe(false);
|
||||
for (const outcome of await Promise.all(outcomes)) {
|
||||
expect(outcome).toBeInstanceOf(Error);
|
||||
expect(outcome).toEqual(expect.objectContaining({
|
||||
message: expect.stringMatching(/closed|disconnect/i),
|
||||
}));
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps a replacement connection active when the old close event arrives', async () => {
|
||||
const old = MockWebSocket.last;
|
||||
await transport.disconnect();
|
||||
await transport.connect();
|
||||
old.emit('close');
|
||||
expect(transport.isConnected()).toBe(true);
|
||||
|
||||
const pending = transport.send({ jsonrpc: '2.0', id: 1, method: 'tools/list' });
|
||||
MockWebSocket.last.emit('message', response('new'));
|
||||
await expect(pending).resolves.toMatchObject({ result: { source: 'new' } });
|
||||
});
|
||||
|
||||
it('ignores a closing connection response when a new request reuses its ID', async () => {
|
||||
const old = MockWebSocket.last;
|
||||
await transport.disconnect();
|
||||
await transport.connect();
|
||||
const received = jest.fn();
|
||||
transport.onMessage(received);
|
||||
const pending = transport.send({ jsonrpc: '2.0', id: 1, method: 'tools/list' });
|
||||
old.emit('message', response('old'));
|
||||
MockWebSocket.last.emit('message', response('new'));
|
||||
|
||||
await expect(pending).resolves.toMatchObject({ result: { source: 'new' } });
|
||||
expect(received).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('keeps a completed response available after its connection closes', async () => {
|
||||
const pending = transport.send({ jsonrpc: '2.0', id: 1, method: 'tools/list' });
|
||||
MockWebSocket.last.emit('message', response('complete'));
|
||||
MockWebSocket.last.emit('close');
|
||||
await expect(pending).resolves.toMatchObject({ result: { source: 'complete' } });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,55 @@
|
||||
import { EventEmitter } from 'events';
|
||||
import { WebSocketTransport } from '../../src/mcp/transports';
|
||||
|
||||
class MockWebSocket extends EventEmitter {
|
||||
static last: MockWebSocket;
|
||||
send = jest.fn();
|
||||
|
||||
constructor(_url: string) {
|
||||
super();
|
||||
MockWebSocket.last = this;
|
||||
queueMicrotask(() => this.emit('open'));
|
||||
}
|
||||
|
||||
close(): void {
|
||||
this.emit('close');
|
||||
}
|
||||
}
|
||||
|
||||
jest.mock('ws', () => ({ __esModule: true, default: MockWebSocket }), { virtual: true });
|
||||
|
||||
describe('WebSocketTransport response correlation', () => {
|
||||
let transport: WebSocketTransport;
|
||||
|
||||
beforeEach(async () => {
|
||||
transport = new WebSocketTransport('ws://localhost');
|
||||
await transport.connect();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await transport.disconnect();
|
||||
});
|
||||
|
||||
it.each([0, 1, 'request-1'])('resolves the request with id %p', async (id) => {
|
||||
const response = { jsonrpc: '2.0' as const, id, result: { tools: [] } };
|
||||
const received = jest.fn();
|
||||
transport.onMessage(received);
|
||||
const pending = transport.send({ jsonrpc: '2.0', id, method: 'tools/list' });
|
||||
|
||||
MockWebSocket.last.emit('message', Buffer.from(JSON.stringify(response)));
|
||||
|
||||
await expect(pending).resolves.toEqual(response);
|
||||
expect(received).toHaveBeenCalledWith(response);
|
||||
});
|
||||
|
||||
it('rejects an error response with id zero', async () => {
|
||||
const pending = transport.send({ jsonrpc: '2.0', id: 0, method: 'tools/list' });
|
||||
const rejected = expect(pending).rejects.toThrow('Tool listing failed');
|
||||
|
||||
MockWebSocket.last.emit('message', JSON.stringify({
|
||||
jsonrpc: '2.0', id: 0, error: { code: -32603, message: 'Tool listing failed' },
|
||||
}));
|
||||
|
||||
await rejected;
|
||||
});
|
||||
});
|
||||
@@ -13,6 +13,31 @@ import {
|
||||
import { promises as fs } from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { Memory } from '../../src/memory/memory';
|
||||
|
||||
describe('Memory history limits', () => {
|
||||
test('an explicit zero returns no recent entries', async () => {
|
||||
const memory = new Memory();
|
||||
const first = await memory.add('First message', 'user');
|
||||
const second = await memory.add('Second message', 'assistant');
|
||||
|
||||
expect(memory.getRecent(0)).toEqual([]);
|
||||
expect(memory.getRecent(1)).toEqual([second]);
|
||||
expect(memory.getRecent(3)).toEqual([first, second]);
|
||||
expect(memory.getAll()).toEqual([first, second]);
|
||||
});
|
||||
|
||||
test('zero context entries differs from an omitted limit', async () => {
|
||||
const memory = new Memory();
|
||||
await memory.add('First message', 'user');
|
||||
await memory.add('Second message', 'assistant');
|
||||
|
||||
expect(memory.buildContext(0)).toBe('');
|
||||
expect(memory.buildContext(1)).toBe('assistant: Second message');
|
||||
expect(memory.buildContext()).toBe('user: First message\nassistant: Second message');
|
||||
expect(memory.size).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('FileMemory', () => {
|
||||
let tempDir: string;
|
||||
@@ -76,6 +101,20 @@ describe('FileMemory', () => {
|
||||
expect(recent[1].content).toBe('Message 3');
|
||||
});
|
||||
|
||||
test('zero history limits survive a file-memory reload', async () => {
|
||||
const memory = createFileMemory({ filePath: testFilePath });
|
||||
const first = await memory.add('First message', 'user');
|
||||
const second = await memory.add('Second message', 'assistant');
|
||||
const reloaded = createFileMemory({ filePath: testFilePath });
|
||||
|
||||
expect(await reloaded.getRecent(0)).toEqual([]);
|
||||
expect(await reloaded.buildContext(0)).toBe('');
|
||||
expect(await reloaded.getRecent(1)).toEqual([second]);
|
||||
expect(await reloaded.buildContext(1)).toBe('assistant: Second message');
|
||||
expect(await reloaded.buildContext()).toBe('user: First message\nassistant: Second message');
|
||||
expect(await reloaded.getAll()).toEqual([first, second]);
|
||||
});
|
||||
|
||||
test('delete removes entry', async () => {
|
||||
const memory = createFileMemory({ filePath: testFilePath });
|
||||
const entry = await memory.add('Hello world', 'user');
|
||||
@@ -268,6 +307,18 @@ describe('FileMemory', () => {
|
||||
});
|
||||
|
||||
describe('AutoMemory', () => {
|
||||
test('zero recent entries does not return stored automatic memories', async () => {
|
||||
const memory = createAutoMemory({
|
||||
policies: [{ name: 'store-all', condition: () => true, action: 'store', priority: 1000 }]
|
||||
});
|
||||
const first = await memory.add('First message', 'user');
|
||||
const second = await memory.add('Second message', 'assistant');
|
||||
|
||||
expect(memory.getRecent(0)).toEqual([]);
|
||||
expect(memory.getRecent(1)).toEqual([second]);
|
||||
expect(memory.getAll()).toEqual([first, second]);
|
||||
});
|
||||
|
||||
test('createAutoMemory creates instance', () => {
|
||||
const memory = createAutoMemory();
|
||||
expect(memory).toBeInstanceOf(AutoMemory);
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
|
||||
import { promises as fs } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { FileSessionStore, createStoredSession } from '../../../src/session/store';
|
||||
|
||||
function gate(): { promise: Promise<void>; release: () => void } {
|
||||
let release!: () => void;
|
||||
const promise = new Promise<void>(resolve => { release = resolve; });
|
||||
return { promise, release };
|
||||
}
|
||||
|
||||
describe('FileSessionStore save ordering', () => {
|
||||
let directory: string;
|
||||
let filePath: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
directory = await fs.mkdtemp(join(tmpdir(), 'praison-session-save-'));
|
||||
filePath = join(directory, 'sessions.json');
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
jest.restoreAllMocks();
|
||||
jest.useRealTimers();
|
||||
await fs.rm(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function pauseFirstWrite() {
|
||||
const entered = gate();
|
||||
const blocked = gate();
|
||||
const writeFile = fs.writeFile.bind(fs);
|
||||
let writes = 0;
|
||||
jest.spyOn(fs, 'writeFile').mockImplementation(async (...args: Parameters<typeof fs.writeFile>) => {
|
||||
if (++writes === 1) {
|
||||
entered.release();
|
||||
await blocked.promise;
|
||||
}
|
||||
return writeFile(...args);
|
||||
});
|
||||
return { entered: entered.promise, release: blocked.release, writes: () => writes };
|
||||
}
|
||||
|
||||
it.each(['set', 'delete', 'clear'] as const)('persists %s during an in-flight save on close', async operation => {
|
||||
const store = new FileSessionStore(filePath);
|
||||
const first = createStoredSession({ turn: 1 });
|
||||
const second = createStoredSession({ turn: 2 });
|
||||
await store.set(first);
|
||||
const paused = pauseFirstWrite();
|
||||
const saving = store.save();
|
||||
await paused.entered;
|
||||
if (operation === 'set') await store.set(second);
|
||||
else if (operation === 'delete') await store.delete(first.id);
|
||||
else await store.clear();
|
||||
paused.release();
|
||||
await saving;
|
||||
await store.close();
|
||||
|
||||
const reloaded = new FileSessionStore(filePath);
|
||||
await reloaded.load();
|
||||
expect((await reloaded.list()).map(session => session.id)).toEqual(operation === 'set' ? [first.id, second.id] : []);
|
||||
});
|
||||
|
||||
it('serializes saves so an older snapshot cannot overwrite a newer one', async () => {
|
||||
const store = new FileSessionStore(filePath);
|
||||
const session = createStoredSession({ turn: 1 });
|
||||
await store.set(session);
|
||||
const paused = pauseFirstWrite();
|
||||
const firstSave = store.save();
|
||||
await paused.entered;
|
||||
await store.set({ ...session, data: { turn: 2 } });
|
||||
const secondSave = store.save();
|
||||
await new Promise<void>(resolve => setImmediate(resolve));
|
||||
const writesBeforeRelease = paused.writes();
|
||||
paused.release();
|
||||
await Promise.all([firstSave, secondSave]);
|
||||
await store.close();
|
||||
|
||||
const reloaded = new FileSessionStore(filePath);
|
||||
await reloaded.load();
|
||||
expect(writesBeforeRelease).toBe(1);
|
||||
expect((await reloaded.get(session.id))?.data).toEqual({ turn: 2 });
|
||||
});
|
||||
|
||||
it('waits for an outstanding save before closing', async () => {
|
||||
const store = new FileSessionStore(filePath);
|
||||
await store.set(createStoredSession());
|
||||
const paused = pauseFirstWrite();
|
||||
const saving = store.save();
|
||||
await paused.entered;
|
||||
let closed = false;
|
||||
const closing = store.close().then(() => { closed = true; });
|
||||
await new Promise<void>(resolve => setImmediate(resolve));
|
||||
const writesBeforeRelease = paused.writes();
|
||||
const closedBeforeRelease = closed;
|
||||
paused.release();
|
||||
await Promise.all([saving, closing]);
|
||||
expect(writesBeforeRelease).toBe(1);
|
||||
expect(closedBeforeRelease).toBe(false);
|
||||
});
|
||||
|
||||
it('does not queue repeated autosaves behind a blocked write', async () => {
|
||||
jest.useFakeTimers();
|
||||
const store = new FileSessionStore(filePath, { autoSaveInterval: 10 });
|
||||
const save = jest.spyOn(store, 'save');
|
||||
const first = createStoredSession({ turn: 1 });
|
||||
const second = createStoredSession({ turn: 2 });
|
||||
await store.set(first);
|
||||
const paused = pauseFirstWrite();
|
||||
const saving = store.save();
|
||||
await paused.entered;
|
||||
await store.set(second);
|
||||
jest.advanceTimersByTime(100);
|
||||
const saveCallsBeforeRelease = save.mock.calls.length;
|
||||
paused.release();
|
||||
await saving;
|
||||
jest.advanceTimersByTime(10);
|
||||
const saveCallsAfterRelease = save.mock.calls.length;
|
||||
await store.close();
|
||||
|
||||
const reloaded = new FileSessionStore(filePath);
|
||||
await reloaded.load();
|
||||
expect(saveCallsBeforeRelease).toBe(1);
|
||||
expect(saveCallsAfterRelease).toBe(2);
|
||||
expect((await reloaded.list()).map(session => session.id)).toEqual([first.id, second.id]);
|
||||
});
|
||||
|
||||
it('retries a failed save on close without poisoning later saves', async () => {
|
||||
const store = new FileSessionStore(filePath);
|
||||
const session = createStoredSession({ turn: 1 });
|
||||
await store.set(session);
|
||||
jest.spyOn(fs, 'writeFile').mockRejectedValueOnce(new Error('disk unavailable'));
|
||||
await expect(store.save()).rejects.toThrow('disk unavailable');
|
||||
await store.close();
|
||||
const reloaded = new FileSessionStore(filePath);
|
||||
await reloaded.load();
|
||||
expect((await reloaded.get(session.id))?.data).toEqual({ turn: 1 });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,81 @@
|
||||
import { describe, it, expect } from '@jest/globals';
|
||||
import { Session } from '../../../src/session/session';
|
||||
import { MemoryDbAdapter } from '../../../src/db/memory-adapter';
|
||||
|
||||
const factories: Array<[string, () => Session]> = [
|
||||
['new', () => new Session()],
|
||||
['setState', () => { const s = new Session(); s.setState({}); return s; }],
|
||||
['clearState', () => { const s = new Session(); s.set('old', 1); s.clearState(); return s; }],
|
||||
['fromJSON', () => Session.fromJSON(JSON.parse(JSON.stringify(new Session().toJSON())))],
|
||||
];
|
||||
|
||||
describe.each(factories)('Session state keys after %s', (_name, create) => {
|
||||
it.each(['toString', 'constructor', '__proto__'])('does not report an unstored %s key', key => {
|
||||
const session = create();
|
||||
expect(session.has(key)).toBe(false);
|
||||
expect(session.get(key)).toBeUndefined();
|
||||
expect(session.delete(key)).toBe(false);
|
||||
});
|
||||
|
||||
it.each(['toString', 'constructor', '__proto__'])('stores and deletes %s as ordinary data', key => {
|
||||
const session = create();
|
||||
const value = { preference: 'window seat' };
|
||||
session.set(key, value);
|
||||
expect(session.has(key)).toBe(true);
|
||||
expect(session.get(key)).toEqual(value);
|
||||
expect(session.keys()).toEqual([key]);
|
||||
expect(JSON.parse(JSON.stringify(session.toJSON())).data[key]).toEqual(value);
|
||||
expect(session.delete(key)).toBe(true);
|
||||
expect(session.has(key)).toBe(false);
|
||||
expect(session.get(key)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('preserves an explicitly stored undefined value until deletion', () => {
|
||||
const session = create();
|
||||
session.set('optional', undefined);
|
||||
expect(session.has('optional')).toBe(true);
|
||||
expect(session.delete('optional')).toBe(true);
|
||||
expect(session.has('optional')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
it('preserves special keys supplied through setState and a JSON round trip', () => {
|
||||
const state = JSON.parse('{"__proto__":{"seat":"window"},"constructor":2,"toString":3}');
|
||||
const session = new Session();
|
||||
session.setState(state);
|
||||
const restored = Session.fromJSON(JSON.parse(JSON.stringify(session.toJSON())));
|
||||
expect(restored.getState()).toEqual(state);
|
||||
expect(restored.delete('__proto__')).toBe(true);
|
||||
expect(restored.has('__proto__')).toBe(false);
|
||||
restored.set('__proto__', { seat: 'aisle' });
|
||||
expect(restored.keys()).toContain('__proto__');
|
||||
expect(restored.get('__proto__')).toEqual({ seat: 'aisle' });
|
||||
});
|
||||
|
||||
it('preserves special keys across database save/load without inheriting unstored keys', async () => {
|
||||
const db = new MemoryDbAdapter();
|
||||
const session = new Session({ id: 'state-keys', db });
|
||||
session.set('__proto__', { preference: 'window seat' });
|
||||
await session.save();
|
||||
const restored = new Session({ id: session.id, db });
|
||||
await restored.load();
|
||||
expect(restored.get('__proto__')).toEqual({ preference: 'window seat' });
|
||||
expect(restored.keys()).toEqual(['__proto__']);
|
||||
expect(restored.has('toString')).toBe(false);
|
||||
restored.delete('__proto__');
|
||||
restored.set('__proto__', { preference: 'aisle' });
|
||||
expect(restored.keys()).toEqual(['__proto__']);
|
||||
expect(restored.get('__proto__')).toEqual({ preference: 'aisle' });
|
||||
});
|
||||
|
||||
it('exports state as an ordinary object while preserving special own keys', () => {
|
||||
const session = new Session();
|
||||
session.set('seat', 'window');
|
||||
session.set('__proto__', { preference: 'quiet' });
|
||||
const exported = session.toJSON().data;
|
||||
expect(Object.getPrototypeOf(exported)).toBe(Object.prototype);
|
||||
expect(exported.hasOwnProperty('seat')).toBe(true);
|
||||
expect(exported.hasOwnProperty('__proto__')).toBe(true);
|
||||
expect(JSON.parse(JSON.stringify(exported))['__proto__']).toEqual({ preference: 'quiet' });
|
||||
expect(session.has('toString')).toBe(false);
|
||||
});
|
||||
@@ -0,0 +1,84 @@
|
||||
import { describe, expect, it } from '@jest/globals';
|
||||
import { ToolsRegistry } from '../../../src/tools/registry/registry';
|
||||
import type { ToolMetadata } from '../../../src/tools/registry/types';
|
||||
|
||||
const missingPackage = '@praisonai-test/not-installed-registry-fixture';
|
||||
function register(registry: ToolsRegistry, packageName: string) {
|
||||
const metadata: ToolMetadata = {
|
||||
id: 'dependency-test', displayName: 'Dependency test', description: 'Local status test',
|
||||
tags: [], requiredEnv: [], optionalEnv: [], docsSlug: 'tools/dependency-test',
|
||||
install: { npm: `npm install ${packageName}`, pnpm: '', yarn: '', bun: '' },
|
||||
capabilities: {}, packageName,
|
||||
};
|
||||
registry.register(metadata, () => ({
|
||||
name: 'dependency-test', description: 'Local status test',
|
||||
parameters: { type: 'object', properties: {} }, execute: async () => 'ok',
|
||||
}));
|
||||
}
|
||||
|
||||
describe('ToolsRegistry installation cache lifecycle', () => {
|
||||
it('does not report a removed cached tool as installed', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
register(registry, 'node:path');
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(true);
|
||||
expect(registry.unregister('dependency-test')).toBe(true);
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(false);
|
||||
expect(await registry.getInstallStatus('dependency-test')).toBeNull();
|
||||
});
|
||||
|
||||
it('rechecks an installed dependency replaced by a missing one', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
register(registry, 'node:path');
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(true);
|
||||
register(registry, missingPackage);
|
||||
expect(await registry.getInstallStatus('dependency-test')).toMatchObject({
|
||||
installed: false, installCommand: `npm install ${missingPackage}`,
|
||||
});
|
||||
});
|
||||
|
||||
it('rechecks a missing dependency replaced by an installed one', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
register(registry, missingPackage);
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(false);
|
||||
register(registry, 'node:path');
|
||||
expect(await registry.getInstallStatus('dependency-test')).toMatchObject({ installed: true, installCommand: undefined });
|
||||
});
|
||||
|
||||
it('does not cache an old successful import after replacement', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
register(registry, 'node:path');
|
||||
const oldCheck = registry.checkInstalled('dependency-test');
|
||||
register(registry, missingPackage);
|
||||
expect(await oldCheck).toBe(true);
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(false);
|
||||
});
|
||||
|
||||
it('does not cache an old failed import after replacement', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
register(registry, missingPackage);
|
||||
const oldCheck = registry.checkInstalled('dependency-test');
|
||||
register(registry, 'node:path');
|
||||
expect(await oldCheck).toBe(false);
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(true);
|
||||
});
|
||||
|
||||
it('reports an unknown tool as unavailable', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
expect(await registry.checkInstalled('unknown')).toBe(false);
|
||||
expect(await registry.getInstallStatus('unknown')).toBeNull();
|
||||
});
|
||||
|
||||
it('keeps repeated installed checks consistent', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
register(registry, 'node:path');
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(true);
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(true);
|
||||
});
|
||||
|
||||
it('keeps repeated missing checks consistent', async () => {
|
||||
const registry = new ToolsRegistry();
|
||||
register(registry, missingPackage);
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(false);
|
||||
expect(await registry.checkInstalled('dependency-test')).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,104 @@
|
||||
import { describe, expect, it, jest } from '@jest/globals';
|
||||
import { ToolsRegistry } from '../../../src/tools/registry/registry';
|
||||
import { composeMiddleware, createRetryMiddleware } from '../../../src/tools/registry/middleware';
|
||||
import type { ToolMetadata, ToolMiddleware, ToolExecutionContext } from '../../../src/tools/registry/types';
|
||||
|
||||
const metadata: ToolMetadata = {
|
||||
id: 'retry-test', displayName: 'Retry test', description: 'Local retry test',
|
||||
tags: [], requiredEnv: [], optionalEnv: [], docsSlug: 'tools/retry-test',
|
||||
install: { npm: '', pnpm: '', yarn: '', bun: '' }, capabilities: {}, packageName: 'node:path',
|
||||
};
|
||||
|
||||
function runPipeline(mode: 'composed' | 'registry', middleware: ToolMiddleware[], execute: () => Promise<unknown>, context: ToolExecutionContext = {}) {
|
||||
if (mode === 'composed') return composeMiddleware(...middleware)({}, context, execute);
|
||||
const registry = new ToolsRegistry();
|
||||
registry.register(metadata, () => ({
|
||||
name: 'retry-test', description: 'Local retry test',
|
||||
parameters: { type: 'object', properties: {} }, execute,
|
||||
}));
|
||||
for (const layer of middleware) registry.use(layer);
|
||||
return registry.create('retry-test').execute({}, context);
|
||||
}
|
||||
|
||||
describe.each(['composed', 'registry'] as const)('%s middleware retry chain', mode => {
|
||||
it('reapplies result shaping when the tool succeeds on retry', async () => {
|
||||
let attempts = 0;
|
||||
let shapingCalls = 0;
|
||||
const shape: ToolMiddleware = async (_input, _context, next) => {
|
||||
shapingCalls++;
|
||||
return { wrapped: await next() };
|
||||
};
|
||||
const result = await runPipeline(mode, [createRetryMiddleware(1, 0), shape], async () => {
|
||||
if (++attempts === 1) throw new Error('temporary tool failure');
|
||||
return 'recovered';
|
||||
});
|
||||
expect(result).toEqual({ wrapped: 'recovered' });
|
||||
expect(attempts).toBe(2);
|
||||
expect(shapingCalls).toBe(2);
|
||||
});
|
||||
|
||||
it('re-enters a downstream middleware that failed before the tool', async () => {
|
||||
let layerAttempts = 0;
|
||||
let toolCalls = 0;
|
||||
const layer: ToolMiddleware = async (_input, _context, next) => {
|
||||
if (++layerAttempts === 1) throw new Error('temporary middleware failure');
|
||||
return { wrapped: await next() };
|
||||
};
|
||||
const result = await runPipeline(mode, [createRetryMiddleware(1, 0), layer], async () => {
|
||||
toolCalls++;
|
||||
return 'result';
|
||||
});
|
||||
expect(result).toEqual({ wrapped: 'result' });
|
||||
expect(layerAttempts).toBe(2);
|
||||
expect(toolCalls).toBe(1);
|
||||
});
|
||||
|
||||
it('runs downstream failure handling on every exhausted attempt', async () => {
|
||||
let attempts = 0;
|
||||
let handled = 0;
|
||||
const layer: ToolMiddleware = async (_input, _context, next) => {
|
||||
try { return await next(); } finally { handled++; }
|
||||
};
|
||||
await expect(runPipeline(mode, [createRetryMiddleware(2, 0), layer], async () => {
|
||||
throw new Error(`failure-${++attempts}`);
|
||||
})).rejects.toThrow('failure-3');
|
||||
expect(attempts).toBe(3);
|
||||
expect(handled).toBe(3);
|
||||
});
|
||||
|
||||
it('does not re-enter downstream middleware after cancellation during backoff', async () => {
|
||||
jest.useFakeTimers();
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
let entries = 0;
|
||||
let calls = 0;
|
||||
const layer: ToolMiddleware = async (_input, _context, next) => { entries++; return next(); };
|
||||
const done = runPipeline(mode, [createRetryMiddleware(1, 1000), layer], async () => {
|
||||
if (++calls === 1) throw new Error('temporary');
|
||||
return 'retried';
|
||||
}, { signal: controller.signal }).then(value => ({ value, error: undefined }), error => ({ value: undefined, error }));
|
||||
await jest.advanceTimersByTimeAsync(0);
|
||||
expect(entries).toBe(1);
|
||||
controller.abort();
|
||||
await jest.advanceTimersByTimeAsync(1000);
|
||||
expect(entries).toBe(1);
|
||||
expect(calls).toBe(1);
|
||||
expect((await done).error).toBe(controller.signal.reason);
|
||||
} finally { jest.clearAllTimers(); jest.useRealTimers(); }
|
||||
});
|
||||
|
||||
it('preserves middleware order for a successful first attempt', async () => {
|
||||
const order: string[] = [];
|
||||
const layer = (name: string): ToolMiddleware => async (_input, _context, next) => {
|
||||
order.push(`${name}:before`);
|
||||
const result = await next();
|
||||
order.push(`${name}:after`);
|
||||
return result;
|
||||
};
|
||||
expect(await runPipeline(mode, [createRetryMiddleware(1, 0), layer('a'), layer('b')], async () => {
|
||||
order.push('tool');
|
||||
return 'ok';
|
||||
})).toBe('ok');
|
||||
expect(order).toEqual(['a:before', 'b:before', 'tool', 'b:after', 'a:after']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,54 @@
|
||||
import { describe, expect, it, jest } from '@jest/globals';
|
||||
import { createValidationMiddleware } from '../../../src/tools/registry/middleware';
|
||||
|
||||
const payloads: Array<[string, unknown]> = [
|
||||
['CJK text', '你好'],
|
||||
['emoji', '🙂'],
|
||||
['nested JSON', { message: '中文🙂', values: ['é'] }],
|
||||
];
|
||||
|
||||
describe('tool payload UTF-8 byte limits', () => {
|
||||
it.each(payloads)('rejects %s above its actual byte budget', async (_label, input) => {
|
||||
const bytes = Buffer.byteLength(JSON.stringify(input), 'utf8');
|
||||
const next = jest.fn(async () => 'executed');
|
||||
await expect(createValidationMiddleware()(input, { limits: { maxPayloadBytes: bytes - 1 } }, next))
|
||||
.rejects.toThrow(`Input payload size (${bytes} bytes) exceeds limit (${bytes - 1} bytes)`);
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(payloads)('accepts %s at the exact byte budget', async (_label, input) => {
|
||||
const bytes = Buffer.byteLength(JSON.stringify(input), 'utf8');
|
||||
const next = jest.fn(async () => 'executed');
|
||||
expect(await createValidationMiddleware()(input, { limits: { maxPayloadBytes: bytes } }, next)).toBe('executed');
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('preserves ASCII boundaries', async () => {
|
||||
const next = jest.fn(async () => 'executed');
|
||||
const validate = createValidationMiddleware();
|
||||
await expect(validate('hello', { limits: { maxPayloadBytes: 6 } }, next)).rejects.toThrow('7 bytes');
|
||||
expect(await validate('hello', { limits: { maxPayloadBytes: 7 } }, next)).toBe('executed');
|
||||
expect(next).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('does not serialize inputs when no payload limit is configured', async () => {
|
||||
const cyclic: { self?: unknown } = {};
|
||||
cyclic.self = cyclic;
|
||||
const next = jest.fn(async () => 'executed');
|
||||
expect(await createValidationMiddleware()(cyclic, {}, next)).toBe('executed');
|
||||
});
|
||||
|
||||
it('continues rejecting non-JSON undefined input when a limit is configured', async () => {
|
||||
const next = jest.fn(async () => 'executed');
|
||||
await expect(createValidationMiddleware()(undefined, { limits: { maxPayloadBytes: 10 } }, next)).rejects.toThrow(TypeError);
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('preserves circular JSON errors when a limit is configured', async () => {
|
||||
const cyclic: { self?: unknown } = {};
|
||||
cyclic.self = cyclic;
|
||||
const next = jest.fn(async () => 'executed');
|
||||
await expect(createValidationMiddleware()(cyclic, { limits: { maxPayloadBytes: 100 } }, next)).rejects.toThrow(TypeError);
|
||||
expect(next).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,94 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
|
||||
import { getEventListeners } from 'events';
|
||||
import { FunctionTool } from '../../../src/tools/decorator';
|
||||
|
||||
function makeTool(execute: () => Promise<string>) {
|
||||
return new FunctionTool({ name: 'retry_test', execute, retryPolicy: { maxAttempts: 2, initialDelayMs: 1000, maxDelayMs: 1000 } });
|
||||
}
|
||||
|
||||
describe('FunctionTool retry backoff cancellation', () => {
|
||||
beforeEach(() => { jest.useFakeTimers(); });
|
||||
afterEach(() => { jest.clearAllTimers(); jest.useRealTimers(); });
|
||||
|
||||
it.each(['execute', 'executeRaw'] as const)('%s settles promptly when cancelled during backoff', async method => {
|
||||
const controller = new AbortController();
|
||||
const failure = new Error('connection dropped');
|
||||
let calls = 0;
|
||||
let settled = false;
|
||||
const tool = makeTool(async () => { if (++calls === 1) throw failure; return 'retried'; });
|
||||
const done = tool[method]({}, { signal: controller.signal }).then(
|
||||
value => { settled = true; return { value, error: undefined }; },
|
||||
error => { settled = true; return { value: undefined, error }; },
|
||||
);
|
||||
await jest.advanceTimersByTimeAsync(0);
|
||||
expect(calls).toBe(1);
|
||||
controller.abort();
|
||||
await jest.advanceTimersByTimeAsync(0);
|
||||
expect(settled).toBe(true);
|
||||
expect((await done).error).toBe(failure);
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
expect(getEventListeners(controller.signal, 'abort')).toHaveLength(0);
|
||||
await jest.advanceTimersByTimeAsync(2000);
|
||||
expect(calls).toBe(1);
|
||||
});
|
||||
|
||||
it.each(['execute', 'executeRaw'] as const)('%s does not start another call after the original delay', async method => {
|
||||
const controller = new AbortController();
|
||||
const failure = new Error('connection dropped');
|
||||
let calls = 0;
|
||||
const tool = makeTool(async () => { if (++calls === 1) throw failure; return 'retried'; });
|
||||
const done = tool[method]({}, { signal: controller.signal }).then(value => ({ value, error: undefined }), error => ({ value: undefined, error }));
|
||||
await jest.advanceTimersByTimeAsync(0);
|
||||
controller.abort();
|
||||
await jest.advanceTimersByTimeAsync(1000);
|
||||
expect(calls).toBe(1);
|
||||
expect((await done).error).toBe(failure);
|
||||
});
|
||||
|
||||
it('retains the configured delay without cancellation and releases its listener', async () => {
|
||||
const controller = new AbortController();
|
||||
let calls = 0;
|
||||
const tool = makeTool(async () => { if (++calls === 1) throw new Error('connection dropped'); return 'recovered'; });
|
||||
const done = tool.execute({}, { signal: controller.signal });
|
||||
await jest.advanceTimersByTimeAsync(999);
|
||||
expect(calls).toBe(1);
|
||||
await jest.advanceTimersByTimeAsync(1);
|
||||
expect(await done).toBe('recovered');
|
||||
expect(calls).toBe(2);
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
expect(getEventListeners(controller.signal, 'abort')).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('retains retries without a signal', async () => {
|
||||
let calls = 0;
|
||||
const tool = makeTool(async () => { if (++calls === 1) throw new Error('connection dropped'); return 'recovered'; });
|
||||
const done = tool.execute({});
|
||||
await jest.advanceTimersByTimeAsync(1000);
|
||||
expect(await done).toBe('recovered');
|
||||
expect(calls).toBe(2);
|
||||
});
|
||||
|
||||
it('preserves the last error when already cancelled at failure', async () => {
|
||||
const controller = new AbortController();
|
||||
const failure = new Error('connection dropped');
|
||||
const tool = makeTool(async () => { controller.abort(); throw failure; });
|
||||
await expect(tool.execute({}, { signal: controller.signal })).rejects.toBe(failure);
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('does not retry a nonretryable failure', async () => {
|
||||
const failure = new Error('invalid argument');
|
||||
let calls = 0;
|
||||
const tool = makeTool(async () => { calls++; throw failure; });
|
||||
await expect(tool.execute({})).rejects.toBe(failure);
|
||||
expect(calls).toBe(1);
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('does not allocate a backoff on immediate success', async () => {
|
||||
const controller = new AbortController();
|
||||
expect(await makeTool(async () => 'ok').execute({}, { signal: controller.signal })).toBe('ok');
|
||||
expect(jest.getTimerCount()).toBe(0);
|
||||
expect(getEventListeners(controller.signal, 'abort')).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
+340
-177
@@ -14,27 +14,145 @@
|
||||
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
|
||||
</p>
|
||||
|
||||
<div align="center">
|
||||
# PraisonAI
|
||||
|
||||
# PraisonAI 🦞
|
||||
**Private AI that earns trust one task at a time.**
|
||||
|
||||
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
PraisonAI runs AI agents on your own computers, even fully offline, so private data never has to leave the building.
|
||||
|
||||
</div>
|
||||
- **For highly regulated teams (early access):** each task moves to your own AI only after it passes a test. If quality slips, it moves back automatically, and every step is recorded for regulators. [How it works](#how-the-handover-works-early-access)
|
||||
- **For everyone else:** a free, open-source Python agent framework. With a local model you need no API key and pay no cloud bill, and your files stay yours.
|
||||
|
||||
PraisonAI 🦞 — **Hire a 24/7 AI Workforce.** Stop writing boilerplate and start shipping autonomous, self-improving agents that research, plan, and execute tasks across your apps. From one agent to an entire organization, deployed in 5 lines of code.
|
||||
[2M+ downloads](https://pepy.tech/projects/praisonai) · [9.3K stars](https://github.com/MervinPraison/PraisonAI) · [600+ releases in the last year](https://github.com/MervinPraison/PraisonAI/releases) · [80+ contributors](https://github.com/MervinPraison/PraisonAI/graphs/contributors) · Python
|
||||
|
||||
```bash
|
||||
curl -fsSL https://praison.ai/install.sh | bash
|
||||
pip install praisonai
|
||||
```
|
||||
|
||||
Only need the Python library? `pip install praisonaiagents` installs just the lightweight core.
|
||||
|
||||
## Using PraisonAI for something else?
|
||||
|
||||
PraisonAI is still a general-purpose, open-source agent framework, and everything works as before: the same packages, imports, commands and examples. Go straight to what you came for:
|
||||
|
||||
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs)
|
||||
- **Workflows:** [AgentFlow: route, parallel, loop, repeat](https://praison.ai/docs/concepts/agentflow)
|
||||
- **Memory and knowledge:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge](https://praison.ai/docs/concepts/knowledge) · [RAG](https://praison.ai/docs/concepts/rag)
|
||||
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
|
||||
- **Bots (Telegram, Discord, Slack, WhatsApp):** [Bot CLI](https://praison.ai/docs/cli/bot) · [Bot gateway](https://praison.ai/docs/features/bot-gateway)
|
||||
- **Terminal CLI:** [`praisonai code`](https://praison.ai/docs/cli/code) · [`praisonai run`](https://praison.ai/docs/cli/run) · [`praisonai chat`](https://praison.ai/docs/cli/chat) · [CLI reference](https://praison.ai/docs/cli/cli-reference)
|
||||
- **Training and fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
|
||||
- **Browser automation:** [Browser agent](https://praison.ai/docs/features/browser-agent) · [Browser CLI](https://praison.ai/docs/cli/browser)
|
||||
- **Sandboxes:** [Sandboxed code execution](https://praison.ai/docs/features/sandbox)
|
||||
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
|
||||
- **Dashboard and UI:** [Claw dashboard](https://praison.ai/docs/concepts/claw) · [Chat UI](https://praison.ai/docs/ui)
|
||||
- **YAML, no code:** [YAML agents](#-using-yaml-no-code) · [YAML reference](https://praison.ai/docs/features/yaml-configuration-reference)
|
||||
- **JavaScript and TypeScript:** [JavaScript SDK](https://praison.ai/docs/js)
|
||||
- **Desktop app:** [PraisonAI Desktop](https://praison.ai/docs/features/desktop/index) · [Downloads](#desktop-app)
|
||||
- **Everything else:** the [full feature reference](#full-feature-reference) further down this page
|
||||
|
||||
## Quick start
|
||||
|
||||
Set a key for any supported provider (OpenAI shown here), or skip the key and use a local model as in the next section.
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY="your-api-key"
|
||||
```
|
||||
|
||||
```python
|
||||
from praisonaiagents import Agent
|
||||
|
||||
agent = Agent(instructions="You are a helpful assistant.")
|
||||
print(agent.start("In two sentences, why do some companies keep their data on their own computers?"))
|
||||
```
|
||||
|
||||
## Run fully offline
|
||||
|
||||
Run the same agent on a model on your own computer with [Ollama](https://ollama.com). You need no API key, and your prompts and answers stay on the machine.
|
||||
|
||||
Install the package and pull the model while you still have internet; after that, the agent runs with no network at all. If you installed only the core library, use `pip install "praisonaiagents[llm]"`, which adds the local-model support.
|
||||
|
||||
```bash
|
||||
# Start Ollama first (the app, or "ollama serve" in another terminal)
|
||||
ollama pull llama3.2
|
||||
export DO_NOT_TRACK=true # keeps telemetry off (PraisonAI sends none unless you turn it on)
|
||||
```
|
||||
|
||||
```python
|
||||
from praisonaiagents import Agent
|
||||
|
||||
agent = Agent(instructions="You are a helpful assistant.", model="ollama/llama3.2")
|
||||
print(agent.start("Why is the sky blue?"))
|
||||
```
|
||||
|
||||
If Ollama runs on another computer on your network, pass `base_url` (or set `OLLAMA_API_BASE`):
|
||||
|
||||
```python
|
||||
agent = Agent(instructions="...", model="ollama/llama3.2", base_url="http://192.168.1.10:11434")
|
||||
```
|
||||
|
||||
vLLM and LM Studio work the same way: use `hosted_vllm/` (vLLM) or `lm_studio/` with your server's `base_url`. More in the [Ollama guide](https://praison.ai/docs/models/ollama) and [Models docs](https://praison.ai/docs/models).
|
||||
|
||||
## How the handover works (early access)
|
||||
|
||||
1. **Watch:** records how each task is done today.
|
||||
2. **Test:** your own AI takes an exam made from real past work it never studied.
|
||||
3. **Hand over:** a task moves to your own AI only when it passes.
|
||||
4. **Roll back:** if quality drops, the task moves back automatically.
|
||||
5. **Record:** every handover leaves a record your regulator can check.
|
||||
|
||||
**What is "your own AI"?** An open model, such as Llama, Qwen or Gemma, running on your own computers. Use it as it is, or train it on your own past work with [PraisonAI Train](https://praison.ai/docs/train) on your own GPU servers. Your team decides when to train; nothing trains automatically. Either way, a task moves to it only after it passes that exam.
|
||||
|
||||
Highly regulated teams, such as banks, insurers and hospitals, want AI but can't send customer data to outside models. From 6 January 2027, OpenAI also takes no new self-serve fine-tuning jobs ([notice](https://developers.openai.com/api/docs/deprecations)), so teams that customise models need models they own. The handover lets these teams move work to AI they own, one task at a time, with a record that shows why each step was trusted.
|
||||
|
||||
It is built on parts of this framework that are live today: tracing, evaluation, fine-tuning, model routing and local models. The handover itself is in early access, and we are taking on our first design partners. To take part, see [praison.ai/enterprise](https://praison.ai/enterprise/).
|
||||
|
||||
## Features
|
||||
|
||||
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs) · [Workflows](https://praison.ai/docs/concepts/agentflow)
|
||||
- **Memory:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge and RAG](https://praison.ai/docs/concepts/rag) · [Sessions](https://praison.ai/docs/concepts/session-management)
|
||||
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
|
||||
- **Guardrails and approval:** [Guardrails](https://praison.ai/docs/concepts/guardrails) · [Human approval](https://praison.ai/docs/concepts/approval) · [Policy engine](https://praison.ai/docs/features/policy-engine)
|
||||
- **Tracing and evaluation:** [Tracing](https://praison.ai/docs/observability/overview) · [Telemetry](https://praison.ai/docs/features/telemetry) · [Langfuse](https://praison.ai/docs/observability/langfuse) · [Evaluation](https://praison.ai/docs/concepts/evaluation)
|
||||
- **Your choice of model:** [Models](https://praison.ai/docs/models) · [Model router](https://praison.ai/docs/features/model-router)
|
||||
- **Fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
|
||||
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
|
||||
|
||||
## JavaScript and TypeScript
|
||||
|
||||
```bash
|
||||
npm install praisonai
|
||||
```
|
||||
|
||||
The TypeScript and JavaScript SDK has its own guide at [praison.ai/docs/js](https://praison.ai/docs/js). A short example is in [Using JavaScript Code](#-using-javascript-code) below.
|
||||
|
||||
## The company
|
||||
|
||||
- **Who:** Praison AI Limited in Cambridge, UK, building PraisonAI since September 2023. Co-founders Mervin Praison (CEO), Leebana Smith (COO) and Karthick Subramanian (CBO). [About us](https://praison.ai/about/)
|
||||
- **What we're building:** the task handover above, as a self-hosted product on top of this framework. It is in early access, and we are taking on our first design partners.
|
||||
- **Why now:** highly regulated teams want AI but can't send customer data to outside models, and from 6 January 2027 OpenAI takes no new self-serve fine-tuning jobs.
|
||||
- **How it fits together:** the framework stays free and MIT-licensed, and brings developers in. Regulated teams license the handover to run inside their own walls.
|
||||
- **Traction:** every number at the top of this page is public and linked to its source.
|
||||
- **Get in touch:** design partners, investors and press: enterprise@praison.ai or [praison.ai/enterprise](https://praison.ai/enterprise/)
|
||||
|
||||
## Docs, community and licence
|
||||
|
||||
- **Documentation:** [praison.ai/docs](https://praison.ai/docs)
|
||||
- **Questions and bugs:** [Discussions](https://github.com/MervinPraison/PraisonAI/discussions) · [Issues](https://github.com/MervinPraison/PraisonAI/issues)
|
||||
- **Contributing:** [Contributing guide](https://github.com/MervinPraison/PraisonAI/blob/main/CONTRIBUTING.md)
|
||||
- **Videos and news:** [YouTube](https://youtube.com/@MervinPraison) · [X](https://x.com/MervinPraison) · [LinkedIn](https://linkedin.com/in/mervinpraison)
|
||||
- **Desktop app:** macOS, Windows and Linux builds are on the [latest release](https://github.com/MervinPraison/PraisonAI/releases/latest)
|
||||
- **Licence:** [MIT](https://github.com/MervinPraison/PraisonAI/blob/main/LICENSE)
|
||||
|
||||
---
|
||||
|
||||
## Full feature reference
|
||||
|
||||
Everything below is the full reference that was on this page before. Nothing has been removed; the longest parts are folded, so click a heading's summary to open it.
|
||||
|
||||
### Desktop app
|
||||
|
||||
<div align="center">
|
||||
<br>
|
||||
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
|
||||
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
|
||||
</a>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
|
||||
<img src="https://img.shields.io/badge/Download_for_macOS-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Download for macOS" />
|
||||
</a>
|
||||
@@ -46,12 +164,17 @@ curl -fsSL https://praison.ai/install.sh | bash
|
||||
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
|
||||
<img src="https://img.shields.io/badge/Download_for_Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download for Linux" />
|
||||
</a>
|
||||
<br>
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
|
||||
</p>
|
||||
### Other ways to install
|
||||
|
||||
One-line installer:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://praison.ai/install.sh | bash
|
||||
```
|
||||
|
||||
The full `praisonai` package, with the CLI, bots, dashboard and integrations:
|
||||
|
||||
```
|
||||
██████╗ ██████╗ █████╗ ██╗███████╗ ██████╗ ███╗ ██╗ █████╗ ██╗
|
||||
@@ -64,10 +187,20 @@ curl -fsSL https://praison.ai/install.sh | bash
|
||||
pip install praisonai
|
||||
```
|
||||
|
||||
<p align="center">
|
||||
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
|
||||
</p>
|
||||
|
||||
<div align="center">
|
||||
<a href="https://docs.praison.ai">
|
||||
|
||||
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<a href="https://praison.ai/docs">
|
||||
<p align="center">
|
||||
<img src="https://img.shields.io/badge/📚_Documentation-Visit_docs.praison.ai-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
|
||||
<img src="https://img.shields.io/badge/📚_Documentation-Visit_praison.ai%2Fdocs-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
|
||||
</p>
|
||||
</a>
|
||||
</div>
|
||||
@@ -114,6 +247,9 @@ Most frameworks hand you one or two layers and leave the rest as homework. Prais
|
||||
|
||||
Each layer wraps the one inside it. When an agent misbehaves, the layer tells you where to look.
|
||||
|
||||
<details>
|
||||
<summary><strong>Show the five layers, examples and Managed Agents</strong></summary>
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ⬡ MANAGED AGENTS — Where does it actually run? │
|
||||
@@ -174,7 +310,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://docs.praison.ai/docs/concepts/handoffs)
|
||||
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://praison.ai/docs/concepts/handoffs)
|
||||
|
||||
### Layer 3 · Harness — *Can it act, and be checked?*
|
||||
|
||||
@@ -217,7 +353,7 @@ print(result.completion_reason)
|
||||
# surfaced here as completion_reason="error")
|
||||
```
|
||||
|
||||
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://docs.praison.ai/docs/features/doom-loop-detection)
|
||||
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://praison.ai/docs/features/doom-loop-detection)
|
||||
|
||||
### Layer 5 · Graph — *Who runs when, and who checks whom?*
|
||||
|
||||
@@ -237,7 +373,7 @@ flow = AgentFlow(steps=[
|
||||
flow.run("Ticket #123: login fails on Safari")
|
||||
```
|
||||
|
||||
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://docs.praison.ai/docs/concepts/agentflow)
|
||||
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://praison.ai/docs/concepts/agentflow)
|
||||
|
||||
### ⬡ Outside the stack: Managed Agents — *Where does it actually run?*
|
||||
|
||||
@@ -332,6 +468,8 @@ Sandboxes shut themselves down when idle (`auto_shutdown`, `idle_timeout_s`), an
|
||||
|
||||
<sub>Stack framing adapted from [The Five-Layer Agent Stack](https://mer.vin/2026/07/five-layer-agent-stack-match-bug-to-right-layer/) and [Agent Harnesses vs Orbs](https://mer.vin/2026/08/agent-harnesses-vs-orbs-why-remote-sandboxes-beat-local-agent-loops/).</sub>
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 🌌 The PraisonAI Ecosystem
|
||||
@@ -411,6 +549,11 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
</details>
|
||||
|
||||
<div align="center">
|
||||
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
|
||||
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
|
||||
</a>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
|
||||
<img src=".github/images/elon_musk_praisonai.png" alt="Highlighted by Elon Musk" width="600" />
|
||||
</a>
|
||||
@@ -426,13 +569,13 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
|--|---------|-----|
|
||||
| 🔌 | **MCP Protocol** — stdio, HTTP, WebSocket, SSE | `tools=MCP("npx ...")` |
|
||||
| 🧠 | **Planning Mode** — plan → execute → reason | `planning=True` |
|
||||
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://docs.praison.ai/docs/agents/deep-research) |
|
||||
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://docs.praison.ai/docs/code/external-agents) |
|
||||
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://praison.ai/docs/agents/deep-research) |
|
||||
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://praison.ai/docs/code/external-agents) |
|
||||
| 🔄 | **Agent Handoffs** — seamless conversation passing | `handoffs=[other_agent]` |
|
||||
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://docs.praison.ai/docs/concepts/guardrails) |
|
||||
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://praison.ai/docs/concepts/guardrails) |
|
||||
| | **Web Search + Fetch** — native browsing | `web=True` |
|
||||
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://docs.praison.ai/docs/concepts/reflection) |
|
||||
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://praison.ai/docs/concepts/reflection) |
|
||||
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://praison.ai/docs/concepts/agentflow) |
|
||||
| 🧠 | **Memory (zero deps)** — works out of the box | `memory=True` |
|
||||
|
||||
<details>
|
||||
@@ -443,18 +586,18 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
| 💡 | **Prompt Caching** — reduce latency + cost | `caching=True` |
|
||||
| 💾 | **Sessions + Auto-Save** — persistent state across restarts | `auto_save="my-project"` |
|
||||
| 💭 | **Thinking Budgets** — control reasoning depth | `agent.thinking_budget = 1024` |
|
||||
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://docs.praison.ai/docs/concepts/rag) |
|
||||
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://docs.praison.ai/docs/features/model-router) |
|
||||
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://docs.praison.ai/docs/features/checkpoints) |
|
||||
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://docs.praison.ai/docs/features/a2a) |
|
||||
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://docs.praison.ai/docs/features/context-compaction) |
|
||||
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://docs.praison.ai/docs/features/telemetry) |
|
||||
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://docs.praison.ai/docs/features/policy-engine) |
|
||||
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://docs.praison.ai/docs/features/background-tasks) |
|
||||
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://docs.praison.ai/docs/features/doom-loop-detection) |
|
||||
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://docs.praison.ai/docs/features/graph-memory) |
|
||||
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://docs.praison.ai/docs/features/sandbox) |
|
||||
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://docs.praison.ai/docs/features/bot-gateway) |
|
||||
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://praison.ai/docs/concepts/rag) |
|
||||
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://praison.ai/docs/features/model-router) |
|
||||
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://praison.ai/docs/features/checkpoints) |
|
||||
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://praison.ai/docs/features/a2a) |
|
||||
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://praison.ai/docs/features/context-compaction) |
|
||||
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://praison.ai/docs/features/telemetry) |
|
||||
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://praison.ai/docs/features/policy-engine) |
|
||||
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://praison.ai/docs/features/background-tasks) |
|
||||
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://praison.ai/docs/features/doom-loop-detection) |
|
||||
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://praison.ai/docs/features/graph-memory) |
|
||||
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://praison.ai/docs/features/sandbox) |
|
||||
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://praison.ai/docs/features/bot-gateway) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -465,6 +608,9 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|
||||
|
||||
## 📘 Using Python Code
|
||||
|
||||
<details>
|
||||
<summary><strong>Show Python examples: single agent, multi-agent, MCP, tools, databases, Claw, Langflow, UI</strong></summary>
|
||||
|
||||
> Unsure whether a result is a bug or model randomness? Check the [SDK Behavior Matrix](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/docs/sdk-behavior-matrix.md) for `stream`, memory prefetch, structured output, teams and AG-UI.
|
||||
|
||||
### 1. Single Agent
|
||||
@@ -510,7 +656,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
> 📖 [Full MCP docs](https://docs.praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
|
||||
> 📖 [Full MCP docs](https://praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
|
||||
|
||||
### 4. Custom Tools
|
||||
|
||||
@@ -562,7 +708,7 @@ agent.start("Search for AI news and calculate 15*4")
|
||||
```
|
||||
|
||||
> ⚠️ **Security Note:** Never use `eval()`, `exec()`, or `subprocess` in tool functions that process LLM-generated or user-supplied input. Always validate and sanitize inputs to prevent code injection attacks.
|
||||
> 📖 [Full tools docs](https://docs.praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
|
||||
> 📖 [Full tools docs](https://praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
|
||||
|
||||
### 5. Persistence (Databases)
|
||||
|
||||
@@ -579,7 +725,7 @@ agent = Agent(
|
||||
agent.chat("Hello!") # Auto-persists messages, runs, traces
|
||||
```
|
||||
|
||||
> 📖 [Full persistence docs](https://docs.praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
|
||||
> 📖 [Full persistence docs](https://praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
|
||||
|
||||
### 6. PraisonAI Claw 🦞 (Dashboard UI)
|
||||
|
||||
@@ -601,7 +747,7 @@ Copy `.env.example` to `.env` and configure the following variables:
|
||||
|
||||
Open **http://localhost:8082** — the dashboard comes with 13 built-in pages: Chat, Agents, Memory, Knowledge, Channels, Guardrails, Cron, and more. Add messaging channels directly from the UI.
|
||||
|
||||
> 📖 [Full Claw docs](https://docs.praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
|
||||
> 📖 [Full Claw docs](https://praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
|
||||
|
||||
### 7. Langflow Integration 🔗 (Visual Flow Builder)
|
||||
|
||||
@@ -614,7 +760,7 @@ praisonai flow
|
||||
|
||||
Open **http://localhost:7861** — use the **Agent** and **Agent Team** components to create sequential or parallel workflows. Connect Chat Input → Agent Team → Chat Output for instant multi-agent pipelines.
|
||||
|
||||
> 📖 [Full Flow docs](https://docs.praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
|
||||
> 📖 [Full Flow docs](https://praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
|
||||
|
||||
### 8. PraisonAI UI 🤖 (Clean Chat)
|
||||
|
||||
@@ -625,10 +771,15 @@ pip install "praisonai[ui]"
|
||||
praisonai ui
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 📄 Using YAML (No Code)
|
||||
|
||||
<details>
|
||||
<summary><strong>Show YAML examples</strong></summary>
|
||||
|
||||
### Example 1: Two Agents Working Together
|
||||
|
||||
Create `agents.yaml`:
|
||||
@@ -700,6 +851,8 @@ praisonai agents.yaml
|
||||
> - Tools in `tools.py` are automatically discovered
|
||||
> - The function's docstring helps the AI understand how to use it
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 🎯 CLI Quick Reference
|
||||
@@ -718,23 +871,26 @@ praisonai agents.yaml
|
||||
| **Development** | `commit`, `docs`, `checkpoint`, `hooks` |
|
||||
| **Scheduling** | `schedule start`, `schedule list`, `schedule stop` |
|
||||
|
||||
> 📖 [Full CLI reference](https://docs.praison.ai/docs/cli/cli-reference)
|
||||
> 📖 [Full CLI reference](https://praison.ai/docs/cli/cli-reference)
|
||||
|
||||
---
|
||||
|
||||
## ✨ Key Features
|
||||
|
||||
<details>
|
||||
<summary><strong>Show all features by category</strong></summary>
|
||||
|
||||
<details open>
|
||||
<summary><strong>🤖 Core Agents</strong></summary>
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://docs.praison.ai/docs/agents/single) |
|
||||
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://docs.praison.ai/docs/concepts/agents) |
|
||||
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://docs.praison.ai/docs/features/autoagents) |
|
||||
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://docs.praison.ai/docs/concepts/reflection) |
|
||||
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://docs.praison.ai/docs/features/reasoning) |
|
||||
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://docs.praison.ai/docs/features/multimodal) |
|
||||
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://praison.ai/docs/agents/single) |
|
||||
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://praison.ai/docs/concepts/agents) |
|
||||
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://praison.ai/docs/features/autoagents) |
|
||||
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://praison.ai/docs/concepts/reflection) |
|
||||
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://praison.ai/docs/features/reasoning) |
|
||||
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://praison.ai/docs/features/multimodal) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -743,16 +899,16 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://docs.praison.ai/docs/features/routing) |
|
||||
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
|
||||
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
|
||||
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
|
||||
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
|
||||
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://praison.ai/docs/features/routing) |
|
||||
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://praison.ai/docs/features/parallelisation) |
|
||||
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://praison.ai/docs/features/repetitive) |
|
||||
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
|
||||
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -761,13 +917,13 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://docs.praison.ai/docs/features/codeagent) |
|
||||
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://docs.praison.ai/docs/code/editing) |
|
||||
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://docs.praison.ai/docs/code/external-agents) |
|
||||
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://docs.praison.ai/docs/code/claude-code) |
|
||||
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://docs.praison.ai/docs/code/gemini-cli) |
|
||||
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://docs.praison.ai/docs/code/codex-cli) |
|
||||
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://docs.praison.ai/docs/code/cursor-cli) |
|
||||
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://praison.ai/docs/features/codeagent) |
|
||||
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://praison.ai/docs/code/editing) |
|
||||
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://praison.ai/docs/code/external-agents) |
|
||||
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://praison.ai/docs/code/claude-code) |
|
||||
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://praison.ai/docs/code/gemini-cli) |
|
||||
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://praison.ai/docs/code/codex-cli) |
|
||||
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://praison.ai/docs/code/cursor-cli) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -776,18 +932,18 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
|
||||
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
|
||||
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://docs.praison.ai/docs/features/claude-memory-tool) |
|
||||
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://docs.praison.ai/docs/concepts/knowledge) |
|
||||
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://docs.praison.ai/docs/concepts/rag) |
|
||||
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://docs.praison.ai/docs/features/chat-with-pdf) |
|
||||
| Data Readers (PDF, DOCX, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/chunking-strategies) |
|
||||
| Vector Store Selection | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/knowledge-backends) |
|
||||
| Retrieval Strategies | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval-strategies) |
|
||||
| Rerankers | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/smart-retrieval) |
|
||||
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/incremental-indexing) |
|
||||
| Query Engines (Sub-Question, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval) |
|
||||
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
|
||||
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
|
||||
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://praison.ai/docs/features/claude-memory-tool) |
|
||||
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://praison.ai/docs/concepts/knowledge) |
|
||||
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://praison.ai/docs/concepts/rag) |
|
||||
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://praison.ai/docs/features/chat-with-pdf) |
|
||||
| Data Readers (PDF, DOCX, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/chunking-strategies) |
|
||||
| Vector Store Selection | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/knowledge-backends) |
|
||||
| Retrieval Strategies | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval-strategies) |
|
||||
| Rerankers | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/smart-retrieval) |
|
||||
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/incremental-indexing) |
|
||||
| Query Engines (Sub-Question, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -796,12 +952,12 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://docs.praison.ai/docs/agents/deep-research) |
|
||||
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://docs.praison.ai/docs/agents/query-rewriter) |
|
||||
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/agents/websearch) |
|
||||
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/tools/tavily) |
|
||||
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://docs.praison.ai/docs/tools/web-search) |
|
||||
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
|
||||
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://praison.ai/docs/agents/deep-research) |
|
||||
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://praison.ai/docs/agents/query-rewriter) |
|
||||
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/agents/websearch) |
|
||||
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/tools/tavily) |
|
||||
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://praison.ai/docs/tools/web-search) |
|
||||
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -810,12 +966,12 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
|
||||
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
|
||||
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
|
||||
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://docs.praison.ai/docs/features/promptchaining) |
|
||||
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
|
||||
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://docs.praison.ai/docs/concepts/orchestration) |
|
||||
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
|
||||
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
|
||||
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
|
||||
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://praison.ai/docs/features/promptchaining) |
|
||||
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
|
||||
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://praison.ai/docs/concepts/orchestration) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -824,15 +980,15 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://docs.praison.ai/docs/agents/data-analyst) |
|
||||
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://docs.praison.ai/docs/agents/finance) |
|
||||
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://docs.praison.ai/docs/agents/shopping) |
|
||||
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://docs.praison.ai/docs/agents/recommendation) |
|
||||
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://docs.praison.ai/docs/agents/wikipedia) |
|
||||
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://docs.praison.ai/docs/agents/programming) |
|
||||
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://docs.praison.ai/docs/features/mathagent) |
|
||||
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://docs.praison.ai/docs/agents/markdown) |
|
||||
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://docs.praison.ai/docs/agents/prompt-expander) |
|
||||
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://praison.ai/docs/agents/data-analyst) |
|
||||
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://praison.ai/docs/agents/finance) |
|
||||
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://praison.ai/docs/agents/shopping) |
|
||||
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://praison.ai/docs/agents/recommendation) |
|
||||
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://praison.ai/docs/agents/wikipedia) |
|
||||
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://praison.ai/docs/agents/programming) |
|
||||
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://praison.ai/docs/features/mathagent) |
|
||||
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://praison.ai/docs/agents/markdown) |
|
||||
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://praison.ai/docs/agents/prompt-expander) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -841,10 +997,10 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://docs.praison.ai/docs/features/image-generation) |
|
||||
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://docs.praison.ai/docs/agents/image-to-text) |
|
||||
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://docs.praison.ai/docs/agents/video) |
|
||||
| Camera Integration | [Example](examples/python/camera/) | [📖](https://docs.praison.ai/docs/features/camera-integration) |
|
||||
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://praison.ai/docs/features/image-generation) |
|
||||
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://praison.ai/docs/agents/image-to-text) |
|
||||
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://praison.ai/docs/agents/video) |
|
||||
| Camera Integration | [Example](examples/python/camera/) | [📖](https://praison.ai/docs/features/camera-integration) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -853,12 +1009,12 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
|
||||
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
|
||||
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Config Management | [Docs](https://docs.praison.ai/docs/cli/mcp) | [📖](https://docs.praison.ai/docs/cli/mcp) |
|
||||
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://docs.praison.ai/docs/features/langchain) |
|
||||
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://praison.ai/docs/mcp/transports) |
|
||||
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://praison.ai/docs/mcp/transports) |
|
||||
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
|
||||
| MCP Config Management | [Docs](https://praison.ai/docs/cli/mcp) | [📖](https://praison.ai/docs/cli/mcp) |
|
||||
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://praison.ai/docs/features/langchain) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -867,9 +1023,9 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://docs.praison.ai/docs/concepts/guardrails) |
|
||||
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://docs.praison.ai/docs/concepts/approval) |
|
||||
| Rules & Instructions | [Docs](https://docs.praison.ai/docs/features/rules) | [📖](https://docs.praison.ai/docs/features/rules) |
|
||||
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://praison.ai/docs/concepts/guardrails) |
|
||||
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://praison.ai/docs/concepts/approval) |
|
||||
| Rules & Instructions | [Docs](https://praison.ai/docs/features/rules) | [📖](https://praison.ai/docs/features/rules) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -878,16 +1034,16 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://docs.praison.ai/docs/features/async) |
|
||||
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
|
||||
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
|
||||
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://docs.praison.ai/docs/concepts/handoffs) |
|
||||
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://docs.praison.ai/docs/features/stateful-agents) |
|
||||
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://docs.praison.ai/docs/concepts/autonomy) |
|
||||
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://docs.praison.ai/docs/features/structured) |
|
||||
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://docs.praison.ai/docs/features/model-router) |
|
||||
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
|
||||
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://docs.praison.ai/docs/features/fast-context) |
|
||||
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://praison.ai/docs/features/async) |
|
||||
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://praison.ai/docs/features/parallelisation) |
|
||||
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://praison.ai/docs/features/repetitive) |
|
||||
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://praison.ai/docs/concepts/handoffs) |
|
||||
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://praison.ai/docs/features/stateful-agents) |
|
||||
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://praison.ai/docs/concepts/autonomy) |
|
||||
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://praison.ai/docs/features/structured) |
|
||||
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://praison.ai/docs/features/model-router) |
|
||||
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
|
||||
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://praison.ai/docs/features/fast-context) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -896,21 +1052,21 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://docs.praison.ai/docs/tools/tools) |
|
||||
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://docs.praison.ai/docs/developers/agents-playbook) |
|
||||
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://docs.praison.ai/docs/models) |
|
||||
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
|
||||
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
|
||||
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://docs.praison.ai/docs/features/middleware) |
|
||||
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://docs.praison.ai/docs/features/configurable-model) |
|
||||
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://docs.praison.ai/docs/features/rate-limiter) |
|
||||
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://docs.praison.ai/docs/features/injected-state) |
|
||||
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://docs.praison.ai/docs/features/checkpoints) |
|
||||
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://docs.praison.ai/docs/features/background-tasks) |
|
||||
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://docs.praison.ai/docs/features/policy-engine) |
|
||||
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://docs.praison.ai/docs/features/thinking-budgets) |
|
||||
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://docs.praison.ai/docs/features/output-styles) |
|
||||
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://docs.praison.ai/docs/features/context-compaction) |
|
||||
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://praison.ai/docs/tools/tools) |
|
||||
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://praison.ai/docs/developers/agents-playbook) |
|
||||
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://praison.ai/docs/models) |
|
||||
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://praison.ai/docs/concepts/hooks) |
|
||||
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://praison.ai/docs/concepts/hooks) |
|
||||
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://praison.ai/docs/features/middleware) |
|
||||
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://praison.ai/docs/features/configurable-model) |
|
||||
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://praison.ai/docs/features/rate-limiter) |
|
||||
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://praison.ai/docs/features/injected-state) |
|
||||
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://praison.ai/docs/features/checkpoints) |
|
||||
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://praison.ai/docs/features/background-tasks) |
|
||||
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://praison.ai/docs/features/policy-engine) |
|
||||
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://praison.ai/docs/features/thinking-budgets) |
|
||||
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://praison.ai/docs/features/output-styles) |
|
||||
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://praison.ai/docs/features/context-compaction) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -919,14 +1075,14 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://docs.praison.ai/docs/concepts/session-management) |
|
||||
| Auto-Save Sessions | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
|
||||
| History in Context | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
|
||||
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://docs.praison.ai/docs/features/telemetry) |
|
||||
| Langfuse Tracing | [Docs](https://docs.praison.ai/docs/observability/langfuse) | [📖](https://docs.praison.ai/docs/observability/langfuse) |
|
||||
| Project Docs (.praison/docs/) | [Docs](https://docs.praison.ai/docs/cli/docs) | [📖](https://docs.praison.ai/docs/cli/docs) |
|
||||
| AI Commit Messages | [Docs](https://docs.praison.ai/docs/cli/commit) | [📖](https://docs.praison.ai/docs/cli/commit) |
|
||||
| @Mentions in Prompts | [Docs](https://docs.praison.ai/docs/cli/mentions) | [📖](https://docs.praison.ai/docs/cli/mentions) |
|
||||
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://praison.ai/docs/concepts/session-management) |
|
||||
| Auto-Save Sessions | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
|
||||
| History in Context | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
|
||||
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://praison.ai/docs/features/telemetry) |
|
||||
| Langfuse Tracing | [Docs](https://praison.ai/docs/observability/langfuse) | [📖](https://praison.ai/docs/observability/langfuse) |
|
||||
| Project Docs (.praison/docs/) | [Docs](https://praison.ai/docs/cli/docs) | [📖](https://praison.ai/docs/cli/docs) |
|
||||
| AI Commit Messages | [Docs](https://praison.ai/docs/cli/commit) | [📖](https://praison.ai/docs/cli/commit) |
|
||||
| @Mentions in Prompts | [Docs](https://praison.ai/docs/cli/mentions) | [📖](https://praison.ai/docs/cli/mentions) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -935,20 +1091,20 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://docs.praison.ai/docs/cli/slash-commands) |
|
||||
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://docs.praison.ai/docs/cli/autonomy-modes) |
|
||||
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://docs.praison.ai/docs/cli/cost-tracking) |
|
||||
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://docs.praison.ai/docs/cli/repo-map) |
|
||||
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://docs.praison.ai/docs/cli/interactive-tui) |
|
||||
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://docs.praison.ai/docs/cli/git-integration) |
|
||||
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://docs.praison.ai/docs/cli/sandbox-execution) |
|
||||
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://docs.praison.ai/docs/cli/compare) |
|
||||
| Profile/Benchmark | [Docs](https://docs.praison.ai/docs/cli/profile) | [📖](https://docs.praison.ai/docs/cli/profile) |
|
||||
| Auto Mode | [Docs](https://docs.praison.ai/docs/cli/auto) | [📖](https://docs.praison.ai/docs/cli/auto) |
|
||||
| Init | [Docs](https://docs.praison.ai/docs/cli/init) | [📖](https://docs.praison.ai/docs/cli/init) |
|
||||
| File Input | [Docs](https://docs.praison.ai/docs/cli/file-input) | [📖](https://docs.praison.ai/docs/cli/file-input) |
|
||||
| Final Agent | [Docs](https://docs.praison.ai/docs/cli/final-agent) | [📖](https://docs.praison.ai/docs/cli/final-agent) |
|
||||
| Max Tokens | [Docs](https://docs.praison.ai/docs/cli/max-tokens) | [📖](https://docs.praison.ai/docs/cli/max-tokens) |
|
||||
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://praison.ai/docs/cli/slash-commands) |
|
||||
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://praison.ai/docs/cli/autonomy-modes) |
|
||||
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://praison.ai/docs/cli/cost-tracking) |
|
||||
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://praison.ai/docs/cli/repo-map) |
|
||||
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://praison.ai/docs/cli/interactive-tui) |
|
||||
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://praison.ai/docs/cli/git-integration) |
|
||||
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://praison.ai/docs/cli/sandbox-execution) |
|
||||
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://praison.ai/docs/cli/compare) |
|
||||
| Profile/Benchmark | [Docs](https://praison.ai/docs/cli/profile) | [📖](https://praison.ai/docs/cli/profile) |
|
||||
| Auto Mode | [Docs](https://praison.ai/docs/cli/auto) | [📖](https://praison.ai/docs/cli/auto) |
|
||||
| Init | [Docs](https://praison.ai/docs/cli/init) | [📖](https://praison.ai/docs/cli/init) |
|
||||
| File Input | [Docs](https://praison.ai/docs/cli/file-input) | [📖](https://praison.ai/docs/cli/file-input) |
|
||||
| Final Agent | [Docs](https://praison.ai/docs/cli/final-agent) | [📖](https://praison.ai/docs/cli/final-agent) |
|
||||
| Max Tokens | [Docs](https://praison.ai/docs/cli/max-tokens) | [📖](https://praison.ai/docs/cli/max-tokens) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -957,10 +1113,10 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
|
||||
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://praison.ai/docs/cli/eval) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -969,8 +1125,8 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
|
||||
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
|
||||
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://praison.ai/docs/concepts/skills) |
|
||||
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://praison.ai/docs/concepts/skills) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -979,7 +1135,9 @@ praisonai agents.yaml
|
||||
|
||||
| Feature | Code | Docs |
|
||||
|---------|:----:|:----:|
|
||||
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://docs.praison.ai/docs/cli/scheduler) |
|
||||
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://praison.ai/docs/cli/scheduler) |
|
||||
|
||||
</details>
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1008,7 +1166,7 @@ PraisonAI keeps framework overhead microsecond-class: creating an `Agent` (one t
|
||||
|--------------------|-----------|----------|
|
||||
| Avg Agent instantiation (1 tool, 1000 iterations) | **46 μs** | 2026-10-15 · praisonaiagents 1.7.12 |
|
||||
|
||||
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://docs.praison.ai/docs/developers/benchmarks)
|
||||
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://praison.ai/docs/developers/benchmarks)
|
||||
|
||||
---
|
||||
|
||||
@@ -1018,7 +1176,7 @@ Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md]
|
||||
|
||||
## ⭐ Star History
|
||||
|
||||
[](https://docs.praison.ai)
|
||||
[](https://praison.ai/docs)
|
||||
|
||||
---
|
||||
|
||||
@@ -1089,6 +1247,9 @@ We welcome contributions! Fork the repo, create a branch, and submit a PR → [C
|
||||
|
||||
## ❓ FAQ & Troubleshooting
|
||||
|
||||
<details>
|
||||
<summary><strong>Show FAQ and troubleshooting</strong></summary>
|
||||
|
||||
<details>
|
||||
<summary><strong>ModuleNotFoundError: No module named 'praisonaiagents'</strong></summary>
|
||||
|
||||
@@ -1107,7 +1268,7 @@ Ensure your API key is set:
|
||||
export OPENAI_API_KEY=your_key_here
|
||||
```
|
||||
|
||||
For other providers, see [Models docs](https://docs.praison.ai/docs/models).
|
||||
For other providers, see [Models docs](https://praison.ai/docs/models).
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1148,7 +1309,7 @@ agent = Agent(instructions="...", llm="ollama/llama3.2", base_url="http://192.16
|
||||
The same shape works for other local runtimes — `lm_studio/`, `vllm/` and
|
||||
`hosted_vllm/` with their server's `base_url`.
|
||||
|
||||
See [Models docs](https://docs.praison.ai/docs/models) for more details.
|
||||
See [Models docs](https://praison.ai/docs/models) for more details.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1168,7 +1329,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
See [Persistence docs](https://docs.praison.ai/docs/databases/overview) for supported databases.
|
||||
See [Persistence docs](https://praison.ai/docs/databases/overview) for supported databases.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1185,7 +1346,7 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
See [Memory docs](https://docs.praison.ai/docs/concepts/memory) for more options.
|
||||
See [Memory docs](https://praison.ai/docs/concepts/memory) for more options.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1201,7 +1362,7 @@ agents = Agents(agents=[agent1, agent2])
|
||||
agents.start()
|
||||
```
|
||||
|
||||
See [Agents docs](https://docs.praison.ai/docs/concepts/agents) for more examples.
|
||||
See [Agents docs](https://praison.ai/docs/concepts/agents) for more examples.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -1216,13 +1377,15 @@ agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport options.
|
||||
See [MCP docs](https://praison.ai/docs/mcp/transports) for all transport options.
|
||||
|
||||
</details>
|
||||
|
||||
</details>
|
||||
|
||||
### Getting Help
|
||||
|
||||
- 📚 [Full Documentation](https://docs.praison.ai)
|
||||
- 📚 [Full Documentation](https://praison.ai/docs)
|
||||
- 🐛 [Report Issues](https://github.com/MervinPraison/PraisonAI/issues)
|
||||
- 💬 [Discussions](https://github.com/MervinPraison/PraisonAI/discussions)
|
||||
|
||||
@@ -1231,7 +1394,7 @@ See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport op
|
||||
<div align="center">
|
||||
<p><strong>Made with ❤️ by the PraisonAI Team</strong></p>
|
||||
<p>
|
||||
<a href="https://docs.praison.ai">📚 Documentation</a> •
|
||||
<a href="https://praison.ai/docs">📚 Documentation</a> •
|
||||
<a href="https://github.com/MervinPraison/PraisonAI">GitHub</a> •
|
||||
<a href="https://youtube.com/@MervinPraison">▶️ YouTube</a> •
|
||||
<a href="https://x.com/MervinPraison">𝕏 X</a> •
|
||||
|
||||
@@ -10,6 +10,7 @@ from typing import Optional, Dict, Any
|
||||
|
||||
from ..diff.diff_strategy import apply_search_replace_diff
|
||||
from ..utils.file_utils import (
|
||||
atomic_write,
|
||||
file_exists,
|
||||
is_path_within_directory,
|
||||
)
|
||||
@@ -142,8 +143,7 @@ def apply_diff(
|
||||
f.write(original_content)
|
||||
|
||||
# Write the modified content
|
||||
with open(abs_path, 'w', encoding=encoding) as f:
|
||||
f.write(result.content)
|
||||
atomic_write(abs_path, result.content, encoding=encoding)
|
||||
|
||||
return {
|
||||
'success': True,
|
||||
|
||||
@@ -10,6 +10,7 @@ import re
|
||||
from typing import Optional, List, Dict, Any
|
||||
|
||||
from ..utils.file_utils import (
|
||||
atomic_write,
|
||||
file_exists,
|
||||
is_path_within_directory,
|
||||
)
|
||||
@@ -191,8 +192,7 @@ def search_replace(
|
||||
f.write(original_content)
|
||||
|
||||
# Write the modified content
|
||||
with open(abs_path, 'w', encoding=encoding) as f:
|
||||
f.write(content)
|
||||
atomic_write(abs_path, content, encoding=encoding)
|
||||
|
||||
return {
|
||||
'success': True,
|
||||
|
||||
@@ -8,6 +8,7 @@ import os
|
||||
from typing import Optional, Dict, Any
|
||||
|
||||
from ..utils.file_utils import (
|
||||
atomic_write,
|
||||
file_exists,
|
||||
create_directories_for_file,
|
||||
is_path_within_directory,
|
||||
@@ -131,8 +132,7 @@ def write_file(
|
||||
pass # If we can't read, just use the content as-is
|
||||
|
||||
# Write the file
|
||||
with open(abs_path, 'w', encoding=encoding) as f:
|
||||
f.write(processed_content)
|
||||
atomic_write(abs_path, processed_content, encoding=encoding)
|
||||
|
||||
return {
|
||||
'success': True,
|
||||
|
||||
Loaded 100 of 111 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user