Compare commits

...
Author SHA1 Message Date
MervinPraisonandClaude Opus 5.5 0b8cd48c16 fix(gateway): stop GitGuardian flagging the DeviceAuth lazy-import tuple
GitGuardian reads ("praisonaiagents.gateway.protocols", "DeviceAuth") as an
'Authentication Tuple'. Every PR that merges main carries that line and fails
the GitGuardian check, which the merge gate treats as failed CI. Use a
module-path constant instead; DeviceAuth still resolves lazily.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-05 09:03:46 +00:00
praisonai-triage-agent[bot] 20d023a864 Merge pull request #6247 from MervinPraison/docs/readme-exam-wording
docs(readme): say what the handover test tests
2026-11-05 08:57:04 +00:00
praisonai-triage-agent[bot] b05518baed Merge pull request #6246 from MervinPraison/claude/issue-6245-20261105
fix: fail loudly on unknown provider prefix and dropped safety YAML fields
2026-11-05 08:49:45 +00:00
MervinPraisonandClaude Opus 5.5 740a887ade docs(readme): say what the handover test tests
Step 2 now says the model takes an exam made from real past work it never
studied, matching the website and docs.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-05 08:43:59 +00:00
MervinPraisonandClaude Opus 5.5 6980ec13e2 fix: only fail closed on active safety gates (keep --trust working on crewai/autogen)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:38:13 +00:00
praisonai-triage-agent[bot] a362afc899 Merge pull request #6239 from MervinPraison/claude/issue-6238-20261105
fix: confine caller-supplied names to storage dirs and use shared async bridge
2026-11-05 08:31:38 +00:00
praisonai-triage-agent[bot] 4d9b514106 Merge pull request #6243 from MervinPraison/claude/issue-6241-20261105
fix: accept OpenAI-style 'messages' chat datasets in praisonai-train llm
2026-11-05 08:27:30 +00:00
praisonai-triage-agent[bot] d3ecf23533 Merge pull request #6244 from MervinPraison/claude/issue-6242-20261105
fix: add --no-push / ollama_push: false to keep Ollama exports local
2026-11-05 08:27:13 +00:00
MervinPraisonandClaude Opus 5.5 85922f349d fix: fail loudly on unknown provider prefix and dropped safety fields (fixes #6245)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:17:57 +00:00
MervinPraisonandClaude Opus 5.5 50a0949063 fix: bound safe_join filename length and keep non-ASCII names unescaped (#6238)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:16:24 +00:00
MervinPraisonandClaude Opus 5.5 8da80d1890 fix: let explicit --push override config ollama_push and test caller paths (#6242)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:09:17 +00:00
MervinPraisonandClaude Opus 5.5 861584bab4 fix: mention 'messages' chat shape in empty-dataset error
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:09:02 +00:00
MervinPraisonandClaude Opus 5.5 79669e7a7d fix: reversible safe_join encoding and stop tests leaking a global ConsoleBackend (#6238)
- safe_join now percent-encodes names (bijective), so distinct names such as
  'a/b' and 'a_b~c14cddc033f6' can no longer share one file; list_todos()
  decodes filenames so listed names round-trip through load/delete_todo.
- Legacy <name>.json fallback skips names containing '%' to avoid aliasing.
- Tests restoring the approval backend snapshotted get_backend(), whose
  ConsoleBackend fallback was then installed globally on restore and leaked
  into tests/unit/skills/test_approval_scoping.py under xdist. Snapshot
  _global_backend directly instead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:08:46 +00:00
25f8d93a4c docs(readme): say what "your own AI" is in the handover section (#6240)
* docs(readme): say what "your own AI" is in the handover section

The five handover steps never said where the own model comes from. Add one
line: an open model you run yourself, used as it is or trained on your own
past work with PraisonAI Train, and only after it passes the test.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix: match handover wording to the Test step ("its tests")

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
2026-11-05 08:05:23 +00:00
MervinPraisonandClaude Opus 5.5 ffa3068203 fix: add --no-push / ollama_push: false to keep Ollama exports local (fixes #6242)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:03:53 +00:00
MervinPraisonandClaude Opus 5.5 82d066299b fix: accept OpenAI-style 'messages' chat datasets in praisonai-train llm (fixes #6241)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 08:03:21 +00:00
Elon-Jiang 87bf746dba fix(ts): release hook deadline timers after settlement (#6195) 2026-11-05 08:59:33 +01:00
Wu Shuwen 2b78ce8804 fix: discover compute backends on first placement lookup (#6237) 2026-11-05 08:59:26 +01:00
Wu Shuwen c244c4db7e test: read Python source consistently on Windows (#6235)
* test: read Python source using its declared encoding

* test: read Python source using its declared encoding

* test: read Python source using its declared encoding
2026-11-05 08:59:21 +01:00
MervinPraisonandClaude Opus 5.5 42e0695bbe fix: keep listed/legacy todo names and custom plan filenames loadable (#6238)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 07:46:30 +00:00
MervinPraisonandClaude Opus 5.5 f024f6b3a1 fix: confine caller-supplied names to storage dirs and use shared async bridge (fixes #6238)
- utils/atomic_io.safe_join: injective, separator-free filename stems
  (same scheme as session store); legacy safe names are unchanged.
- PlanStorage save/load/delete_todo and save_plan use safe_join, so
  names like '../../x' can no longer escape todos_dir/plans_dir.
- ContextAgent creates PRPs/ and writes PRPs atomically via safe_join.
- generate_title and crawl4ai_*_sync use run_coroutine_from_any_context,
  so they work inside a running event loop instead of crashing or
  silently returning the fallback.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-05 07:29:53 +00:00
7d26272fa0 fix(ts): honor the configured FileCache TTL (#6180)
Co-authored-by: Elon-Jiang <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-04 21:33:53 +01:00
Elon-Jiang 85c01743c8 fix(ts): re-enter downstream tool middleware on retry (#6196)
* fix(ts): re-enter downstream tool middleware on retry

* fix(tools): stop middleware retries after cancellation
2026-11-04 21:04:36 +01:00
Elon-Jiang 8420d1f93f fix(ts): preserve sentence chunk content and source offsets (#6188) 2026-11-04 21:04:31 +01:00
Elon-JiangandElon-Jiang 4c59b795d5 fix(ts): settle MCP requests when WebSocket connections close (#6184)
Co-authored-by: Elon-Jiang <[email protected]>
2026-11-04 21:04:25 +01:00
Wu Shuwen 790878b092 test: include agent root regression tests in core shard (#6233) 2026-11-04 20:58:23 +01:00
MervinPraison 9a84e99ecf docs: auto-update feature parity trackers [skip ci] 2026-11-04 17:40:37 +00:00
praisonai-triage-agent[bot] fa738028d1 Merge pull request #6232 from MervinPraison/fix/edit-hint-nested-tool-dicts
fix: count only tool dicts in nested lists for the edit hint
2026-11-04 17:39:50 +00:00
MervinPraisonandClaude Opus 5.5 71de7217fd fix: hide permission-denied tools from bridge tool_search/tool_describe
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 16:36:23 +00:00
MervinPraisonandClaude Opus 5.5 3d7ed12a3c fix: cache unpruned formatted tools so lifted denials take effect
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 15:34:37 +00:00
MervinPraisonandClaude Opus 5.5 45019589cf fix: re-prune permission-denied tools on formatted-tools cache hits
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 14:36:43 +00:00
praisonai-triage-agent[bot] e3074e5ac1 Merge pull request #6229 from MervinPraison/claude/issue-6228-20261104
fix: CLI keyless explicit ollama/ models and tolerant local probe
2026-11-04 13:53:48 +00:00
MervinPraisonandClaude Opus 5.5 09c443a71f fix: count only function-schema dicts in nested lists for the edit hint
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 13:48:09 +00:00
MervinPraisonandClaude Opus 5.5 a0ad968af0 fix: skip the edit hint for permission-denied edit tools
_prune_denied_tools drops denied tools from the payload, so the hint must
not name them. Tests now record the formatted tool payload, which also
checks that a nested edit schema is actually sent.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 13:39:58 +00:00
Mervin PraisonandClaude Opus 5.5 70e3b7cc70 docs(ts): finish the repositioning — same main line as the README, praison.ai/docs link (#6231)
* docs(ts): match the README's main line; link praison.ai/docs

Keeps the hosted-model privacy qualifier from the review fix.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(ts): 'never has to leave' to match README and docs

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-04 13:19:29 +00:00
1b3f576e00 docs(readme): finish the repositioning — regulated teams and everyone, pip install praisonai, praison.ai/docs links (#6230)
* docs(readme): pitch to regulated teams and everyone; pip install praisonai; praison.ai/docs links

- Top: one main line, then one line for highly regulated teams (the handover,
  early access) and one for everyone else (free, open source, no API key with
  a local model). Numbers now link to their public sources.
- Install line is pip install praisonai (praisonaiagents noted as the light core).
  The offline section no longer needs the [llm] extra: praisonai ships litellm.
- How the handover works: adds why now and what it is built on today.
- New 'The company' section for investors and partners: who, what, how the open
  framework and the licensed handover fit, traction, contact.
- All 128 docs.praison.ai links moved to praison.ai/docs after checking each
  new URL returns 200; every other link checked too.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(readme): 'never has to leave' (true for hosted models too)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(readme): investor review: label early access first, cite the OpenAI notice, add why now, co-founders and contact email; no unproven partner or release-pace claims

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix: note praisonaiagents[llm] extra for core-only offline use in both READMEs

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
2026-11-04 13:19:02 +00:00
MervinPraison 69ca4f9808 fix: count only tool dicts in nested lists for the edit hint
_format_tools_for_completion flattens a nested tool list but keeps only
its tool dicts, so tools=[["edit_file"]] sends no edit tool. Counting the
nested string told the model to use a tool it did not have.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
(cherry picked from commit 29e4d4e77f)
2026-11-04 13:18:13 +00:00
MervinPraison 8d9b32de91 docs: auto-update feature parity trackers [skip ci] 2026-11-04 13:04:25 +00:00
Elon-Jiang 318e15054c fix(ts): interrupt tool retry backoff on cancellation (#6199) 2026-11-04 14:03:52 +01:00
Elon-Jiang 6984acc862 fix(ts): enforce tool payload limits in UTF-8 bytes (#6198) 2026-11-04 14:03:47 +01:00
Elon-Jiang a563e08bf1 fix(ts): invalidate tool installation cache on registration changes (#6197) 2026-11-04 14:03:41 +01:00
Elon-Jiang 636586d7df fix(ts): keep hook inspection from freezing registration (#6194) 2026-11-04 14:03:35 +01:00
MervinPraisonandClaude Opus 5.5 5baea8dc57 fix: gate explicit Ollama agents on their declared base_url, not OLLAMA_HOST
Addresses review on #6229: run --agent with llm.base_url probed the env/default
host instead of the server the agent actually calls.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 13:03:30 +00:00
Elon-Jiang 26b93f9201 fix(ts): preserve concurrent FileCache list appends (#6193)
* fix(ts): preserve concurrent FileCache list appends

* test: cover FileCache appends queued behind a failed write
2026-11-04 14:03:30 +01:00
Elon-Jiang d6b8d4ff11 fix(ts): remove stale graph neighbors when an edge is rewired (#6191) 2026-11-04 14:03:25 +01:00
Elon-Jiang 534b6e3535 fix(ts): preserve session changes during file saves (#6186)
* fix(ts): preserve session changes during file saves

* fix(ts): avoid redundant autosave queue entries
2026-11-04 14:03:19 +01:00
yunyv 2e2081b38c fix(memory): enforce long-term capacity during promotion (#6185)
* fix(memory): enforce long-term capacity during promotion

* test(memory): cover bounded automatic promotion
2026-11-04 14:03:14 +01:00
Elon-JiangandElon-Jiang 0c5f2998f0 fix(ts): evict empty keys from bounded MemoryCache (#6183)
Co-authored-by: Elon-Jiang <[email protected]>
2026-11-04 14:03:08 +01:00
yunyv eca66b1efc fix(tools): match zero directories in grep fallback recursive globs (#6182)
* fix(tools): match zero directories in grep fallback recursive globs

* test(tools): cover zero-directory recursive grep globs
2026-11-04 14:03:02 +01:00
Elon-JiangandElon-Jiang bdd3d1655d fix(ts): resolve MCP WebSocket responses with zero request IDs (#6178)
Co-authored-by: Elon-Jiang <[email protected]>
2026-11-04 14:02:57 +01:00
sarankumar 82b776fa66 fix(standardise): escape braces in the tools basic example template (#6166)
* fix(standardise): escape braces in the tools basic example template

_get_feature_basic_structure builds every basic example structure as an
f-string. The tools template embeds a calculator whose code has literal
braces (the ops dict, {type(node)} and {e}), which were evaluated as
replacement fields when the dict was built. That raised TypeError
(unsupported format string passed to type.__format__) for every feature,
so basic example prompt generation always failed.

Double the braces so they render literally, and add regression tests.

Signed-off-by: sarankumar1325 <[email protected]>

* test(standardise): assert the tools example has no doubled braces

Signed-off-by: sarankumar1325 <[email protected]>

* fix(standardise): accept only numeric constants in the tools example calculator

A string constant such as 'x' * 1000000000 made the generated calculator
allocate a huge string. Reject non-numeric constants; the existing except
clause turns the TypeError into an error string. Tighten the calculator
test to check an operator entry and execute the generated calculator.

Signed-off-by: sarankumar1325 <[email protected]>

* fix(standardise): bound the tools example calculator's results

Catch OverflowError, reject integer powers whose result would exceed
4096 bits, and treat bools as non-numeric. The test also checks that the
example registers the calculator with tools=[calculator].

Signed-off-by: sarankumar1325 <[email protected]>

* fix(standardise): use an exact bound in the calculator power guard

The guard multiplied abs(left).bit_length() by the exponent, so it
rejected results that fit in 4096 bits (2 ** 3000, 1 ** 5000). Reject
only powers whose lower bound already exceeds the limit, then check the
real result size. The test now checks the Agent call's tools keyword
via the AST instead of a substring.

Signed-off-by: sarankumar1325 <[email protected]>

* fix(standardise): cap bare integer literals in calculator example

Apply the 4096-bit limit to integer constants too, so a huge literal is
rejected like an oversized operation result. Tighten the tools assertion
to require exactly [calculator] with no non-name entries.

Signed-off-by: sarankumar1325 <[email protected]>

---------

Signed-off-by: sarankumar1325 <[email protected]>
2026-11-04 14:02:52 +01:00
sarankumar 9122182753 fix(agent): generate final response after tool calls in _achat_completion (#6165)
* fix(agent): generate final response after tool calls in _achat_completion

The legacy async tool path referenced an undefined `stream` name and
`_get_llm_functions` (defined only in agent.py), and called
`self.console.print()` although `console` is None unless verbose. The
follow-up completion therefore always failed, the error was swallowed and
the raw tool output was returned instead of the model's final answer.

Request a streamed completion (the code consumes it with `async for`),
import the lazy LLM helper from .agent, and only print when a console is
available. Adds a regression test.

Signed-off-by: sarankumar1325 <[email protected]>

* fix(agent): keep the conversation in the post-tool follow-up request

_achat_completion sent only the system prompt and the tool results, so the
final answer never saw the user's question. Pass the original messages
through and append the tool results to them. Also skip stream chunks with
no choices (e.g. Azure content-filter annotations) instead of raising
IndexError, and add a live integration test (skipped without OPENAI_API_KEY).

Signed-off-by: sarankumar1325 <[email protected]>

* fix(agent): fall back to tool results when the follow-up stream has no answer

If every chunk of the post-tool follow-up stream has empty choices,
raise so the existing handler logs the error and returns the tool
results instead of an empty answer.

Tests: cover the legacy _achat_impl branch forwarding the conversation
to _achat_completion, and the no-answer-chunk fallback. The live test
now disables unified dispatch so it actually reaches _achat_completion,
and asserts the answer is model-written rather than the raw tool output.

Signed-off-by: sarankumar1325 <[email protected]>

* fix(agent): fall back to tool results when the follow-up has no answer text

The guard only checked whether any chunk had choices, so a follow-up
stream whose choices carry no delta.content returned an empty string
instead of the tool results. Fall back whenever neither answer text nor
(enabled) reasoning content was produced, and cover the contentless
choice case in the unit test.

Signed-off-by: sarankumar1325 <[email protected]>

* fix(agent): treat a whitespace-only follow-up stream as no answer

A follow-up stream that only yields whitespace passed the empty-text check
and was returned as the final response. Strip the accumulated text and
reasoning before the check so the tool-result fallback runs.

Signed-off-by: sarankumar1325 <[email protected]>

---------

Signed-off-by: sarankumar1325 <[email protected]>
2026-11-04 14:02:46 +01:00
yi-coding 2f5e07092c fix(ts): honor explicit zero memory history limits (#6143) 2026-11-04 14:02:40 +01:00
Elon-JiangandElon-Jiang bd1f8773ce fix(ts): recover FileMemory writes after append failures (#6142)
* fix(ts): recover FileMemory writes after append failures

* fix(ts): isolate memory records after partial append failures

* fix(memory): separate unterminated records after restart

---------

Co-authored-by: Elon-Jiang <[email protected]>
2026-11-04 14:02:35 +01:00
Elon-JiangandElon-Jiang 250c463c4b fix(ts): preserve arbitrary session state keys (#6149)
* fix(ts): preserve arbitrary session state keys

* fix(ts): preserve plain-object session JSON exports

---------

Co-authored-by: Elon-Jiang <[email protected]>
2026-11-04 14:02:30 +01:00
MervinPraisonandClaude Opus 5.5 b284b67011 fix: let explicit ollama/ models run keyless and make local probe tolerant (fixes #6228)
- Gate explicit ollama/ and ollama_chat/ models on the server answering
  /api/tags (OLLAMA_API_BASE, OLLAMA_HOST, or 127.0.0.1:11434) instead of
  requiring OLLAMA_HOST as a credential.
- Unreachable Ollama now says to start it with `ollama serve` rather than
  asking for an API key.
- Raise the local probe timeout from 0.15s to 1s; a running Ollama can take
  ~0.35s on its first call.
- Only print the 'would be used automatically' hint when auto-detection
  actually ran; name the model when one was given.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 12:56:04 +00:00
praisonai-triage-agent[bot] 29dd51cde1 Merge pull request #6221 from MervinPraison/fix/benchmark-scripts-fair-runs
fix(benchmarks): warm up, interleave and drop failed runs in execution benchmark
2026-11-04 12:38:43 +00:00
01fbdf524e docs(ts): lead the npm README and description with the new focus (#6227)
* docs(ts): lead the npm README and description with the new focus

Text only: the README intro paragraph and the package.json description.
No code, exports or keywords change.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ts): qualify README privacy claim for hosted models

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
2026-11-04 12:35:24 +00:00
9f5a30790d docs(readme): lead with private, offline AI agents; keep every feature below (#6226)
* docs(readme): lead with private, offline AI agents; keep full reference below

New top: title, approved tagline and one-liner, approved numbers,
pip install praisonaiagents, "Using PraisonAI for something else?",
Quick start, Run fully offline (Ollama), How the handover works
(early access), Features, JavaScript SDK, Docs/community/licence.

All previous sections are kept under "Full feature reference" with
their headings unchanged; the longest are folded in <details>. The
hero badges (Elon, desktop downloads, Trendshift, docs) move lower;
every link target from the old README is still present.

pyproject description for praisonaiagents and praisonai updated to the
approved short line. No code, version or dependency changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(readme): apply review fixes

- vLLM: use hosted_vllm/ (vllm/ runs in-process, not a server)
- Ollama block: don't put the blocking 'ollama serve' before 'ollama pull'
- Licence link absolute so it works on pypi.org
- New JS heading renamed so the existing #javascript-sdk anchor is unchanged
- Tracing links to the Observability overview; quick starts print the answer

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix: clarify praisonai CLI install and offline model pre-download in README

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
2026-11-04 12:35:08 +00:00
MervinPraisonandClaude Opus 5.5 dc35781e2f fix(benchmarks): rotate framework order each timed round
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 12:24:53 +00:00
MervinPraisonandClaude Opus 5.5 6713492023 fix(benchmarks): reuse Agno's model object, validate rounds, test the rules
- Each Agno OpenAIChat opens its own HTTP client, so a fresh one per round
  put a new connection inside the timer, while PraisonAI and CrewAI agents
  share theirs. One model object per runner (new Agent per round as before).
  Measured: Agno median 0.96 s -> 0.63 s.
- --iterations must be at least 1 and --warmup 0 or more.
- tests/unit/test_execution_benchmark_rules.py checks the counting rules
  with fake runners (no API key): warm-ups untimed and rounds interleaved,
  errors and empty answers are failures, a failed warm-up keeps the
  framework, ranking by median, only the call is timed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-04 12:23:33 +00:00
MervinPraison 16f148d6c2 docs: auto-update feature parity trackers [skip ci] 2026-11-04 12:18:10 +00:00
praisonai-triage-agent[bot] 3e314cf09b Merge pull request #6220 from MervinPraison/fix/system-prompt-only-relevant-hints
fix(agent): add the file-edit hint only when an edit tool is attached
2026-11-04 12:17:29 +00:00
praisonai-triage-agent[bot] 46263b2d76 Merge pull request #6225 from MervinPraison/claude/issue-6223-20261104
fix: add model-isolated 'secret' gateway request kind
2026-11-04 12:10:08 +00:00
MervinPraison e05b16b366 docs: auto-update feature parity trackers [skip ci] 2026-11-04 12:01:04 +00:00
praisonai-triage-agent[bot] 81aace69ec Merge pull request #6219 from MervinPraison/fix/litellm-route-duplicate-prompt
fix(agent): send the user turn once on the custom-LLM path
2026-11-04 12:00:16 +00:00
MervinPraisonandClaude Opus 5.5 d67a39ec40 fix: mention secret kind in server_reply handler comments (#6223)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 11:47:45 +00:00
MervinPraisonandClaude Opus 5.5 bcba4110ff fix: add model-isolated 'secret' gateway request kind (fixes #6223)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 11:36:40 +00:00
MervinPraisonandClaude Opus 5.5 4c3eecfb8f fix(benchmarks): time only the execution call and keep frameworks after a failed warm-up
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 11:34:01 +00:00
MervinPraisonandClaude Opus 5.5 d41f427f59 fix: detect edit tools inside nested tool lists for the edit hint
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-04 11:18:20 +00:00
MervinPraisonandClaude Opus 5.5 d1071a2ed9 fix(agent): add the file-edit hint only when an edit tool is attached
The model-family harness fragment ("When editing files, prefer targeted
string-replacement edits: use edit_file ..." for GPT, an apply_patch
hint for Claude) was prepended to every agent's system prompt, tools or
not. It now applies only when the tools for the call include edit_file
or apply_patch (by name: strings, OpenAI-style dicts, @tool(name=...),
BaseTool, functions). An explicit harness_base_prompt is still always
used.

- chat()/achat()/streamed start() on the native OpenAI path now pass the
  per-call tools to _build_messages, as the custom-LLM path already did,
  so the prompt follows the tools actually sent
- the system-prompt cache key includes the edit-tool flag and backstory,
  so a cached prefix follows changes to agent.tools / agent.backstory

Agent(instructions=X) sets backstory and goal to X, so the prompt said X
twice ("X / Your Role: Assistant / Your Goal: X"); the Goal line is now
left out when it repeats the backstory, in _build_system_prompt and in
the agent.system_prompt attribute.

A tool-less gpt-4o-mini agent's request drops from 149 to 123 prompt
tokens.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-04 10:45:10 +00:00
MervinPraisonandClaude Opus 5.5 c02b02454f fix(agent): send the user turn once on the custom-LLM path
chat(), achat() and streamed start() store the user turn in chat_history
before the model call, then pass that history as chat_history= and the
same turn as prompt= to LLM.get_response*. LLM._build_messages appends
prompt after chat_history, so every litellm route (ollama/, anthropic/,
groq/, openai/, base_url=) sent the prompt twice: ['system', 'user',
'user']. Measured on gpt-4o-mini via openai/: 177 billed prompt tokens
vs 149 on the native route for the same prompt; 149 on both with this fix.
Present since at least 1.2.4.

_history_before_turn() drops that trailing stored copy from the history
forwarded to the LLM only; the local histories (written back to
chat_history on the async path and handed to guardrail retries) are
unchanged, and messages a durable replay restores after the turn keep
their order.

test_stream_context_management_custom_llm counted the duplicate as part
of the history; it now expects the six earlier turns plus the prompt.

With Anthropic prompt caching, the history cache breakpoint now lands
where the other callers put it (it used to count the duplicate).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-04 10:45:01 +00:00
MervinPraisonandClaude Opus 5.5 dde474cc91 fix(benchmarks): warm up, interleave and drop failed runs in execution benchmark
execution_benchmark.py timed a framework's first call (cold imports and
the first connection), ran each framework's runs back to back so network
drift landed on one framework, averaged by mean over 3 runs, and timed a
run that returned None as if it had answered: with litellm broken, every
LiteLLM run returned None and the script reported a 15 s "average".

- one untimed warm-up call per framework (--warmup, default 1)
- timed runs interleaved round by round; default 10 rounds
- a run that raises or returns an empty answer is a failure, not a
  timing; Agno reports a failed run in RunOutput.status with the error
  text as content, so that status counts as a failure too
- summary and saved table report median, mean and ok/failed counts
- docstring no longer advertises a --no-save flag that never existed

simple_benchmark.py's "without tools" run built LangGraph and CrewAI
with a tool; they are now tool-free like every other framework there.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-11-04 10:35:31 +00:00
4e419c4f05 fix: keep tool turns intact across auto-save, compaction and history mutators (#6218)
* fix: keep tool turns intact across auto-save, compaction and history mutators (fixes #6217)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: save messages appended by history replacement and serialize it with auto-save (#6217)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 11:20:43 +01:00
7ae123e68d fix: deprecate You.com and Jira connectors in core in favour of PraisonAI-Tools (fixes #6168) (#6169)
Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 11:20:36 +01:00
Wu Shuwen f8b1f3ae6c fix: surface weekly package test failures (#6189) 2026-11-04 11:17:10 +01:00
Wu Shuwen 8f1d50b5ff test: include four core unit regression files in shard (#6187)
* test: gate agent token budget unit tests

* test: include more core unit regression files
2026-11-04 11:16:59 +01:00
praisonai-triage-agent[bot] ac94b9f8b7 Merge pull request #6216 from MervinPraison/claude/issue-6215-20261103
fix: optional per-user inbound request budget at bot/gateway admission
2026-11-03 11:56:00 +00:00
MervinPraisonandClaude Opus 5.5 d01c37d434 fix: key API rate budget only on a validated bearer, never fall back to client-chosen session id (#6215)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 11:42:45 +00:00
praisonai-triage-agent[bot] feae4f936d Merge pull request #6212 from MervinPraison/claude/issue-6208-20261103
fix: gateway single-instance guard is state-scoped, not only port-scoped
2026-11-03 11:32:32 +00:00
MervinPraisonandClaude Opus 5.5 831490316f fix: start() keeps the state lock owned by start_with_config until channels stop; make concurrent-claim test deterministic
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 11:18:32 +00:00
MervinPraisonandClaude Opus 5.5 d20762ea11 fix: stable per-caller rate key, reload-applied and validated per-user budget, bounded pruning (#6215)
- API/WebSocket turns charge the budget to a stable caller identity
  (bearer-token digest or peer IP) so a new session id cannot reset it.
- gateway.per_user_per_minute changes now take effect on config reload
  (validated before cutover; restored on rollback).
- Fractional/boolean/negative budgets are rejected instead of truncated.
- Stale-identity pruning runs at most every 5s instead of on every request.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 11:05:38 +00:00
MervinPraisonandClaude Opus 5.5 8528cf3da5 fix: enforce optional per-user inbound request budget at bot/gateway admission (fixes #6215)
Wires the existing core SlidingWindowRateLimitPolicy into the shared
AdmissionGate so a single user cannot start unbounded turns. Configured
via gateway.per_user_per_minute (YAML) or BotOS(per_user_per_minute=...);
default 0 keeps ingress unlimited.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 10:56:28 +00:00
praisonai-triage-agent[bot] f16a6971bf Merge pull request #6211 from MervinPraison/claude/issue-6210-20261103
fix: BotOS.from_config rejects gateway.yaml sections instead of silently dropping them
2026-11-03 10:45:52 +00:00
MervinPraisonandClaude Opus 5.5 de3a1df83b fix: make gateway state lock an exclusive claim taken before channel recovery and released on any startup failure
- port_utils: serialise claim with flock + exclusive link()/O_EXCL so racing starts cannot both win
- key state lock on ~/.praisonai (where the stores actually live), not PRAISONAI_HOME
- server: acquire state lock in start_with_config before start_channels (journal replay)
- server: release both locks on any failure after claim, not only serve() errors

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 10:38:46 +00:00
MervinPraisonandClaude Opus 5.5 0b4cb4db60 fix: BotOS.from_config warns instead of crashing on non-string unknown YAML keys
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 10:32:13 +00:00
MervinPraisonandClaude Opus 5.5 2c8dd09aa5 fix: lock gateway single-instance guard on shared state dir, not just host:port (fixes #6208)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 10:29:53 +00:00
MervinPraisonandClaude Opus 5.5 25729bec08 fix: BotOS.from_config rejects gateway-only YAML instead of silently dropping it (fixes #6210)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 10:28:34 +00:00
praisonai-triage-agent[bot] 688bb55d51 Merge pull request #6205 from MervinPraison/claude/issue-6204-20261103
fix: crash-safe writes in praisonai.code edit tools
2026-11-03 09:56:01 +00:00
MervinPraisonandClaude Opus 5.5 60495e8d26 fix: atomic_write refuses read-only targets like open(path, 'w') instead of replacing them via inode swap
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 09:41:23 +00:00
MervinPraisonandClaude Opus 5.5 dfa448d57b fix: create atomic_write temp file with target's permission bits so private files are never exposed mid-write
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 09:39:26 +00:00
praisonai-triage-agent[bot] b5acce53be Merge pull request #6207 from MervinPraison/claude/issue-6206-20261103
fix: expose --max-budget on praisonai loop and pass YAML execution block through
2026-11-03 09:22:23 +00:00
MervinPraisonandClaude Opus 5.5 e128c541d5 fix: surface budget stops in sync chat, keep execution in roles YAML, validate --max-budget
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 09:07:53 +00:00
MervinPraisonandClaude Opus 5.5 91f0cc7727 fix: expose max-budget on praisonai loop and pass YAML execution block through (fixes #6206)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 08:58:57 +00:00
MervinPraisonandClaude Opus 5.5 41acf94def fix: atomic_write writes through symlinks instead of replacing the link
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 08:53:47 +00:00
MervinPraisonandClaude Opus 5.5 40e04fdf44 fix: create atomic_write temp file with 0o666 instead of reading process-wide umask
os.umask() is process-wide, so the read-and-restore window could make files
created by other threads world-writable. The temp file is now created via
os.open(O_CREAT|O_EXCL, 0o666) so the kernel applies the current umask.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 08:51:47 +00:00
MervinPraisonandClaude Opus 5.5 d21d60600f fix: keep umask-based mode for new files and locale encoding in multiedit atomic writes
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 08:17:07 +00:00
MervinPraisonandClaude Opus 5.5 0b04d2f017 fix: write praisonai.code edit tools atomically so interrupts keep the original file (fixes #6204)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-03 08:11:56 +00:00
111 changed files with 5712 additions and 899 deletions

No files matched your search

+1 -1
View File
@@ -41,4 +41,4 @@ jobs:
- name: Run tests
run: |
cd src/praisonai && python -m pytest tests/unit/ -v --tb=short --disable-warnings --timeout=60 || echo "Some tests failed"
cd src/praisonai && python -m pytest tests/unit/ -v --tb=short --disable-warnings --timeout=60
+5 -1
View File
@@ -155,7 +155,11 @@ jobs:
tests/unit/plugins/ tests/unit/runtime/ tests/unit/tools/
tests/unit/bus/ tests/unit/skills/ tests/unit/agents/
tests/unit/rag/ tests/unit/streaming/
tests/unit/test_adapter_registry.py
tests/unit/test_adapter_registry.py tests/unit/test_token_budget.py
tests/unit/test_streaming_events.py tests/unit/test_tool_decorator_approval.py
tests/unit/test_task_execution_config.py
tests/unit/test_agent_unknown_kwarg_guidance.py
tests/unit/test_agent_instantiation_lazy.py tests/unit/test_agent_os_exports.py
# Both tests previously deselected here now pass in combination.
# test_lazy_litellm_import was looking the loader module up by
# name and could get a different object than the one get_litellm()
+340 -177
View File
@@ -14,27 +14,145 @@
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
</p>
<div align="center">
# PraisonAI
# PraisonAI 🦞
**Private AI that earns trust one task at a time.**
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
PraisonAI runs AI agents on your own computers, even fully offline, so private data never has to leave the building.
</div>
- **For highly regulated teams (early access):** each task moves to your own AI only after it passes a test. If quality slips, it moves back automatically, and every step is recorded for regulators. [How it works](#how-the-handover-works-early-access)
- **For everyone else:** a free, open-source Python agent framework. With a local model you need no API key and pay no cloud bill, and your files stay yours.
PraisonAI 🦞 — **Hire a 24/7 AI Workforce.** Stop writing boilerplate and start shipping autonomous, self-improving agents that research, plan, and execute tasks across your apps. From one agent to an entire organization, deployed in 5 lines of code.
[2M+ downloads](https://pepy.tech/projects/praisonai) · [9.3K stars](https://github.com/MervinPraison/PraisonAI) · [600+ releases in the last year](https://github.com/MervinPraison/PraisonAI/releases) · [80+ contributors](https://github.com/MervinPraison/PraisonAI/graphs/contributors) · Python
```bash
curl -fsSL https://praison.ai/install.sh | bash
pip install praisonai
```
Only need the Python library? `pip install praisonaiagents` installs just the lightweight core.
## Using PraisonAI for something else?
PraisonAI is still a general-purpose, open-source agent framework, and everything works as before: the same packages, imports, commands and examples. Go straight to what you came for:
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs)
- **Workflows:** [AgentFlow: route, parallel, loop, repeat](https://praison.ai/docs/concepts/agentflow)
- **Memory and knowledge:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge](https://praison.ai/docs/concepts/knowledge) · [RAG](https://praison.ai/docs/concepts/rag)
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
- **Bots (Telegram, Discord, Slack, WhatsApp):** [Bot CLI](https://praison.ai/docs/cli/bot) · [Bot gateway](https://praison.ai/docs/features/bot-gateway)
- **Terminal CLI:** [`praisonai code`](https://praison.ai/docs/cli/code) · [`praisonai run`](https://praison.ai/docs/cli/run) · [`praisonai chat`](https://praison.ai/docs/cli/chat) · [CLI reference](https://praison.ai/docs/cli/cli-reference)
- **Training and fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
- **Browser automation:** [Browser agent](https://praison.ai/docs/features/browser-agent) · [Browser CLI](https://praison.ai/docs/cli/browser)
- **Sandboxes:** [Sandboxed code execution](https://praison.ai/docs/features/sandbox)
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
- **Dashboard and UI:** [Claw dashboard](https://praison.ai/docs/concepts/claw) · [Chat UI](https://praison.ai/docs/ui)
- **YAML, no code:** [YAML agents](#-using-yaml-no-code) · [YAML reference](https://praison.ai/docs/features/yaml-configuration-reference)
- **JavaScript and TypeScript:** [JavaScript SDK](https://praison.ai/docs/js)
- **Desktop app:** [PraisonAI Desktop](https://praison.ai/docs/features/desktop/index) · [Downloads](#desktop-app)
- **Everything else:** the [full feature reference](#full-feature-reference) further down this page
## Quick start
Set a key for any supported provider (OpenAI shown here), or skip the key and use a local model as in the next section.
```bash
export OPENAI_API_KEY="your-api-key"
```
```python
from praisonaiagents import Agent
agent = Agent(instructions="You are a helpful assistant.")
print(agent.start("In two sentences, why do some companies keep their data on their own computers?"))
```
## Run fully offline
Run the same agent on a model on your own computer with [Ollama](https://ollama.com). You need no API key, and your prompts and answers stay on the machine.
Install the package and pull the model while you still have internet; after that, the agent runs with no network at all. If you installed only the core library, use `pip install "praisonaiagents[llm]"`, which adds the local-model support.
```bash
# Start Ollama first (the app, or "ollama serve" in another terminal)
ollama pull llama3.2
export DO_NOT_TRACK=true # keeps telemetry off (PraisonAI sends none unless you turn it on)
```
```python
from praisonaiagents import Agent
agent = Agent(instructions="You are a helpful assistant.", model="ollama/llama3.2")
print(agent.start("Why is the sky blue?"))
```
If Ollama runs on another computer on your network, pass `base_url` (or set `OLLAMA_API_BASE`):
```python
agent = Agent(instructions="...", model="ollama/llama3.2", base_url="http://192.168.1.10:11434")
```
vLLM and LM Studio work the same way: use `hosted_vllm/` (vLLM) or `lm_studio/` with your server's `base_url`. More in the [Ollama guide](https://praison.ai/docs/models/ollama) and [Models docs](https://praison.ai/docs/models).
## How the handover works (early access)
1. **Watch:** records how each task is done today.
2. **Test:** your own AI takes an exam made from real past work it never studied.
3. **Hand over:** a task moves to your own AI only when it passes.
4. **Roll back:** if quality drops, the task moves back automatically.
5. **Record:** every handover leaves a record your regulator can check.
**What is "your own AI"?** An open model, such as Llama, Qwen or Gemma, running on your own computers. Use it as it is, or train it on your own past work with [PraisonAI Train](https://praison.ai/docs/train) on your own GPU servers. Your team decides when to train; nothing trains automatically. Either way, a task moves to it only after it passes that exam.
Highly regulated teams, such as banks, insurers and hospitals, want AI but can't send customer data to outside models. From 6 January 2027, OpenAI also takes no new self-serve fine-tuning jobs ([notice](https://developers.openai.com/api/docs/deprecations)), so teams that customise models need models they own. The handover lets these teams move work to AI they own, one task at a time, with a record that shows why each step was trusted.
It is built on parts of this framework that are live today: tracing, evaluation, fine-tuning, model routing and local models. The handover itself is in early access, and we are taking on our first design partners. To take part, see [praison.ai/enterprise](https://praison.ai/enterprise/).
## Features
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs) · [Workflows](https://praison.ai/docs/concepts/agentflow)
- **Memory:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge and RAG](https://praison.ai/docs/concepts/rag) · [Sessions](https://praison.ai/docs/concepts/session-management)
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
- **Guardrails and approval:** [Guardrails](https://praison.ai/docs/concepts/guardrails) · [Human approval](https://praison.ai/docs/concepts/approval) · [Policy engine](https://praison.ai/docs/features/policy-engine)
- **Tracing and evaluation:** [Tracing](https://praison.ai/docs/observability/overview) · [Telemetry](https://praison.ai/docs/features/telemetry) · [Langfuse](https://praison.ai/docs/observability/langfuse) · [Evaluation](https://praison.ai/docs/concepts/evaluation)
- **Your choice of model:** [Models](https://praison.ai/docs/models) · [Model router](https://praison.ai/docs/features/model-router)
- **Fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
## JavaScript and TypeScript
```bash
npm install praisonai
```
The TypeScript and JavaScript SDK has its own guide at [praison.ai/docs/js](https://praison.ai/docs/js). A short example is in [Using JavaScript Code](#-using-javascript-code) below.
## The company
- **Who:** Praison AI Limited in Cambridge, UK, building PraisonAI since September 2023. Co-founders Mervin Praison (CEO), Leebana Smith (COO) and Karthick Subramanian (CBO). [About us](https://praison.ai/about/)
- **What we're building:** the task handover above, as a self-hosted product on top of this framework. It is in early access, and we are taking on our first design partners.
- **Why now:** highly regulated teams want AI but can't send customer data to outside models, and from 6 January 2027 OpenAI takes no new self-serve fine-tuning jobs.
- **How it fits together:** the framework stays free and MIT-licensed, and brings developers in. Regulated teams license the handover to run inside their own walls.
- **Traction:** every number at the top of this page is public and linked to its source.
- **Get in touch:** design partners, investors and press: enterprise@praison.ai or [praison.ai/enterprise](https://praison.ai/enterprise/)
## Docs, community and licence
- **Documentation:** [praison.ai/docs](https://praison.ai/docs)
- **Questions and bugs:** [Discussions](https://github.com/MervinPraison/PraisonAI/discussions) · [Issues](https://github.com/MervinPraison/PraisonAI/issues)
- **Contributing:** [Contributing guide](https://github.com/MervinPraison/PraisonAI/blob/main/CONTRIBUTING.md)
- **Videos and news:** [YouTube](https://youtube.com/@MervinPraison) · [X](https://x.com/MervinPraison) · [LinkedIn](https://linkedin.com/in/mervinpraison)
- **Desktop app:** macOS, Windows and Linux builds are on the [latest release](https://github.com/MervinPraison/PraisonAI/releases/latest)
- **Licence:** [MIT](https://github.com/MervinPraison/PraisonAI/blob/main/LICENSE)
---
## Full feature reference
Everything below is the full reference that was on this page before. Nothing has been removed; the longest parts are folded, so click a heading's summary to open it.
### Desktop app
<div align="center">
<br>
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
</a>
<br>
<br>
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
<img src="https://img.shields.io/badge/Download_for_macOS-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Download for macOS" />
</a>
@@ -46,12 +164,17 @@ curl -fsSL https://praison.ai/install.sh | bash
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
<img src="https://img.shields.io/badge/Download_for_Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download for Linux" />
</a>
<br>
</div>
<p align="center">
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
</p>
### Other ways to install
One-line installer:
```bash
curl -fsSL https://praison.ai/install.sh | bash
```
The full `praisonai` package, with the CLI, bots, dashboard and integrations:
```
██████╗ ██████╗ █████╗ ██╗███████╗ ██████╗ ███╗ ██╗ █████╗ ██╗
@@ -64,10 +187,20 @@ curl -fsSL https://praison.ai/install.sh | bash
pip install praisonai
```
<p align="center">
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
</p>
<div align="center">
<a href="https://docs.praison.ai">
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<div align="center">
<a href="https://praison.ai/docs">
<p align="center">
<img src="https://img.shields.io/badge/📚_Documentation-Visit_docs.praison.ai-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
<img src="https://img.shields.io/badge/📚_Documentation-Visit_praison.ai%2Fdocs-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
</p>
</a>
</div>
@@ -114,6 +247,9 @@ Most frameworks hand you one or two layers and leave the rest as homework. Prais
Each layer wraps the one inside it. When an agent misbehaves, the layer tells you where to look.
<details>
<summary><strong>Show the five layers, examples and Managed Agents</strong></summary>
```
┌─────────────────────────────────────────────────────────────────┐
│ ⬡ MANAGED AGENTS — Where does it actually run? │
@@ -174,7 +310,7 @@ agent = Agent(
)
```
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://docs.praison.ai/docs/concepts/handoffs)
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://praison.ai/docs/concepts/handoffs)
### Layer 3 · Harness — *Can it act, and be checked?*
@@ -217,7 +353,7 @@ print(result.completion_reason)
# surfaced here as completion_reason="error")
```
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://docs.praison.ai/docs/features/doom-loop-detection)
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://praison.ai/docs/features/doom-loop-detection)
### Layer 5 · Graph — *Who runs when, and who checks whom?*
@@ -237,7 +373,7 @@ flow = AgentFlow(steps=[
flow.run("Ticket #123: login fails on Safari")
```
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://docs.praison.ai/docs/concepts/agentflow)
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://praison.ai/docs/concepts/agentflow)
### ⬡ Outside the stack: Managed Agents — *Where does it actually run?*
@@ -332,6 +468,8 @@ Sandboxes shut themselves down when idle (`auto_shutdown`, `idle_timeout_s`), an
<sub>Stack framing adapted from [The Five-Layer Agent Stack](https://mer.vin/2026/07/five-layer-agent-stack-match-bug-to-right-layer/) and [Agent Harnesses vs Orbs](https://mer.vin/2026/08/agent-harnesses-vs-orbs-why-remote-sandboxes-beat-local-agent-loops/).</sub>
</details>
---
## 🌌 The PraisonAI Ecosystem
@@ -411,6 +549,11 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
</details>
<div align="center">
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
</a>
<br>
<br>
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
<img src=".github/images/elon_musk_praisonai.png" alt="Highlighted by Elon Musk" width="600" />
</a>
@@ -426,13 +569,13 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|--|---------|-----|
| 🔌 | **MCP Protocol** — stdio, HTTP, WebSocket, SSE | `tools=MCP("npx ...")` |
| 🧠 | **Planning Mode** — plan → execute → reason | `planning=True` |
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://docs.praison.ai/docs/agents/deep-research) |
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://docs.praison.ai/docs/code/external-agents) |
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://praison.ai/docs/agents/deep-research) |
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://praison.ai/docs/code/external-agents) |
| 🔄 | **Agent Handoffs** — seamless conversation passing | `handoffs=[other_agent]` |
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://docs.praison.ai/docs/concepts/guardrails) |
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://praison.ai/docs/concepts/guardrails) |
| | **Web Search + Fetch** — native browsing | `web=True` |
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://docs.praison.ai/docs/concepts/reflection) |
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://docs.praison.ai/docs/concepts/agentflow) |
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://praison.ai/docs/concepts/reflection) |
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://praison.ai/docs/concepts/agentflow) |
| 🧠 | **Memory (zero deps)** — works out of the box | `memory=True` |
<details>
@@ -443,18 +586,18 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
| 💡 | **Prompt Caching** — reduce latency + cost | `caching=True` |
| 💾 | **Sessions + Auto-Save** — persistent state across restarts | `auto_save="my-project"` |
| 💭 | **Thinking Budgets** — control reasoning depth | `agent.thinking_budget = 1024` |
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://docs.praison.ai/docs/concepts/rag) |
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://docs.praison.ai/docs/features/model-router) |
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://docs.praison.ai/docs/features/checkpoints) |
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://docs.praison.ai/docs/features/a2a) |
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://docs.praison.ai/docs/features/context-compaction) |
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://docs.praison.ai/docs/features/telemetry) |
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://docs.praison.ai/docs/features/policy-engine) |
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://docs.praison.ai/docs/features/background-tasks) |
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://docs.praison.ai/docs/features/doom-loop-detection) |
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://docs.praison.ai/docs/features/graph-memory) |
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://docs.praison.ai/docs/features/sandbox) |
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://docs.praison.ai/docs/features/bot-gateway) |
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://praison.ai/docs/concepts/rag) |
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://praison.ai/docs/features/model-router) |
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://praison.ai/docs/features/checkpoints) |
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://praison.ai/docs/features/a2a) |
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://praison.ai/docs/features/context-compaction) |
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://praison.ai/docs/features/telemetry) |
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://praison.ai/docs/features/policy-engine) |
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://praison.ai/docs/features/background-tasks) |
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://praison.ai/docs/features/doom-loop-detection) |
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://praison.ai/docs/features/graph-memory) |
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://praison.ai/docs/features/sandbox) |
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://praison.ai/docs/features/bot-gateway) |
</details>
@@ -465,6 +608,9 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
## 📘 Using Python Code
<details>
<summary><strong>Show Python examples: single agent, multi-agent, MCP, tools, databases, Claw, Langflow, UI</strong></summary>
> Unsure whether a result is a bug or model randomness? Check the [SDK Behavior Matrix](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/docs/sdk-behavior-matrix.md) for `stream`, memory prefetch, structured output, teams and AG-UI.
### 1. Single Agent
@@ -510,7 +656,7 @@ agent = Agent(
)
```
> 📖 [Full MCP docs](https://docs.praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
> 📖 [Full MCP docs](https://praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
### 4. Custom Tools
@@ -562,7 +708,7 @@ agent.start("Search for AI news and calculate 15*4")
```
> ⚠️ **Security Note:** Never use `eval()`, `exec()`, or `subprocess` in tool functions that process LLM-generated or user-supplied input. Always validate and sanitize inputs to prevent code injection attacks.
> 📖 [Full tools docs](https://docs.praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
> 📖 [Full tools docs](https://praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
### 5. Persistence (Databases)
@@ -579,7 +725,7 @@ agent = Agent(
agent.chat("Hello!") # Auto-persists messages, runs, traces
```
> 📖 [Full persistence docs](https://docs.praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
> 📖 [Full persistence docs](https://praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
### 6. PraisonAI Claw 🦞 (Dashboard UI)
@@ -601,7 +747,7 @@ Copy `.env.example` to `.env` and configure the following variables:
Open **http://localhost:8082** — the dashboard comes with 13 built-in pages: Chat, Agents, Memory, Knowledge, Channels, Guardrails, Cron, and more. Add messaging channels directly from the UI.
> 📖 [Full Claw docs](https://docs.praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
> 📖 [Full Claw docs](https://praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
### 7. Langflow Integration 🔗 (Visual Flow Builder)
@@ -614,7 +760,7 @@ praisonai flow
Open **http://localhost:7861** — use the **Agent** and **Agent Team** components to create sequential or parallel workflows. Connect Chat Input → Agent Team → Chat Output for instant multi-agent pipelines.
> 📖 [Full Flow docs](https://docs.praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
> 📖 [Full Flow docs](https://praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
### 8. PraisonAI UI 🤖 (Clean Chat)
@@ -625,10 +771,15 @@ pip install "praisonai[ui]"
praisonai ui
```
</details>
---
## 📄 Using YAML (No Code)
<details>
<summary><strong>Show YAML examples</strong></summary>
### Example 1: Two Agents Working Together
Create `agents.yaml`:
@@ -700,6 +851,8 @@ praisonai agents.yaml
> - Tools in `tools.py` are automatically discovered
> - The function's docstring helps the AI understand how to use it
</details>
---
## 🎯 CLI Quick Reference
@@ -718,23 +871,26 @@ praisonai agents.yaml
| **Development** | `commit`, `docs`, `checkpoint`, `hooks` |
| **Scheduling** | `schedule start`, `schedule list`, `schedule stop` |
> 📖 [Full CLI reference](https://docs.praison.ai/docs/cli/cli-reference)
> 📖 [Full CLI reference](https://praison.ai/docs/cli/cli-reference)
---
## ✨ Key Features
<details>
<summary><strong>Show all features by category</strong></summary>
<details open>
<summary><strong>🤖 Core Agents</strong></summary>
| Feature | Code | Docs |
|---------|:----:|:----:|
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://docs.praison.ai/docs/agents/single) |
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://docs.praison.ai/docs/concepts/agents) |
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://docs.praison.ai/docs/features/autoagents) |
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://docs.praison.ai/docs/concepts/reflection) |
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://docs.praison.ai/docs/features/reasoning) |
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://docs.praison.ai/docs/features/multimodal) |
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://praison.ai/docs/agents/single) |
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://praison.ai/docs/concepts/agents) |
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://praison.ai/docs/features/autoagents) |
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://praison.ai/docs/concepts/reflection) |
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://praison.ai/docs/features/reasoning) |
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://praison.ai/docs/features/multimodal) |
</details>
@@ -743,16 +899,16 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://docs.praison.ai/docs/features/routing) |
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://praison.ai/docs/features/routing) |
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://praison.ai/docs/features/parallelisation) |
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://praison.ai/docs/features/repetitive) |
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
</details>
@@ -761,13 +917,13 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://docs.praison.ai/docs/features/codeagent) |
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://docs.praison.ai/docs/code/editing) |
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://docs.praison.ai/docs/code/external-agents) |
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://docs.praison.ai/docs/code/claude-code) |
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://docs.praison.ai/docs/code/gemini-cli) |
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://docs.praison.ai/docs/code/codex-cli) |
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://docs.praison.ai/docs/code/cursor-cli) |
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://praison.ai/docs/features/codeagent) |
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://praison.ai/docs/code/editing) |
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://praison.ai/docs/code/external-agents) |
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://praison.ai/docs/code/claude-code) |
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://praison.ai/docs/code/gemini-cli) |
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://praison.ai/docs/code/codex-cli) |
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://praison.ai/docs/code/cursor-cli) |
</details>
@@ -776,18 +932,18 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://docs.praison.ai/docs/features/claude-memory-tool) |
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://docs.praison.ai/docs/concepts/knowledge) |
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://docs.praison.ai/docs/concepts/rag) |
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://docs.praison.ai/docs/features/chat-with-pdf) |
| Data Readers (PDF, DOCX, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/chunking-strategies) |
| Vector Store Selection | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/knowledge-backends) |
| Retrieval Strategies | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval-strategies) |
| Rerankers | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/smart-retrieval) |
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/incremental-indexing) |
| Query Engines (Sub-Question, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval) |
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://praison.ai/docs/features/claude-memory-tool) |
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://praison.ai/docs/concepts/knowledge) |
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://praison.ai/docs/concepts/rag) |
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://praison.ai/docs/features/chat-with-pdf) |
| Data Readers (PDF, DOCX, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/chunking-strategies) |
| Vector Store Selection | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/knowledge-backends) |
| Retrieval Strategies | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval-strategies) |
| Rerankers | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/smart-retrieval) |
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/incremental-indexing) |
| Query Engines (Sub-Question, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval) |
</details>
@@ -796,12 +952,12 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://docs.praison.ai/docs/agents/deep-research) |
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://docs.praison.ai/docs/agents/query-rewriter) |
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/agents/websearch) |
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/tools/tavily) |
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://docs.praison.ai/docs/tools/web-search) |
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://praison.ai/docs/agents/deep-research) |
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://praison.ai/docs/agents/query-rewriter) |
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/agents/websearch) |
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/tools/tavily) |
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://praison.ai/docs/tools/web-search) |
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
</details>
@@ -810,12 +966,12 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://docs.praison.ai/docs/features/promptchaining) |
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://docs.praison.ai/docs/concepts/orchestration) |
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://praison.ai/docs/features/promptchaining) |
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://praison.ai/docs/concepts/orchestration) |
</details>
@@ -824,15 +980,15 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://docs.praison.ai/docs/agents/data-analyst) |
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://docs.praison.ai/docs/agents/finance) |
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://docs.praison.ai/docs/agents/shopping) |
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://docs.praison.ai/docs/agents/recommendation) |
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://docs.praison.ai/docs/agents/wikipedia) |
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://docs.praison.ai/docs/agents/programming) |
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://docs.praison.ai/docs/features/mathagent) |
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://docs.praison.ai/docs/agents/markdown) |
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://docs.praison.ai/docs/agents/prompt-expander) |
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://praison.ai/docs/agents/data-analyst) |
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://praison.ai/docs/agents/finance) |
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://praison.ai/docs/agents/shopping) |
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://praison.ai/docs/agents/recommendation) |
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://praison.ai/docs/agents/wikipedia) |
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://praison.ai/docs/agents/programming) |
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://praison.ai/docs/features/mathagent) |
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://praison.ai/docs/agents/markdown) |
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://praison.ai/docs/agents/prompt-expander) |
</details>
@@ -841,10 +997,10 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://docs.praison.ai/docs/features/image-generation) |
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://docs.praison.ai/docs/agents/image-to-text) |
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://docs.praison.ai/docs/agents/video) |
| Camera Integration | [Example](examples/python/camera/) | [📖](https://docs.praison.ai/docs/features/camera-integration) |
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://praison.ai/docs/features/image-generation) |
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://praison.ai/docs/agents/image-to-text) |
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://praison.ai/docs/agents/video) |
| Camera Integration | [Example](examples/python/camera/) | [📖](https://praison.ai/docs/features/camera-integration) |
</details>
@@ -853,12 +1009,12 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
| MCP Config Management | [Docs](https://docs.praison.ai/docs/cli/mcp) | [📖](https://docs.praison.ai/docs/cli/mcp) |
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://docs.praison.ai/docs/features/langchain) |
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://praison.ai/docs/mcp/transports) |
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://praison.ai/docs/mcp/transports) |
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
| MCP Config Management | [Docs](https://praison.ai/docs/cli/mcp) | [📖](https://praison.ai/docs/cli/mcp) |
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://praison.ai/docs/features/langchain) |
</details>
@@ -867,9 +1023,9 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://docs.praison.ai/docs/concepts/guardrails) |
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://docs.praison.ai/docs/concepts/approval) |
| Rules & Instructions | [Docs](https://docs.praison.ai/docs/features/rules) | [📖](https://docs.praison.ai/docs/features/rules) |
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://praison.ai/docs/concepts/guardrails) |
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://praison.ai/docs/concepts/approval) |
| Rules & Instructions | [Docs](https://praison.ai/docs/features/rules) | [📖](https://praison.ai/docs/features/rules) |
</details>
@@ -878,16 +1034,16 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://docs.praison.ai/docs/features/async) |
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://docs.praison.ai/docs/concepts/handoffs) |
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://docs.praison.ai/docs/features/stateful-agents) |
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://docs.praison.ai/docs/concepts/autonomy) |
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://docs.praison.ai/docs/features/structured) |
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://docs.praison.ai/docs/features/model-router) |
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://docs.praison.ai/docs/features/fast-context) |
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://praison.ai/docs/features/async) |
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://praison.ai/docs/features/parallelisation) |
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://praison.ai/docs/features/repetitive) |
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://praison.ai/docs/concepts/handoffs) |
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://praison.ai/docs/features/stateful-agents) |
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://praison.ai/docs/concepts/autonomy) |
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://praison.ai/docs/features/structured) |
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://praison.ai/docs/features/model-router) |
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://praison.ai/docs/features/fast-context) |
</details>
@@ -896,21 +1052,21 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://docs.praison.ai/docs/tools/tools) |
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://docs.praison.ai/docs/developers/agents-playbook) |
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://docs.praison.ai/docs/models) |
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://docs.praison.ai/docs/features/middleware) |
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://docs.praison.ai/docs/features/configurable-model) |
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://docs.praison.ai/docs/features/rate-limiter) |
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://docs.praison.ai/docs/features/injected-state) |
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://docs.praison.ai/docs/features/checkpoints) |
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://docs.praison.ai/docs/features/background-tasks) |
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://docs.praison.ai/docs/features/policy-engine) |
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://docs.praison.ai/docs/features/thinking-budgets) |
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://docs.praison.ai/docs/features/output-styles) |
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://docs.praison.ai/docs/features/context-compaction) |
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://praison.ai/docs/tools/tools) |
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://praison.ai/docs/developers/agents-playbook) |
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://praison.ai/docs/models) |
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://praison.ai/docs/concepts/hooks) |
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://praison.ai/docs/concepts/hooks) |
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://praison.ai/docs/features/middleware) |
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://praison.ai/docs/features/configurable-model) |
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://praison.ai/docs/features/rate-limiter) |
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://praison.ai/docs/features/injected-state) |
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://praison.ai/docs/features/checkpoints) |
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://praison.ai/docs/features/background-tasks) |
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://praison.ai/docs/features/policy-engine) |
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://praison.ai/docs/features/thinking-budgets) |
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://praison.ai/docs/features/output-styles) |
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://praison.ai/docs/features/context-compaction) |
</details>
@@ -919,14 +1075,14 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://docs.praison.ai/docs/concepts/session-management) |
| Auto-Save Sessions | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
| History in Context | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://docs.praison.ai/docs/features/telemetry) |
| Langfuse Tracing | [Docs](https://docs.praison.ai/docs/observability/langfuse) | [📖](https://docs.praison.ai/docs/observability/langfuse) |
| Project Docs (.praison/docs/) | [Docs](https://docs.praison.ai/docs/cli/docs) | [📖](https://docs.praison.ai/docs/cli/docs) |
| AI Commit Messages | [Docs](https://docs.praison.ai/docs/cli/commit) | [📖](https://docs.praison.ai/docs/cli/commit) |
| @Mentions in Prompts | [Docs](https://docs.praison.ai/docs/cli/mentions) | [📖](https://docs.praison.ai/docs/cli/mentions) |
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://praison.ai/docs/concepts/session-management) |
| Auto-Save Sessions | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
| History in Context | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://praison.ai/docs/features/telemetry) |
| Langfuse Tracing | [Docs](https://praison.ai/docs/observability/langfuse) | [📖](https://praison.ai/docs/observability/langfuse) |
| Project Docs (.praison/docs/) | [Docs](https://praison.ai/docs/cli/docs) | [📖](https://praison.ai/docs/cli/docs) |
| AI Commit Messages | [Docs](https://praison.ai/docs/cli/commit) | [📖](https://praison.ai/docs/cli/commit) |
| @Mentions in Prompts | [Docs](https://praison.ai/docs/cli/mentions) | [📖](https://praison.ai/docs/cli/mentions) |
</details>
@@ -935,20 +1091,20 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://docs.praison.ai/docs/cli/slash-commands) |
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://docs.praison.ai/docs/cli/autonomy-modes) |
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://docs.praison.ai/docs/cli/cost-tracking) |
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://docs.praison.ai/docs/cli/repo-map) |
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://docs.praison.ai/docs/cli/interactive-tui) |
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://docs.praison.ai/docs/cli/git-integration) |
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://docs.praison.ai/docs/cli/sandbox-execution) |
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://docs.praison.ai/docs/cli/compare) |
| Profile/Benchmark | [Docs](https://docs.praison.ai/docs/cli/profile) | [📖](https://docs.praison.ai/docs/cli/profile) |
| Auto Mode | [Docs](https://docs.praison.ai/docs/cli/auto) | [📖](https://docs.praison.ai/docs/cli/auto) |
| Init | [Docs](https://docs.praison.ai/docs/cli/init) | [📖](https://docs.praison.ai/docs/cli/init) |
| File Input | [Docs](https://docs.praison.ai/docs/cli/file-input) | [📖](https://docs.praison.ai/docs/cli/file-input) |
| Final Agent | [Docs](https://docs.praison.ai/docs/cli/final-agent) | [📖](https://docs.praison.ai/docs/cli/final-agent) |
| Max Tokens | [Docs](https://docs.praison.ai/docs/cli/max-tokens) | [📖](https://docs.praison.ai/docs/cli/max-tokens) |
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://praison.ai/docs/cli/slash-commands) |
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://praison.ai/docs/cli/autonomy-modes) |
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://praison.ai/docs/cli/cost-tracking) |
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://praison.ai/docs/cli/repo-map) |
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://praison.ai/docs/cli/interactive-tui) |
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://praison.ai/docs/cli/git-integration) |
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://praison.ai/docs/cli/sandbox-execution) |
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://praison.ai/docs/cli/compare) |
| Profile/Benchmark | [Docs](https://praison.ai/docs/cli/profile) | [📖](https://praison.ai/docs/cli/profile) |
| Auto Mode | [Docs](https://praison.ai/docs/cli/auto) | [📖](https://praison.ai/docs/cli/auto) |
| Init | [Docs](https://praison.ai/docs/cli/init) | [📖](https://praison.ai/docs/cli/init) |
| File Input | [Docs](https://praison.ai/docs/cli/file-input) | [📖](https://praison.ai/docs/cli/file-input) |
| Final Agent | [Docs](https://praison.ai/docs/cli/final-agent) | [📖](https://praison.ai/docs/cli/final-agent) |
| Max Tokens | [Docs](https://praison.ai/docs/cli/max-tokens) | [📖](https://praison.ai/docs/cli/max-tokens) |
</details>
@@ -957,10 +1113,10 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://praison.ai/docs/cli/eval) |
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://praison.ai/docs/cli/eval) |
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://praison.ai/docs/cli/eval) |
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://praison.ai/docs/cli/eval) |
</details>
@@ -969,8 +1125,8 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://praison.ai/docs/concepts/skills) |
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://praison.ai/docs/concepts/skills) |
</details>
@@ -979,7 +1135,9 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://docs.praison.ai/docs/cli/scheduler) |
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://praison.ai/docs/cli/scheduler) |
</details>
</details>
@@ -1008,7 +1166,7 @@ PraisonAI keeps framework overhead microsecond-class: creating an `Agent` (one t
|--------------------|-----------|----------|
| Avg Agent instantiation (1 tool, 1000 iterations) | **46 μs** | 2026-10-15 · praisonaiagents 1.7.12 |
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://docs.praison.ai/docs/developers/benchmarks)
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://praison.ai/docs/developers/benchmarks)
---
@@ -1018,7 +1176,7 @@ Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md]
## ⭐ Star History
[![Star History Chart](https://api.star-history.com/svg?repos=MervinPraison/PraisonAI&type=Date)](https://docs.praison.ai)
[![Star History Chart](https://api.star-history.com/svg?repos=MervinPraison/PraisonAI&type=Date)](https://praison.ai/docs)
---
@@ -1089,6 +1247,9 @@ We welcome contributions! Fork the repo, create a branch, and submit a PR → [C
## ❓ FAQ & Troubleshooting
<details>
<summary><strong>Show FAQ and troubleshooting</strong></summary>
<details>
<summary><strong>ModuleNotFoundError: No module named 'praisonaiagents'</strong></summary>
@@ -1107,7 +1268,7 @@ Ensure your API key is set:
export OPENAI_API_KEY=your_key_here
```
For other providers, see [Models docs](https://docs.praison.ai/docs/models).
For other providers, see [Models docs](https://praison.ai/docs/models).
</details>
@@ -1148,7 +1309,7 @@ agent = Agent(instructions="...", llm="ollama/llama3.2", base_url="http://192.16
The same shape works for other local runtimes — `lm_studio/`, `vllm/` and
`hosted_vllm/` with their server's `base_url`.
See [Models docs](https://docs.praison.ai/docs/models) for more details.
See [Models docs](https://praison.ai/docs/models) for more details.
</details>
@@ -1168,7 +1329,7 @@ agent = Agent(
)
```
See [Persistence docs](https://docs.praison.ai/docs/databases/overview) for supported databases.
See [Persistence docs](https://praison.ai/docs/databases/overview) for supported databases.
</details>
@@ -1185,7 +1346,7 @@ agent = Agent(
)
```
See [Memory docs](https://docs.praison.ai/docs/concepts/memory) for more options.
See [Memory docs](https://praison.ai/docs/concepts/memory) for more options.
</details>
@@ -1201,7 +1362,7 @@ agents = Agents(agents=[agent1, agent2])
agents.start()
```
See [Agents docs](https://docs.praison.ai/docs/concepts/agents) for more examples.
See [Agents docs](https://praison.ai/docs/concepts/agents) for more examples.
</details>
@@ -1216,13 +1377,15 @@ agent = Agent(
)
```
See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport options.
See [MCP docs](https://praison.ai/docs/mcp/transports) for all transport options.
</details>
</details>
### Getting Help
- 📚 [Full Documentation](https://docs.praison.ai)
- 📚 [Full Documentation](https://praison.ai/docs)
- 🐛 [Report Issues](https://github.com/MervinPraison/PraisonAI/issues)
- 💬 [Discussions](https://github.com/MervinPraison/PraisonAI/discussions)
@@ -1231,7 +1394,7 @@ See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport op
<div align="center">
<p><strong>Made with ❤️ by the PraisonAI Team</strong></p>
<p>
<a href="https://docs.praison.ai">📚 Documentation</a> •
<a href="https://praison.ai/docs">📚 Documentation</a> •
<a href="https://github.com/MervinPraison/PraisonAI">GitHub</a> •
<a href="https://youtube.com/@MervinPraison">▶️ YouTube</a> •
<a href="https://x.com/MervinPraison">𝕏 X</a> •
@@ -5,23 +5,39 @@ PraisonAI Agents - Real Execution Benchmark
Benchmarks ACTUAL agent execution with LLM API calls.
Tests single agent running a simple task across frameworks.
Each framework gets untimed warm-up calls first (imports, client set-up and
the first connection are not what this measures), then the timed runs are
interleaved round by round so network drift hits every framework alike.
A run that raises or returns an empty answer is counted as a failure, not
timed. The summary reports the median.
Usage:
export OPENAI_API_KEY=your_key
python benchmarks/execution_benchmark.py
python benchmarks/execution_benchmark.py --no-save
python benchmarks/execution_benchmark.py --iterations 5
python benchmarks/execution_benchmark.py --save
python benchmarks/execution_benchmark.py --iterations 10
python benchmarks/execution_benchmark.py --model gpt-4o --iterations 10
"""
import time
import os
import argparse
import os
import statistics
import time
from importlib.metadata import version as get_version
# Defaults (can be overridden via CLI)
DEFAULT_MODEL = "gpt-4o-mini"
DEFAULT_ITERATIONS = 3
DEFAULT_ITERATIONS = 10
DEFAULT_WARMUP = 1
DEFAULT_PROMPT = "What is 2+2? Reply with just the number."
INSTRUCTIONS = "You are a helpful assistant. Be very brief."
METHODS = {
'PraisonAI': 'agent.start()',
'PraisonAI (LiteLLM)': 'agent.start()',
'Agno': 'agent.run()',
'CrewAI': 'crew.kickoff()',
}
def get_package_versions():
@@ -35,260 +51,235 @@ def get_package_versions():
for name, pkg in packages.items():
try:
versions[name] = get_version(pkg)
except:
except Exception:
versions[name] = 'not installed'
return versions
def benchmark_praisonai(model, iterations, prompt):
"""Benchmark PraisonAI agent execution using agent.start()"""
def praisonai_runner(model, prompt):
"""PraisonAI: a new agent per run; only agent.start() is timed."""
from praisonaiagents import Agent
print("\n--- PraisonAI ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=model
)
start = time.perf_counter()
response = agent.start(prompt, output="silent")
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def build():
agent = Agent(name="Calculator", instructions=INSTRUCTIONS, llm=model)
return lambda: agent.start(prompt, output="silent")
return build
def benchmark_praisonai_litellm(model, iterations, prompt):
"""Benchmark PraisonAI with LiteLLM backend using agent.start()"""
from praisonaiagents import Agent
print("\n--- PraisonAI (LiteLLM) ---")
print("Method: agent.start()")
times = []
for i in range(iterations):
agent = Agent(
name="Calculator",
instructions="You are a helpful assistant. Be very brief.",
llm=f"openai/{model}"
)
start = time.perf_counter()
response = agent.start(prompt, output="silent")
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def benchmark_agno(model, iterations, prompt):
"""Benchmark Agno agent execution using agent.run()"""
def agno_runner(model, prompt):
"""Agno: a new agent per run; only agent.run() is timed."""
from agno.agent import Agent
from agno.models.openai import OpenAIChat
print("\n--- Agno ---")
print("Method: agent.run()")
times = []
# One model object per runner: each OpenAIChat opens its own HTTP client,
# so a fresh one per round would put a new connection inside the timer,
# while PraisonAI and CrewAI agents share theirs.
llm = OpenAIChat(id=model)
def build():
agent = Agent(model=llm, instructions=[INSTRUCTIONS])
def call():
response = agent.run(prompt)
# Agno reports a failed run in ``status`` and puts the error text in
# ``content``, so the content alone would pass for an answer.
if str(getattr(response, 'status', '')).lower().endswith('error'):
raise RuntimeError(f"Agno run failed: {str(response.content)[:200]}")
return getattr(response, 'content', response)
return call
return build
def crewai_runner(model, prompt):
"""CrewAI: a new one-agent crew per run; only crew.kickoff() is timed."""
from crewai import Agent, Crew, Task
def build():
agent = Agent(role="Calculator", goal="Answer math questions",
backstory=INSTRUCTIONS, llm=model, verbose=False)
task = Task(description=prompt, expected_output="A number", agent=agent)
return Crew(agents=[agent], tasks=[task], verbose=False).kickoff
return build
def build_runners(model, prompt):
"""Return {framework: build()} for every installed framework.
``build()`` creates fresh objects (untimed) and returns the call to time.
"""
factories = [
('PraisonAI', lambda: praisonai_runner(model, prompt)),
('PraisonAI (LiteLLM)', lambda: praisonai_runner(f"openai/{model}", prompt)),
('Agno', lambda: agno_runner(model, prompt)),
('CrewAI', lambda: crewai_runner(model, prompt)),
]
runners = {}
for name, factory in factories:
try:
runners[name] = factory()
except ImportError as e:
print(f" {name}: skipped ({e})")
return runners
def answer_text(response):
"""The answer as text, or '' when the run produced nothing usable."""
if response is None:
return ''
text = str(response).strip()
return '' if text == 'None' else text
def run_benchmark(runners, iterations, warmup):
"""Warm up each framework, then time interleaved rounds.
A failed warm-up is reported but the framework stays in the timed rounds,
so one transient error does not discard it. The order within a round
rotates every round. Object construction happens
before the timer starts; only the execution call is timed.
Returns {framework: {"times": [...], "failures": [...]}}.
"""
results = {name: {"times": [], "failures": []} for name in runners}
for name, build in runners.items():
for _ in range(warmup):
try:
if not answer_text(build()()):
raise RuntimeError("empty answer")
except Exception as e:
print(f" {name}: warm-up failed ({type(e).__name__}: {str(e)[:120]})")
# Rotate the start each round so no framework always holds the same slot.
names = list(runners)
for i in range(iterations):
agent = Agent(
model=OpenAIChat(id=model),
instructions=["You are a helpful assistant. Be very brief."],
)
start = time.perf_counter()
response = agent.run(prompt)
elapsed = time.perf_counter() - start
times.append(elapsed)
# Extract content from response
content = str(response.content)[:50] if hasattr(response, 'content') else str(response)[:50]
print(f" Run {i+1}: {elapsed:.2f}s - Response: {content}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
shift = i % len(names) if names else 0
for name in names[shift:] + names[:shift]:
build = runners[name]
try:
call = build()
start = time.perf_counter()
text = answer_text(call())
elapsed = time.perf_counter() - start
error = None if text else "empty answer"
except Exception as e:
text, error = '', f"{type(e).__name__}: {e}"
if error:
results[name]["failures"].append(error)
print(f" Round {i + 1} {name}: FAILED ({error[:120]})")
else:
results[name]["times"].append(elapsed)
print(f" Round {i + 1} {name}: {elapsed:.2f}s - {text[:40]}")
return results
def benchmark_crewai(model, iterations, prompt):
"""Benchmark CrewAI agent execution using crew.kickoff()"""
from crewai import Agent, Task, Crew
print("\n--- CrewAI ---")
print("Method: crew.kickoff()")
times = []
for i in range(iterations):
agent = Agent(
role="Calculator",
goal="Answer math questions",
backstory="You are a helpful assistant.",
llm=model,
verbose=False
)
task = Task(
description=prompt,
expected_output="A number",
agent=agent
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=False
)
start = time.perf_counter()
response = crew.kickoff()
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f" Run {i+1}: {elapsed:.2f}s - Response: {str(response)[:50]}")
avg = sum(times) / len(times)
print(f" Average: {avg:.2f}s")
return avg
def summarise(results):
"""Rows of (framework, median, mean, ok, failed), fastest median first."""
rows = []
for name, r in results.items():
times = r["times"]
if times:
rows.append((name, statistics.median(times), statistics.fmean(times),
len(times), len(r["failures"])))
return sorted(rows, key=lambda row: row[1])
def save_results(results: dict, model: str, iterations: int, prompt: str):
def save_results(rows, results, model, iterations, warmup, prompt):
"""Save results to markdown file."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), 'EXECUTION_BENCHMARK_RESULTS.md')
versions = get_package_versions()
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
fastest = rows[0][1]
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Real Execution Benchmark\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write(f'**Model:** {model}\n')
f.write(f'**Iterations:** {iterations}\n')
f.write(f'**Iterations:** {iterations} interleaved rounds after {warmup} warm-up call(s) per framework\n')
f.write(f'**Prompt:** "{prompt}"\n\n')
f.write('## Results\n\n')
f.write('| Framework | Method | Avg Time | Relative |\n')
f.write('|-----------|--------|----------|----------|\n')
methods = {
'PraisonAI': 'agent.start()',
'PraisonAI (LiteLLM)': 'agent.start()',
'Agno': 'agent.run()',
'CrewAI': 'crew.kickoff()',
}
for name, avg in sorted_results:
ratio = avg / fastest
method = methods.get(name, 'unknown')
f.write('| Framework | Method | Median | Mean | Runs ok / failed | Relative |\n')
f.write('|-----------|--------|--------|------|------------------|----------|\n')
for name, median, mean, ok, failed in rows:
ratio = median / fastest
method = METHODS.get(name, 'unknown')
if ratio == 1.0:
f.write(f'| **{name}** | `{method}` | **{avg:.2f}s** | **1.00x (fastest)** |\n')
f.write(f'| **{name}** | `{method}` | **{median:.2f}s** | {mean:.2f}s | {ok} / {failed} | **1.00x (fastest)** |\n')
else:
f.write(f'| {name} | `{method}` | {avg:.2f}s | {ratio:.2f}x |\n')
f.write(f'| {name} | `{method}` | {median:.2f}s | {mean:.2f}s | {ok} / {failed} | {ratio:.2f}x |\n')
excluded = [name for name, r in results.items() if not r["times"]]
if excluded:
f.write(f'\nNo successful runs (not ranked): {", ".join(excluded)}\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('export OPENAI_API_KEY=your_key\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/execution_benchmark.py\n')
f.write('cd src/praisonai-agents\n')
f.write('python benchmarks/execution_benchmark.py --save\n')
f.write('```\n')
print(f'\nResults saved to: {filepath}')
if __name__ == '__main__':
def main():
parser = argparse.ArgumentParser(description='PraisonAI Agents - Real Execution Benchmark')
parser.add_argument('--save', action='store_true', help='Save results to file')
parser.add_argument('--save', action='store_true', help='Save results to EXECUTION_BENCHMARK_RESULTS.md')
parser.add_argument('--model', '-m', type=str, default=DEFAULT_MODEL, help=f'Model to use (default: {DEFAULT_MODEL})')
parser.add_argument('--iterations', '-i', type=int, default=DEFAULT_ITERATIONS, help=f'Number of iterations (default: {DEFAULT_ITERATIONS})')
parser.add_argument('--iterations', '-i', type=int, default=DEFAULT_ITERATIONS, help=f'Timed rounds (default: {DEFAULT_ITERATIONS})')
parser.add_argument('--warmup', '-w', type=int, default=DEFAULT_WARMUP, help=f'Untimed warm-up calls per framework (default: {DEFAULT_WARMUP})')
parser.add_argument('--prompt', '-p', type=str, default=DEFAULT_PROMPT, help=f'Prompt to use (default: "{DEFAULT_PROMPT}")')
args = parser.parse_args()
# Check API key
if args.iterations < 1:
parser.error('--iterations must be at least 1')
if args.warmup < 0:
parser.error('--warmup must be 0 or more')
if not os.environ.get('OPENAI_API_KEY'):
print("ERROR: Set OPENAI_API_KEY environment variable")
exit(1)
model = args.model
iterations = args.iterations
prompt = args.prompt
return 1
print('=' * 60)
print('PraisonAI Agents - Real Execution Benchmark')
print('=' * 60)
print(f'\nModel: {model}')
print(f'Iterations: {iterations}')
print(f'Prompt: "{prompt}"')
versions = get_package_versions()
print(f'\nModel: {args.model}')
print(f'Rounds: {args.iterations} (after {args.warmup} warm-up call(s) per framework)')
print(f'Prompt: "{args.prompt}"')
print('\nPackage versions:')
for pkg, ver in versions.items():
for pkg, ver in get_package_versions().items():
print(f' {pkg}: {ver}')
results = {}
# Run benchmarks
try:
results['PraisonAI'] = benchmark_praisonai(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI error: {e}")
try:
results['PraisonAI (LiteLLM)'] = benchmark_praisonai_litellm(model, iterations, prompt)
except Exception as e:
print(f"PraisonAI (LiteLLM) error: {e}")
try:
results['Agno'] = benchmark_agno(model, iterations, prompt)
except Exception as e:
print(f"Agno error: {e}")
try:
results['CrewAI'] = benchmark_crewai(model, iterations, prompt)
except Exception as e:
print(f"CrewAI error: {e}")
# Summary
print()
runners = build_runners(args.model, args.prompt)
results = run_benchmark(runners, args.iterations, args.warmup)
rows = summarise(results)
print('\n' + '=' * 60)
print('SUMMARY')
print('SUMMARY (median of successful runs)')
print('=' * 60)
if results:
sorted_results = sorted(results.items(), key=lambda x: x[1])
fastest = sorted_results[0][1]
print(f"\n{'Framework':<25} {'Avg Time':<12} {'Relative':<10}")
print('-' * 47)
for name, avg in sorted_results:
ratio = avg / fastest
print(f'{name:<25} {avg:<12.2f}s {ratio:.2f}x')
if rows:
fastest = rows[0][1]
print(f"\n{'Framework':<22} {'Median':>8} {'Mean':>8} {'ok/failed':>10} {'Relative':>9}")
print('-' * 61)
for name, median, mean, ok, failed in rows:
print(f'{name:<22} {median:>7.2f}s {mean:>7.2f}s {ok:>5}/{failed:<4} {median / fastest:>8.2f}x')
for name, r in results.items():
if not r["times"]:
print(f'{name:<22} no successful runs: {(r["failures"] or ["?"])[0][:80]}')
print('\n' + '=' * 60)
# Save results (if --save)
if results and args.save:
save_results(results, model, iterations, prompt)
if rows and args.save:
save_results(rows, results, args.model, args.iterations, args.warmup, args.prompt)
elif not args.save:
print('\nResults not saved (use --save flag to save results to file)')
return 0
if __name__ == '__main__':
raise SystemExit(main())
@@ -109,17 +109,11 @@ def run_benchmark(skip_crewai: bool = False, quiet: bool = False,
pass
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
@langchain_tool
def get_weather_lg(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['LangGraph'] = measure(
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[get_weather_lg])
lambda: create_react_agent(model=ChatOpenAI(model='gpt-4o-mini'), tools=[])
)
except ImportError:
pass
@@ -129,19 +123,12 @@ def run_benchmark(skip_crewai: bool = False, quiet: bool = False,
else:
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal['nyc', 'sf']):
"""Get weather info."""
return 'sunny' if city == 'sf' else 'cloudy'
results['CrewAI'] = measure(
lambda: CrewAgent(
role='Weather Agent',
goal='Provide weather info',
backstory='A weather expert',
tools=[get_weather_crew],
verbose=False
)
)
@@ -2698,6 +2698,8 @@ class Agent(GoalLoopMixin, SteeringMixin, SandboxMixin, SkillReviewMixin, Unifie
self.system_prompt = f"""{self.backstory}\n
Your Role: {self.role}\n
Your Goal: {self.goal}
""" if self.goal != self.backstory else f"""{self.backstory}\n
Your Role: {self.role}
"""
# Per-invocation system-prompt append (never persisted). Sourced from the
@@ -3571,8 +3573,26 @@ Your Goal: {self.goal}
def _replace_chat_history(self, new_history: List[Dict[str, Any]]):
"""Thread-safe replacement of entire chat history using proper locking."""
with self._history_lock.lock():
self._history_lock.value[:] = new_history
# Share _auto_save_session's lock so a save cannot interleave with the
# rewrite and the cursor update below (#6217).
with self._auto_save_lock, self._history_lock.lock():
history = self._history_lock.value
# Re-sync the positional auto-save cursor (#6217): resume at the
# first unsaved survivor, or right after the last survivor so
# appended messages (e.g. durable replays) are saved; summaries
# placed before survivors are not re-persisted.
last = getattr(self, '_auto_save_last_index', 0)
old = {id(m): i >= last for i, m in enumerate(history)}
cursor = len(new_history)
survivors = [i for i, m in enumerate(new_history) if id(m) in old]
if survivors:
cursor = survivors[-1] + 1
for i in survivors:
if old[id(new_history[i])]:
cursor = i
break
history[:] = new_history
self._auto_save_last_index = cursor
@property
def _cache_lock(self):
@@ -543,17 +543,53 @@ class ChatMixin:
return int(getattr(execution, "max_tool_calls_per_turn", default))
return default
def _resolve_harness_base_prompt(self):
_EDIT_TOOL_NAMES = frozenset({"edit_file", "apply_patch"})
def _offers_edit_tool(self, tools=None):
"""True when ``tools`` (default: the agent's tools) include a file-edit primitive."""
for tool in (tools if tools is not None else getattr(self, "tools", None)) or ():
if isinstance(tool, list):
# _format_tools_for_completion flattens a nested list but keeps
# only its function-schema dicts, so only those can offer an edit tool.
if self._offers_edit_tool([
t for t in tool
if isinstance(t, dict) and t.get("type") == "function"
and isinstance(t.get("function"), dict)
]):
return True
continue
if isinstance(tool, str):
name = tool
elif isinstance(tool, dict):
function = tool.get("function")
name = function.get("name") if isinstance(function, dict) else tool.get("name")
else:
# A tool's ``name`` is what the model is sent (``@tool(name=...)``
# keeps the function's own ``__name__``), so it wins.
name = getattr(tool, "name", None)
if not isinstance(name, str):
name = getattr(tool, "__name__", None)
# _prune_denied_tools drops denied tools from the payload, so a
# denied edit tool must not earn the hint either.
if name in self._EDIT_TOOL_NAMES and self._tool_name_allowed(name):
return True
return False
def _resolve_harness_base_prompt(self, tools=None):
"""Resolve the model-family harness base-prompt fragment, if any.
Honours an explicit ``harness_base_prompt`` override on the agent when
set; otherwise resolves lazily from the active model id. Returns an
empty string when no fragment applies (behaviour-preserving default).
Never raises: any resolution error collapses to no fragment.
set; otherwise resolves lazily from the active model id. The family
fragments are file-editing guidance, so they apply only when ``tools``
(default: the agent's tools) include ``edit_file`` or ``apply_patch``.
Returns an empty string when no fragment applies. Never raises: any
resolution error collapses to no fragment.
"""
override = getattr(self, "harness_base_prompt", None)
if override is not None:
return override
if not self._offers_edit_tool(tools):
return ""
try:
from ..model_harness import resolve_harness
# self.llm may be a string model id or an LLM object; resolve the
@@ -580,7 +616,10 @@ class ChatMixin:
# Check cache first (skip cache if memory is enabled since context is dynamic)
if not self._memory_instance:
tools_key = self._get_tools_cache_key(tools)
cache_key = f"{self.role}:{self.goal}:{tools_key}"
# tools=None keys as "none" whatever self.tools holds, so key the
# edit-hint decision too; otherwise a cached prefix would keep (or
# miss) the hint after the agent's tools change.
cache_key = f"{self.role}:{self.goal}:{hash(self.backstory)}:{tools_key}:{self._offers_edit_tool(tools)}"
cached_prompt = self._cache_get(self._system_prompt_cache, cache_key)
if cached_prompt is not None:
@@ -591,13 +630,16 @@ class ChatMixin:
cache_key = None # Don't cache when memory is enabled
system_prompt = f"""{self.backstory}\n
Your Role: {self.role}\n
Your Goal: {self.goal}"""
Your Role: {self.role}"""
# Agent(instructions=...) sets backstory and goal to the same text;
# state it once.
if self.goal != self.backstory:
system_prompt += f"\n\nYour Goal: {self.goal}"
# Prepend the model-family harness base prompt when one applies.
# Resolution is lazy and falls back to a behaviour-preserving default
# (no fragment) for unknown models, so output is unchanged by default.
harness_prompt = self._resolve_harness_base_prompt()
# (no fragment) for unknown models or agents without edit tools.
harness_prompt = self._resolve_harness_base_prompt(tools)
if harness_prompt:
system_prompt = f"{harness_prompt}\n\n{system_prompt}"
@@ -1204,7 +1246,8 @@ Your Goal: {self.goal}"""
if cached_entry is not None:
cached_tools, cached_metadata = cached_entry
self._tool_search_metadata = cached_metadata
return cached_tools
# Re-prune: permissions may have changed since the entry was cached.
return self._prune_denied_tools(cached_tools)
formatted_tools = []
# Provider-hosted tool specs have no local callable; the provider
@@ -1324,20 +1367,19 @@ Your Goal: {self.goal}"""
return sorted(tools_list, key=sort_key)
cleaned_tools = sort_formatted_tools(cleaned_tools)
# Prune permission-denied tools from the advertised set so the model is
# only ever offered tools it can actually call. Mirrors the execution-time
# enforcement in tool_execution.py (_perm_deny / _perm_allow). No-op when
# no deny set and no allow set are configured (backward compatible).
cleaned_tools = self._prune_denied_tools(cleaned_tools)
# Cache the formatted tools with LRU eviction, including tool search metadata
# Cache the unpruned tools with LRU eviction, including tool search
# metadata, so a later change to the deny/allow sets takes effect.
self._cache_put(
self._formatted_tools_cache,
cache_key,
(cleaned_tools, getattr(self, "_tool_search_metadata", None)),
)
return cleaned_tools
# Prune permission-denied tools from the advertised set so the model is
# only ever offered tools it can actually call. Mirrors the execution-time
# enforcement in tool_execution.py (_perm_deny / _perm_allow). No-op when
# no deny set and no allow set are configured (backward compatible).
return self._prune_denied_tools(cleaned_tools)
def _build_before_tool_definitions_input(self, formatted_tools):
"""Build the BEFORE_TOOL_DEFINITIONS hook input.
@@ -2042,6 +2084,21 @@ Your Goal: {self.goal}"""
return fn(*args, **kwargs)
return {**llm_kwargs, 'execute_tool_fn': tracked}, ran
@staticmethod
def _history_before_turn(history, content):
"""``history`` without the copy of this turn's user message stored before the call.
``LLM._build_messages`` appends ``prompt`` after ``chat_history``, so
forwarding the stored copy as well sends the prompt twice. Only a
trailing match is dropped: messages a durable replay restored after
the turn keep their order.
"""
if history:
last = history[-1]
if isinstance(last, dict) and last.get("role") == "user" and last.get("content") == content:
return list(history[:-1])
return history
def _custom_llm_hook_messages(self, llm_kwargs):
"""The custom-LLM request as the message list BEFORE_LLM / AFTER_LLM hooks see."""
messages = []
@@ -4090,7 +4147,7 @@ Your Goal: {self.goal}"""
llm_kwargs = dict(
prompt=llm_prompt,
system_prompt=system_prompt_for_llm,
chat_history=processed_history,
chat_history=self._history_before_turn(processed_history, normalized_content),
temperature=temperature,
tools=tool_param,
output_json=output_json,
@@ -4207,6 +4264,7 @@ Your Goal: {self.goal}"""
# Pass llm_prompt (which includes multimodal content if attachments present)
messages, original_prompt = self._build_messages(
llm_prompt, temperature, output_json, output_pydantic,
tools=tools,
use_native_format=use_native_format,
memory_prefetch_context=memory_prefetch_context,
)
@@ -4485,7 +4543,9 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
except Exception:
# Catch any exception from the inner try block and re-raise to outer handler
raise
except ToolExecutionError:
except (ToolExecutionError, BudgetExceededError):
# Parity with the custom-LLM and async paths: a budget stop must
# reach the caller, not turn into a ``None`` reply.
raise
except Exception as e:
self._last_stop_reason = "error"
@@ -4832,7 +4892,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
tools,
memory_prefetch_context=memory_prefetch_context,
),
'chat_history': effective_history,
'chat_history': self._history_before_turn(effective_history, normalized_content),
'temperature': temperature,
'tools': tools,
'output_json': output_json,
@@ -4926,6 +4986,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
temperature,
output_json,
output_pydantic,
tools=tools,
restore_durable=False,
memory_prefetch_context=memory_prefetch_context,
)
@@ -5264,7 +5325,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
response = await self._openai_client.async_client.chat.completions.create(
**tool_call_kwargs
)
result = await self._achat_completion(response, tools)
result = await self._achat_completion(response, tools, messages=messages)
if get_logger().getEffectiveLevel() == logging.DEBUG:
total_time = time.time() - start_time
logging.debug(f"Agent.achat completed in {total_time:.2f} seconds")
@@ -5444,8 +5505,12 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
# the except blocks above already read ownership before this runs.
self._clear_turn_tracking()
async def _achat_completion(self, response, tools, reasoning_steps=False):
"""Async version of _chat_completion method"""
async def _achat_completion(self, response, tools, reasoning_steps=False, messages=None):
"""Async version of _chat_completion method.
``messages`` is the conversation that produced ``response``; it is kept in
the follow-up request so the final answer still sees the user's question.
"""
try:
message = response.choices[0].message
if not hasattr(message, 'tool_calls') or not message.tool_calls:
@@ -5531,17 +5596,20 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
if results:
formatted_results = "\n".join([str(r) for r in results if r is not None])
if formatted_results:
messages = [
{"role": "system", "content": self.system_prompt},
if messages:
follow_up_messages = list(messages)
else:
follow_up_messages = [{"role": "system", "content": self.system_prompt}]
follow_up_messages.extend([
{"role": "assistant", "content": "Here are the tool results:"},
{"role": "user", "content": formatted_results + "\nPlease process these results and provide a final response."}
]
])
try:
final_response = await self._openai_client.async_client.chat.completions.create(
model=self.llm,
messages=messages,
messages=follow_up_messages,
temperature=1.0,
stream=stream
stream=True
)
full_response_text = ""
reasoning_content = ""
@@ -5551,6 +5619,9 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
# Process stream without display_generating since streaming is active
async for chunk in final_response:
chunks.append(chunk)
if not chunk.choices:
# e.g. Azure content-filter annotation chunks
continue
if chunk.choices[0].delta.content:
full_response_text += chunk.choices[0].delta.content
@@ -5558,9 +5629,18 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
rc = chunk.choices[0].delta.reasoning_content
if rc:
reasoning_content += rc
if not full_response_text.strip() and not reasoning_content.strip():
# No answer text (only annotation, contentless or
# whitespace-only chunks): fall back to the tool results via the
# except below.
raise ValueError("follow-up stream returned no answer text")
self.console.print()
console = self.console # None unless verbose
if console is not None:
console.print()
from .agent import _get_llm_functions
final_response = _get_llm_functions()['process_stream_chunks'](chunks)
# Return only reasoning content if reasoning_steps is True
if reasoning_steps and hasattr(final_response.choices[0].message, 'reasoning_content') and final_response.choices[0].message.reasoning_content:
@@ -5878,7 +5958,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
for chunk in self.llm_instance.get_response_stream(
prompt=actual_prompt,
system_prompt=stream_system_prompt,
chat_history=stream_history,
chat_history=self._history_before_turn(stream_history, normalized_content),
temperature=kwargs.get('temperature', 1.0),
tools=tool_param,
output_json=kwargs.get('output_json'),
@@ -5953,6 +6033,7 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
# Build messages using the helper method
messages, original_prompt = self._build_messages(actual_prompt, kwargs.get('temperature', 1.0),
kwargs.get('output_json'), kwargs.get('output_pydantic'),
tools=tool_param,
memory_prefetch_context=memory_prefetch_context)
# Apply context management so the streaming path compacts long
@@ -213,7 +213,8 @@ class ContextAgent(Agent):
self.output_dir / "agent_responses",
self.output_dir / "markdown_outputs",
self.output_dir / "debug_logs",
self.output_dir / "final_results"
self.output_dir / "final_results",
self.output_dir / "PRPs",
]
for directory in directories:
@@ -1270,11 +1271,12 @@ Every agent interaction has been saved for full audit trail and reproducibility.
return self._generate_combined_analysis(feature_request)
# Otherwise just save and return the PRP
from ..utils.atomic_io import atomic_write_text, safe_join
prp_dir = os.path.join(self.output_dir, "PRPs")
os.makedirs(prp_dir, exist_ok=True)
safe_filename = feature_request.lower().replace(' ', '_').replace('-', '_')[:50]
prp_path = os.path.join(self.output_dir, "PRPs", f"{safe_filename}.md")
with open(prp_path, "w", encoding="utf-8") as f:
f.write(prp)
prp_path = safe_join(prp_dir, safe_filename, ".md")
atomic_write_text(prp_path, prp)
print(f"📄 Comprehensive PRP following PRD template saved to: {prp_path}")
return prp
@@ -55,6 +55,27 @@ def _live_turn_message_ids() -> set:
ids.update(id(m) for m in owned)
return ids
def _exchange_spans(history: list) -> list:
"""Split history into ``(start, end)`` units that must stay together.
An assistant turn carrying ``tool_calls`` and the ``role="tool"`` results
that follow it form one unit; every other message is its own unit. History
mutators cut/delete on unit boundaries so they never leave an orphaned tool
result or a dangling ``tool_calls`` turn, which providers reject (#6217).
"""
spans = []
i, n = 0, len(history)
while i < n:
j = i + 1
msg = history[i]
if msg.get("role") == "assistant" and msg.get("tool_calls"):
while j < n and history[j].get("role") == "tool":
j += 1
spans.append((i, j))
i = j
return spans
# Shared lazy display helpers (cached, thread-safe; avoid circular imports)
from ._lazy_display import _get_console, _get_live, _get_display_functions
@@ -178,6 +199,11 @@ class MemoryMixin:
length: Target length for chat history
"""
with self._history_lock:
# Never cut inside a tool exchange: back off to its start (#6217).
for start, end in _exchange_spans(self.chat_history):
if start < length < end:
length = start
break
self.chat_history = self.chat_history[:length]
def _begin_turn_tracking(self):
@@ -299,8 +325,14 @@ class MemoryMixin:
if len(self.chat_history) <= keep_last:
return 0
deleted_count = len(self.chat_history) - keep_last
self.chat_history = self.chat_history[-keep_last:]
cut = len(self.chat_history) - keep_last
# Never start the kept tail inside a tool exchange (#6217).
for start, end in _exchange_spans(self.chat_history):
if start < cut < end:
cut = end
break
deleted_count = cut
self.chat_history = self.chat_history[cut:]
# Reset auto-save index to match new history length
self._auto_save_last_index = len(self.chat_history)
return deleted_count
@@ -318,8 +350,17 @@ class MemoryMixin:
True if deleted, False if index out of range
"""
with self._history_lock:
n = len(self.chat_history)
if not -n <= index < n:
return False
index %= n
# Deleting part of a tool exchange removes the whole exchange so no
# orphaned tool result / dangling tool_calls is left (#6217).
for start, end in _exchange_spans(self.chat_history):
if start <= index < end:
break
try:
del self.chat_history[index]
del self.chat_history[start:end]
# Adjust auto-save index if deletion affects saved range
if hasattr(self, '_auto_save_last_index'):
self._auto_save_last_index = min(
@@ -344,10 +385,18 @@ class MemoryMixin:
"""
with self._history_lock:
original_len = len(self.chat_history)
self.chat_history = [
msg for msg in self.chat_history
if pattern.lower() not in msg.get("content", "").lower()
]
needle = pattern.lower()
history = self.chat_history
kept = []
# Match per exchange so a hit inside a tool exchange drops the whole
# exchange; content may be None on tool-calling turns (#6217).
for start, end in _exchange_spans(history):
unit = history[start:end]
if not any(
needle in str(m.get("content") or "").lower() for m in unit
):
kept.extend(unit)
self.chat_history = kept
deleted_count = original_len - len(self.chat_history)
# Adjust auto-save index if deletion affects saved range
if deleted_count > 0 and hasattr(self, '_auto_save_last_index'):
@@ -569,11 +618,10 @@ class MemoryMixin:
if history:
# Only restore if chat_history is empty (avoid duplicates)
if not self.chat_history:
# The store returns LLM-shaped dicts (to_llm_message), so
# keep tool_calls / tool_call_id intact (#6217).
for msg in history:
self.chat_history.append({
"role": msg["role"],
"content": msg["content"]
})
self.chat_history.append(dict(msg))
logging.info(f"Restored session {self._session_id} with {len(history)} messages from JSON store")
# Set agent info
@@ -862,7 +910,7 @@ class MemoryMixin:
except Exception as e:
logging.debug(f"Discard partial assistant turn skipped: {e}")
def _add_message_with_tool_fields(self, role, content, **tool_fields):
def _add_message_with_tool_fields(self, role, content, session_id=None, **tool_fields):
"""Add a message carrying tool fields, tolerating stores without them.
The default JSON store and the SQLite store accept ``tool_calls`` /
@@ -874,11 +922,13 @@ class MemoryMixin:
back to a plain-text turn so no exchange is silently lost.
"""
add_message = self._session_store.add_message
if self._store_supports_tool_fields(add_message):
add_message(self._session_id, role, content, **tool_fields)
if session_id is None:
session_id = self._session_id
if tool_fields and self._store_supports_tool_fields(add_message):
add_message(session_id, role, content, **tool_fields)
return
# Legacy store: preserve the turn as plain text so resume still sees it.
add_message(self._session_id, role, content)
add_message(session_id, role, content)
def _store_supports_tool_fields(self, add_message) -> bool:
"""Return True if ``add_message`` accepts the tool keyword fields."""
@@ -1195,10 +1245,17 @@ class MemoryMixin:
if self._session_store is not None:
# Persist only NEW messages to SessionStore
for msg in new_messages:
self._session_store.add_message(
self.auto_save, # Use auto_save name as session_id
role=msg.get("role", "user"),
content=msg.get("content", ""),
tool_fields = {
k: msg[k]
for k in ("tool_calls", "tool_call_id")
if msg.get(k) is not None
}
# Persist tool fields so resumed tool turns stay valid (#6217)
self._add_message_with_tool_fields(
msg.get("role", "user"),
msg.get("content") or "",
session_id=self.auto_save,
**tool_fields,
)
# Update last saved index
self._auto_save_last_index = len(clean_history)
@@ -4362,8 +4362,13 @@ class ToolExecutionMixin:
return "Tool search not in bridge mode"
# Get deferrable tools from metadata
deferrable_tools = metadata.get("deferrable_tools", [])
# Hide permission-denied tools from search/describe; the cached
# metadata is built before pruning, so filter locally here.
deferrable_tools = [
t for t in metadata.get("deferrable_tools", [])
if self._tool_name_allowed(t.get("function", {}).get("name", ""))
]
if function_name == "tool_search":
# Handle tool_search bridge call
try:
@@ -957,7 +957,8 @@ def _validate_feature_config(feature: str, config: Dict[str, Any]) -> List[str]:
"guardrails": {"enabled", "max_retries", "validator"},
"web": {"enabled", "search", "fetch", "provider"},
"output": {"enabled", "verbose", "markdown", "stream", "metrics", "preset"},
"execution": {"enabled", "max_iter", "max_rpm", "max_execution_time", "preset"},
"execution": {"enabled", "max_iter", "max_rpm", "max_execution_time", "preset",
"max_budget", "on_budget_exceeded", "durable", "journal_path", "resume_run_id"},
"caching": {"enabled", "prompt_caching", "ttl"},
"autonomy": {"enabled", "max_steps", "approval_required"},
"context": {"enabled", "max_tokens", "strategy"},
@@ -18,6 +18,8 @@ from .._lazy import create_lazy_getattr
# 283 KB / 7,201-line ``protocols.py`` (and siblings) out of package-import
# time; each submodule is parsed only when one of its symbols is first
# touched. Mirrors the ``mcp``/``skills`` packages (AGENTS.md §4.2).
_PROTOCOLS_MODULE = "praisonaiagents.gateway.protocols"
_LAZY_IMPORTS = {
# protocols.py (parsed lazily, on first touch)
"GatewayProtocol": ("praisonaiagents.gateway.protocols", "GatewayProtocol"),
@@ -229,7 +231,9 @@ _LAZY_IMPORTS = {
# Schema-validated inbound frame codec (Issue #2831)
"HelloParams": ("praisonaiagents.gateway.protocols", "HelloParams"),
# Device-identity handshake contract (Issue #5998)
"DeviceAuth": ("praisonaiagents.gateway.protocols", "DeviceAuth"),
# Module path via a constant: GitGuardian reads a ("...", "DeviceAuth")
# string pair as an "authentication tuple" and fails every PR that merges main.
"DeviceAuth": (_PROTOCOLS_MODULE, "DeviceAuth"),
"DeviceIdentityProtocol": ("praisonaiagents.gateway.protocols", "DeviceIdentityProtocol"),
"device_id_from_public_key": ("praisonaiagents.gateway.protocols", "device_id_from_public_key"),
"HelloResult": ("praisonaiagents.gateway.protocols", "HelloResult"),
@@ -3841,12 +3841,19 @@ class ConversationRequestProtocol(Protocol):
# import lives in core.
# ---------------------------------------------------------------------------
GatewayRequestKind = Literal["approval", "choice", "input"]
GatewayRequestKind = Literal["approval", "choice", "input", "secret"]
"""Closed set of server→client interactive request kinds.
* ``approval`` — allow/deny; the reply ``value`` is ``"allow"`` or ``"deny"``.
* ``choice`` — select one of ``options``; the reply ``value`` is the option.
* ``input`` — free text; the reply ``value`` is the entered text.
* ``secret`` — a credential (password, API key, one-time code) captured
mid-turn (Issue #6223). Clients MUST render a masked input. Isolation
contract: the reply ``value`` is returned *only* to the caller of
:meth:`GatewayRequestChannelProtocol.request`; implementations MUST NOT log,
journal, echo, or persist it, and the caller (a tool/callback) MUST NOT
return it to the model or append it to history. A tool obtains one via
``get_request_channel().request(GatewayServerRequest(kind="secret", ...))``.
"""
@@ -3893,11 +3900,13 @@ class GatewayServerReply:
Attributes:
request_id: The id of the request being answered.
value: ``"allow"``/``"deny"`` for approval, the chosen option for
choice, or the entered text for input.
choice, the entered text for input, or the credential for secret.
Excluded from ``repr`` so a logged/traced reply never leaks a
secret.
"""
request_id: str
value: str
value: str = field(repr=False)
def as_dict(self) -> Dict[str, Any]:
"""Convert to a serializable dictionary."""
@@ -572,6 +572,14 @@ class FileMemory:
promoted.append(item.id)
if promoted:
# Promotion writes the same store as add_long_term, so retain the
# same highest-importance, newest-first items within its capacity.
limit = self.config["long_term_limit"]
if len(self._long_term) > limit:
self._long_term.sort(
key=lambda item: (item.importance, item.created_at), reverse=True
)
self._long_term = self._long_term[:limit]
self._save_long_term()
self._log(f"Auto-promoted {len(promoted)} items to long-term memory")
@@ -104,7 +104,12 @@ class PlanStorage:
if filename is None:
filename = self._get_plan_filename(plan)
path = os.path.join(self.plans_dir, filename)
from ..utils.atomic_io import safe_join
# Keep ".md" as the suffix so listing/lookup still find the file.
stem, ext = os.path.splitext(filename)
if ext.lower() != ".md":
stem, ext = filename, ".md"
path = safe_join(self.plans_dir, stem, ext)
# Convert plan to markdown
markdown = plan.to_markdown()
@@ -215,6 +220,29 @@ class PlanStorage:
return False
def _todo_path(self, name: str) -> str:
"""Confined path for a todo name, reusing a pre-existing legacy file.
Legacy files (``<name>.json`` written before Issue #6238) are reused
only when ``name`` is a plain filename, so upgrades keep working
without ever allowing traversal out of ``todos_dir``.
"""
from ..utils.atomic_io import safe_join
path = safe_join(self.todos_dir, name, ".json")
name = str(name)
if (
not os.path.exists(path)
and name not in ("", ".", "..")
and "\0" not in name
and "%" not in name
and os.path.basename(name) == name
and not (os.altsep and os.altsep in name)
):
legacy = os.path.join(self.todos_dir, f"{name}.json")
if os.path.isfile(legacy):
return legacy
return path
def save_todo(self, todo: TodoList, name: str = "current") -> str:
"""
Save a todo list to storage.
@@ -226,11 +254,10 @@ class PlanStorage:
Returns:
Path to the saved file
"""
filename = f"{name}.json"
path = os.path.join(self.todos_dir, filename)
from ..utils.atomic_io import atomic_write_text
path = self._todo_path(name)
# Atomic write so a crash mid-write never truncates the previous todos.
from ..utils.atomic_io import atomic_write_text
atomic_write_text(path, todo.to_json())
logger.debug(f"Saved todo list to {path}")
@@ -246,8 +273,7 @@ class PlanStorage:
Returns:
TodoList instance, or None if not found
"""
filename = f"{name}.json"
path = os.path.join(self.todos_dir, filename)
path = self._todo_path(name)
if not os.path.exists(path):
return None
@@ -265,12 +291,13 @@ class PlanStorage:
Returns:
List of todo list names
"""
from urllib.parse import unquote
todos = []
for filename in os.listdir(self.todos_dir):
if filename.endswith(".json"):
name = filename[:-5] # Remove .json extension
todos.append(name)
# Decode so listed names round-trip through load/delete_todo.
todos.append(unquote(filename[:-5]))
return todos
@@ -284,8 +311,7 @@ class PlanStorage:
Returns:
True if deleted, False if not found
"""
filename = f"{name}.json"
path = os.path.join(self.todos_dir, filename)
path = self._todo_path(name)
if os.path.exists(path):
os.remove(path)
@@ -73,28 +73,15 @@ def generate_title(
fallback_title = _create_fallback_title(user_msg, max_length)
try:
# Check if we're already in an event loop
asyncio.get_running_loop()
except RuntimeError:
# No running loop - safe to use asyncio.run
try:
return asyncio.run(generate_title_async(
user_msg, assistant_msg, llm_model, timeout, max_length,
primary_model=primary_model,
))
except Exception:
import logging
logger = logging.getLogger(__name__)
logger.debug("Title generation failed in sync context", exc_info=True)
return fallback_title
else:
# Already in event loop - cannot use asyncio.run
# Works with or without a running event loop (Issue #6238).
from ..utils.async_bridge import run_coroutine_from_any_context
return run_coroutine_from_any_context(generate_title_async(
user_msg, assistant_msg, llm_model, timeout, max_length,
primary_model=primary_model,
), timeout=None)
except Exception:
import logging
logger = logging.getLogger(__name__)
logger.warning(
"generate_title() called from running event loop; "
"use generate_title_async() instead. Returning fallback title."
)
logging.getLogger(__name__).debug("Title generation failed", exc_info=True)
return fallback_title
@@ -645,9 +645,8 @@ def crawl4ai_sync(
Returns:
Dict with crawl results
"""
return asyncio.run(
crawl4ai(url, css_selector, js_code, wait_for)
)
from ..utils.async_bridge import run_coroutine_from_any_context
return run_coroutine_from_any_context(crawl4ai(url, css_selector, js_code, wait_for), timeout=None)
def crawl4ai_extract_sync(
url: str,
@@ -664,9 +663,8 @@ def crawl4ai_extract_sync(
Returns:
Dict with extracted data
"""
return asyncio.run(
crawl4ai_extract(url, schema, js_code)
)
from ..utils.async_bridge import run_coroutine_from_any_context
return run_coroutine_from_any_context(crawl4ai_extract(url, schema, js_code), timeout=None)
if __name__ == "__main__":
# Example usage
@@ -7,6 +7,15 @@ from datetime import datetime
from typing import Dict, List, Optional, Any
from .decorator import tool
import warnings
warnings.warn(
"praisonaiagents.tools.jira_tools is deprecated and will be removed in a future release; "
"use praisonai_tools.tools.jira_watch_tool instead (pip install praisonai-tools)",
DeprecationWarning,
stacklevel=2,
)
logger = logging.getLogger(__name__)
def _get_jira_connection(
@@ -126,11 +126,31 @@ def _matches_glob(fname: str, rel: str, glob: Optional[str]) -> bool:
if not glob:
return True
posix = rel.replace(os.sep, "/")
return (
if (
fnmatch.fnmatch(fname, glob)
or fnmatch.fnmatch(rel, glob)
or fnmatch.fnmatch(posix, glob)
)
):
return True
# fnmatch treats "**/" as requiring a slash. Recursive glob segments
# also match zero directories, as they do on the ripgrep path.
segments = glob.split("/")
if "**" not in segments:
return False
parts = posix.split("/")
positions = {0}
for segment in segments:
if not positions:
return False
if segment == "**":
positions = set(range(min(positions), len(parts) + 1))
else:
positions = {
i + 1 for i in positions
if i < len(parts) and fnmatch.fnmatch(parts[i], segment)
}
return len(parts) in positions
def _iter_files(root: str, glob: Optional[str], ignore_matcher):
@@ -26,6 +26,15 @@ import logging
import os
from importlib import util
import warnings
warnings.warn(
"praisonaiagents.tools.youdotcom_tools is deprecated and will be removed in a future release; "
"use praisonai_tools.tools.youdotcom_tool instead (pip install praisonai-tools)",
DeprecationWarning,
stacklevel=2,
)
def _check_youdotcom_available() -> tuple[bool, Optional[str]]:
"""Check if You.com SDK is available and API key is set.
@@ -9,8 +9,10 @@ the previous good file: the old bytes stay intact and ``os.replace`` swaps in th
new content atomically on both POSIX and Windows.
"""
import hashlib
import json
import os
import re
import tempfile
import threading
import time
@@ -21,9 +23,52 @@ __all__ = [
"atomic_write_json",
"atomic_write_text",
"quarantine_corrupt_file",
"safe_join",
"update_json",
]
# Max UTF-8 bytes for the final basename; leaves room under the common
# 255-byte limit for ``atomic_write_text``'s ``<basename>.XXXXXXXX.tmp`` file.
_MAX_NAME_BYTES = 200
_HASHED = re.compile(r"(.*)%~([0-9a-f]{16})", re.DOTALL)
_ESCAPE = re.compile(r'[%/\\:*?"<>|\x00-\x1f\x7f]')
def _escape(name: str) -> str:
stem = _ESCAPE.sub(lambda m: "%%%02X" % ord(m.group()), name)
return "%2E" + stem[1:] if stem.startswith(".") else stem
def safe_join(base: str, name: str, suffix: str = "") -> str:
"""Join a caller-supplied ``name`` under ``base`` without escaping it.
Only ``%``, path separators, control and Windows-reserved characters (and
a leading ``.``) are percent-encoded, so ordinary names - including spaces
and non-ASCII - keep their pre-#6238 filename, ``..`` and separators can
never leave ``base``, and ``urllib.parse.unquote`` recovers the name.
Names whose filename would exceed ``_MAX_NAME_BYTES`` are truncated and
tagged ``%~<sha256[:16]>``; that listed (unquoted) form maps back to the
same file, while no ordinary name can produce a raw ``%~`` (Issue #6238).
"""
name = str(name)
hashed = _HASHED.fullmatch(name)
if hashed:
stem = _escape(hashed.group(1)) + "%~" + hashed.group(2)
else:
stem = _escape(name)
budget = _MAX_NAME_BYTES - len(suffix.encode("utf-8"))
if len(stem.encode("utf-8", "surrogatepass")) > budget:
tag = "%~" + hashlib.sha256(name.encode("utf-8", "surrogatepass")).hexdigest()[:16]
kept, used = [], len(tag)
for tok in re.findall(r"%[0-9A-F]{2}|.", _escape(name), re.DOTALL):
used += len(tok.encode("utf-8", "surrogatepass"))
if used > budget:
break
kept.append(tok)
stem = "".join(kept) + tag
return os.path.join(base, stem + suffix)
# Per-path in-process mutexes so threads in one process serialise before they
# even reach the cross-process ``FileLock``. Keyed by absolute path.
_path_mutexes: "defaultdict[str, threading.RLock]" = defaultdict(threading.RLock)
@@ -473,7 +473,10 @@ class YAMLWorkflowParser:
if 'reasoning' in role_config:
agent['reasoning'] = role_config['reasoning']
if isinstance(role_config.get('execution'), dict):
agent['execution'] = role_config['execution']
agents[role_id] = agent
return agents
@@ -591,7 +594,11 @@ class YAMLWorkflowParser:
# so we don't override the Agent's own caching default.
if cache is False:
agent_kwargs['caching'] = False
_execution: Dict[str, Any] = {}
# An agent-level `execution:` mapping passes straight through to
# ExecutionConfig (e.g. max_budget, on_budget_exceeded, durable,
# journal_path, resume_run_id); flat max_iter/max_execution_time win.
_yaml_execution = config.get('execution')
_execution: Dict[str, Any] = dict(_yaml_execution) if isinstance(_yaml_execution, dict) else {}
if max_iter is not None:
_execution['max_iter'] = max_iter
if max_execution_time is not None:
+1 -1
View File
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "1.7.21"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
description = "Open-source Python AI agent framework that runs on your own computers, even fully offline."
readme = "README.md"
requires-python = ">=3.10"
authors = [
@@ -0,0 +1,38 @@
"""Live check that async chat with a tool returns a model-written final answer.
Exercises the post-tool follow-up request in ``ChatMixin._achat_completion``
by turning off the default unified dispatch, so ``_achat_impl`` takes the
legacy OpenAI-client branch. Skipped unless ``OPENAI_API_KEY`` is set.
"""
import os
import pytest
from praisonaiagents import Agent
def get_secret_number() -> str:
"""Return the secret number."""
return "4217"
@pytest.mark.asyncio
@pytest.mark.skipif(not os.getenv("OPENAI_API_KEY"), reason="No API key")
async def test_achat_with_tool_returns_final_answer_live():
agent = Agent(
name="test",
instructions="You are a helpful assistant",
llm="gpt-4o-mini",
tools=[get_secret_number],
)
# The default unified path never calls _achat_completion.
agent._use_unified_llm_dispatch = False
result = await agent.astart(
"Use the get_secret_number tool, then reply with one sentence that "
"contains the secret number."
)
print(f"Agent output: {result}")
assert result
assert "4217" in str(result)
# Without the fix the raw tool output ("4217") is returned unchanged.
assert str(result).strip() != "4217"
@@ -169,3 +169,264 @@ async def test_execute_tool_async_resolves_aliased_function_name():
)
assert result == "aliased-ran"
@pytest.mark.asyncio
async def test_achat_completion_streams_final_response_after_tool_calls():
"""After running tools, the legacy async path must ask the model for a
final answer instead of failing and returning the raw tool output.
``_achat_completion`` used to reference an undefined ``stream`` name and
``_get_llm_functions`` (only defined in ``agent.py``), so the follow-up
completion always raised ``NameError``, which was swallowed and the raw
tool results were returned.
"""
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
def lookup(query: str) -> str:
return "tool output"
tool_call = SimpleNamespace(
id="call-1",
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
)
response = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
)
def _chunk(content):
return SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(content=content))]
)
async def _stream():
for part in ("final ", "answer"):
yield _chunk(part)
create = AsyncMock(return_value=_stream())
fake_client = SimpleNamespace(
async_client=SimpleNamespace(
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
)
)
final = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="final answer"))]
)
process_stream_chunks = MagicMock(return_value=final)
with patch.object(
type(agent), "_openai_client", new=fake_client
), patch.object(
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
), patch(
"praisonaiagents.agent.agent._get_llm_functions",
return_value={"process_stream_chunks": process_stream_chunks},
), patch(
"praisonaiagents.agent.chat_mixin._get_display_functions"
) as display:
result = await agent._achat_completion(response, tools=[lookup])
display.return_value.__getitem__.return_value.assert_not_called()
create.assert_awaited_once()
assert create.await_args.kwargs["stream"] is True
process_stream_chunks.assert_called_once()
assert result == "final answer"
@pytest.mark.asyncio
async def test_achat_completion_follow_up_keeps_conversation_and_skips_empty_chunks():
"""The post-tool follow-up request must keep the original conversation (so
the final answer still sees the user's question), and stream chunks with no
``choices`` (e.g. Azure content-filter annotations) must be skipped rather
than raising ``IndexError`` and falling back to the raw tool output.
"""
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
def lookup(query: str) -> str:
return "tool output"
tool_call = SimpleNamespace(
id="call-1",
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
)
response = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
)
conversation = [
{"role": "system", "content": "You are helpful"},
{"role": "user", "content": "Look up x and answer in French"},
]
async def _stream():
yield SimpleNamespace(choices=[])
yield SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(content="réponse"))]
)
create = AsyncMock(return_value=_stream())
fake_client = SimpleNamespace(
async_client=SimpleNamespace(
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
)
)
final = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="réponse"))]
)
process_stream_chunks = MagicMock(return_value=final)
with patch.object(
type(agent), "_openai_client", new=fake_client
), patch.object(
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
), patch(
"praisonaiagents.agent.agent._get_llm_functions",
return_value={"process_stream_chunks": process_stream_chunks},
), patch(
"praisonaiagents.agent.chat_mixin._get_display_functions"
) as display:
result = await agent._achat_completion(
response, tools=[lookup], messages=conversation
)
display.return_value.__getitem__.return_value.assert_not_called()
sent = create.await_args.kwargs["messages"]
assert sent[: len(conversation)] == conversation
assert "tool output" in sent[-1]["content"]
# The caller's list is not mutated.
assert len(conversation) == 2
assert result == "réponse"
@pytest.mark.asyncio
async def test_legacy_achat_impl_forwards_conversation_to_tool_follow_up():
"""The legacy ``_achat_impl`` tool branch must pass the conversation to
``_achat_completion`` so the follow-up request still has the user's prompt.
"""
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
agent._using_custom_llm = False
agent._use_unified_llm_dispatch = False
def lookup(query: str) -> str:
return "tool output"
tool_call = SimpleNamespace(
id="call-1",
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
)
tool_response = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
)
async def _stream():
yield SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(content="final answer"))]
)
create = AsyncMock(side_effect=[tool_response, _stream()])
fake_client = SimpleNamespace(
async_client=SimpleNamespace(
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
)
)
final = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="final answer"))]
)
prompt_messages = [{"role": "user", "content": "Look up x for me"}]
with patch.object(type(agent), "_openai_client", new=fake_client), patch.object(
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
), patch(
"praisonaiagents.agent.agent._get_llm_functions",
return_value={"process_stream_chunks": MagicMock(return_value=final)},
), patch.object(
agent, "_atrigger_after_agent_hook", new_callable=AsyncMock, side_effect=lambda p, r, t: r
), patch.object(agent, "_execute_callback_and_display"), patch.object(
agent, "_build_messages", return_value=(list(prompt_messages), "Look up x for me")
), patch.object(agent, "_persist_message"), patch.object(
agent, "_format_tools_for_completion", return_value=[{"type": "function"}]
):
result = await agent._achat_impl(
prompt="Look up x for me",
temperature=1.0,
tools=[lookup],
output_json=None,
output_pydantic=None,
reasoning_steps=False,
stream=False,
task_name=None,
task_description=None,
task_id=None,
config=None,
force_retrieval=False,
skip_retrieval=True,
attachments=None,
_trace_emitter=MagicMock(),
)
assert create.await_count == 2
follow_up = create.await_args_list[1].kwargs["messages"]
assert {"role": "user", "content": "Look up x for me"} in follow_up
assert result == "final answer"
@pytest.mark.asyncio
@pytest.mark.parametrize(
"follow_up_chunk",
[
pytest.param(SimpleNamespace(choices=[]), id="empty-choices"),
pytest.param(
SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(content=None))]
),
id="choice-without-content",
),
pytest.param(
SimpleNamespace(
choices=[SimpleNamespace(delta=SimpleNamespace(content="\n "))]
),
id="whitespace-only-content",
),
],
)
async def test_achat_completion_falls_back_when_follow_up_has_no_answer_chunks(follow_up_chunk):
"""A follow-up stream with no answer text (only empty-``choices`` chunks,
choices without ``delta.content`` or whitespace-only content) must log an error and return the tool
results, not an empty string.
"""
agent = Agent(name="test", instructions="You are helpful", llm="gpt-4o-mini")
def lookup(query: str) -> str:
return "tool output"
tool_call = SimpleNamespace(
id="call-1",
function=SimpleNamespace(name="lookup", arguments='{"query": "x"}'),
)
response = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(tool_calls=[tool_call]))]
)
async def _stream():
yield follow_up_chunk
create = AsyncMock(return_value=_stream())
fake_client = SimpleNamespace(
async_client=SimpleNamespace(
chat=SimpleNamespace(completions=SimpleNamespace(create=create))
)
)
process_stream_chunks = MagicMock()
with patch.object(type(agent), "_openai_client", new=fake_client), patch.object(
agent, "execute_tool_async", new_callable=AsyncMock, return_value="tool output"
), patch(
"praisonaiagents.agent.agent._get_llm_functions",
return_value={"process_stream_chunks": process_stream_chunks},
), patch(
"praisonaiagents.agent.chat_mixin._get_display_functions"
) as display:
result = await agent._achat_completion(response, tools=[lookup])
display.return_value.__getitem__.return_value.assert_called()
process_stream_chunks.assert_not_called()
assert result == "tool output"
@@ -0,0 +1,20 @@
"""Sync OpenAI-client chat must surface BudgetExceededError, not return None."""
from unittest.mock import patch
import pytest
from praisonaiagents import Agent, BudgetExceededError, ExecutionConfig
def test_sync_chat_reraises_budget_error(monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "sk-test") # client is built lazily; no request is sent
agent = Agent(
name="b", instructions="x", llm="gpt-4o-mini", output="silent",
execution=ExecutionConfig(max_budget=0.01),
)
with patch.object(
agent, "_chat_completion",
side_effect=BudgetExceededError("b", 0.02, 0.01),
):
with pytest.raises(BudgetExceededError):
agent.chat("hello")
@@ -0,0 +1,137 @@
"""Chat-history integrity for tool turns (Issue #6217)."""
import pytest
from praisonaiagents import Agent
from praisonaiagents.config import MemoryConfig
from praisonaiagents.session import DefaultSessionStore
def _tool_history():
return [
{"role": "user", "content": "weather in Paris?"},
{"role": "assistant", "content": None, "tool_calls": [{
"id": "call_1", "type": "function",
"function": {"name": "weather", "arguments": "{\"city\":\"Paris\"}"},
}]},
{"role": "tool", "tool_call_id": "call_1", "content": "18C sunny"},
{"role": "assistant", "content": "It is 18C and sunny."},
]
def _agent(session_id=None, store=None):
kwargs = {}
if session_id:
kwargs["memory"] = MemoryConfig(history=True, auto_save=session_id)
a = Agent(name="a", instructions="x", model="gpt-4o-mini", **kwargs)
if store is not None:
a._session_store = store
a._init_session_store()
return a
@pytest.fixture
def store(tmp_path):
return DefaultSessionStore(session_dir=str(tmp_path))
def _roles(a):
return [m["role"] for m in a.chat_history]
def test_auto_save_round_trips_tool_fields(store):
a = _agent("sess_rt", store)
a.chat_history.extend(_tool_history())
a._auto_save_session()
b = _agent("sess_rt", store)
restored = b.chat_history
assert restored[1]["tool_calls"][0]["id"] == "call_1"
assert restored[2]["tool_call_id"] == "call_1"
assert restored[2]["content"] == "18C sunny"
def test_replace_chat_history_resyncs_auto_save_cursor(store):
a = _agent("sess_cmp", store)
def turn(i):
a._append_to_chat_history({"role": "user", "content": f"q{i}"})
a._append_to_chat_history({"role": "assistant", "content": f"a{i}"})
a._auto_save_session()
for i in range(3):
turn(i)
a._replace_chat_history([
{"role": "system", "content": "summary"},
{"role": "assistant", "content": "a2"},
])
for i in range(3, 6):
turn(i)
saved = [m["content"] for m in store.get_chat_history("sess_cmp")]
assert saved == ["q0", "a0", "q1", "a1", "q2", "a2",
"q3", "a3", "q4", "a4", "q5", "a5"]
def test_replace_chat_history_keeps_unsaved_survivors(store):
a = _agent("sess_unsaved", store)
a._append_to_chat_history({"role": "user", "content": "q0"})
a._auto_save_session()
pending = {"role": "user", "content": "q1"}
a._append_to_chat_history(pending)
# Compaction mid-turn keeps the not-yet-saved message in its tail.
a._replace_chat_history([{"role": "system", "content": "summary"}, pending])
a._auto_save_session()
saved = [m["content"] for m in store.get_chat_history("sess_unsaved")]
assert saved == ["q0", "q1"]
def test_replace_chat_history_saves_appended_messages(store):
# Durable resume appends replayed tool pairs to a fully-saved history.
a = _agent("sess_replay", store)
a._append_to_chat_history({"role": "user", "content": "q0"})
a._auto_save_session()
replay = _tool_history()[1:3]
a._replace_chat_history(list(a.chat_history) + replay)
a._auto_save_session()
saved = store.get_chat_history("sess_replay")
assert [m["role"] for m in saved] == ["user", "assistant", "tool"]
assert saved[2]["tool_call_id"] == "call_1"
def test_prune_history_does_not_orphan_tool_result():
a = _agent()
a.chat_history.extend(_tool_history())
a.prune_history(keep_last=2)
assert _roles(a) == ["assistant"]
def test_delete_history_removes_whole_exchange():
for idx in (1, 2):
a = _agent()
a.chat_history.extend(_tool_history())
assert a.delete_history(idx) is True
assert _roles(a) == ["user", "assistant"]
a = _agent()
a.chat_history.extend(_tool_history())
assert a.delete_history(10) is False
assert a.delete_history(-1) is True
assert len(a.chat_history) == 3
def test_truncate_does_not_leave_dangling_tool_calls():
a = _agent()
a.chat_history.extend(_tool_history())
a._truncate_chat_history(2)
assert _roles(a) == ["user"]
a = _agent()
a.chat_history.extend(_tool_history())
a._truncate_chat_history(3)
assert len(a.chat_history) == 3
def test_delete_history_matching_tolerates_none_content():
a = _agent()
a.chat_history.extend(_tool_history())
assert a.delete_history_matching("image") == 0
# A hit on the tool result drops its whole exchange.
assert a.delete_history_matching("18c sunny") == 2
assert _roles(a) == ["user", "assistant"]
@@ -0,0 +1,113 @@
"""The custom-LLM (``provider/model``) path sends each user turn once.
``chat()`` stores the user turn in ``chat_history`` before the model call, and
``LLM._build_messages`` appends ``prompt`` after ``chat_history`` itself, so
the stored copy must not be forwarded as history too. It was, on every
litellm route (``ollama/``, ``anthropic/``, ``openai/``, ``base_url=``), which
billed the prompt twice. ``litellm.completion`` is stubbed; nothing leaves the
machine.
"""
import asyncio
import pytest
pytest.importorskip("litellm")
from praisonaiagents import Agent # noqa: E402
MODEL = "anthropic/claude-3-5-sonnet-20240620"
def _litellm():
import litellm
return litellm
def _reply(model, text):
return _litellm().ModelResponse(model=model, usage={
"prompt_tokens": 10, "completion_tokens": 1, "total_tokens": 11},
choices=[{"index": 0, "finish_reason": "stop",
"message": {"role": "assistant", "content": text}}])
def _chunks(model, text):
litellm = _litellm()
yield litellm.ModelResponseStream(model=model, choices=[{
"index": 0, "delta": {"role": "assistant", "content": text}}])
yield litellm.ModelResponseStream(model=model, choices=[{
"index": 0, "delta": {}, "finish_reason": "stop"}])
@pytest.fixture
def sent(monkeypatch):
"""Record the ``messages`` of every model call; answer "A1", "A2", ..."""
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test")
monkeypatch.setattr(Agent, "_default_model", None)
monkeypatch.setattr(Agent, "_default_model_checked", False)
requests = []
def fake(*a, **k):
requests.append(k.get("messages"))
text = f"A{len(requests)}"
if k.get("stream"):
return _chunks(k.get("model"), text)
return _reply(k.get("model"), text)
async def afake(*a, **k):
return fake(*a, **k)
litellm = _litellm()
monkeypatch.setattr(litellm, "completion", fake)
monkeypatch.setattr(litellm, "acompletion", afake)
return requests
def _turns(messages):
return [(m["role"], m["content"]) for m in messages if m["role"] != "system"]
def _agent():
return Agent(name="p", instructions="Be brief.", llm=MODEL, output="silent")
def test_chat_sends_the_prompt_once(sent):
agent = _agent()
agent.chat("Q1")
agent.chat("Q2")
assert _turns(sent[0]) == [("user", "Q1")]
assert _turns(sent[1]) == [("user", "Q1"), ("assistant", "A1"), ("user", "Q2")]
def test_achat_sends_the_prompt_once(sent):
agent = _agent()
async def run():
await agent.achat("Q1")
await agent.achat("Q2")
asyncio.run(run())
assert _turns(sent[0]) == [("user", "Q1")]
assert _turns(sent[1]) == [("user", "Q1"), ("assistant", "A1"), ("user", "Q2")]
def test_streamed_turn_sends_the_prompt_once(sent):
agent = _agent()
assert "".join(agent.start("Q1", stream=True)) == "A1"
assert _turns(sent[0]) == [("user", "Q1")]
def test_history_still_records_each_turn_once(sent):
agent = _agent()
agent.chat("Q1")
agent.chat("Q2")
assert [(m["role"], m["content"]) for m in agent.chat_history] == [
("user", "Q1"), ("assistant", "A1"), ("user", "Q2"), ("assistant", "A2")]
def test_repeating_the_last_question_is_still_one_turn(sent):
# A user who asks the same thing twice in a row sends it once per request,
# with the earlier exchange kept as history.
agent = _agent()
agent.chat("Q1")
agent.chat("Q1")
assert _turns(sent[1]) == [("user", "Q1"), ("assistant", "A1"), ("user", "Q1")]
@@ -38,7 +38,7 @@ def test_cache_hit_refreshes_current_default_tools(agent, monkeypatch):
builds = []
monkeypatch.setattr(
agent, '_resolve_harness_base_prompt',
lambda: builds.append('build') or 'stable prefix',
lambda tools=None: builds.append('build') or 'stable prefix',
)
first = agent._build_system_prompt(tools=None)
assert 'sample_tool' in first
@@ -73,7 +73,7 @@ def test_cache_hit_refreshes_session_context(agent, monkeypatch):
def test_cache_hit_retains_prefix_reuse_and_appends_suffix_once(agent, monkeypatch):
"""Reuse the stable prefix while adding the suffix exactly once."""
calls = []
agent._resolve_harness_base_prompt = lambda: calls.append('build') or 'stable prefix'
agent._resolve_harness_base_prompt = lambda tools=None: calls.append('build') or 'stable prefix'
monkeypatch.setenv('PRAISONAI_APPEND_SYSTEM_PROMPT', 'per-turn suffix')
first = agent._build_system_prompt()
second = agent._build_system_prompt()
@@ -0,0 +1,230 @@
"""The system prompt carries only guidance that applies to the agent.
* The model-family file-editing hint ("use edit_file ..." / "use
apply_patch ...") is for agents that can edit files; every GPT or Claude
agent got it, tools or not.
* ``Agent(instructions=...)`` sets both backstory and goal to the
instructions, so the prompt repeated them under "Your Goal:".
"""
import pytest
from praisonaiagents import Agent
from praisonaiagents.tools.edit_tools import apply_patch, edit_file
OPENAI_HINT = "use edit_file to modify existing files precisely"
ANTHROPIC_HINT = "use apply_patch"
def lookup(query: str) -> str:
"""Look something up."""
return query
@pytest.fixture
def make(monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "sk-test-not-used")
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-not-used")
made = []
def _make(**kw):
kw.setdefault("instructions", "Be brief.")
agent = Agent(context=False, **kw)
made.append(agent)
return agent
yield _make
for agent in made:
agent.close()
@pytest.mark.parametrize("tools", [None, [lookup]])
def test_agent_without_edit_tools_gets_no_edit_hint(make, tools):
agent = make(llm="gpt-4o-mini", tools=tools)
assert OPENAI_HINT not in agent._build_system_prompt()
def test_agent_with_edit_file_gets_the_openai_hint(make):
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
assert OPENAI_HINT in agent._build_system_prompt()
def test_agent_with_apply_patch_gets_the_anthropic_hint(make):
agent = make(llm="anthropic/claude-sonnet-4-5", tools=[apply_patch])
assert ANTHROPIC_HINT in agent._build_system_prompt()
def test_edit_tool_named_by_string_counts(make):
agent = make(llm="gpt-4o-mini")
assert OPENAI_HINT in agent._build_system_prompt(tools=["edit_file"])
def test_per_call_tools_decide_the_hint(make):
agent = make(llm="gpt-4o-mini", tools=[edit_file])
assert OPENAI_HINT not in agent._build_system_prompt(tools=[lookup])
assert OPENAI_HINT in agent._build_system_prompt(tools=[edit_file])
def test_cached_prompt_follows_a_change_to_the_agents_tools(make):
agent = make(llm="gpt-4o-mini", tools=[lookup])
assert OPENAI_HINT not in agent._build_system_prompt()
agent.tools = [edit_file]
assert OPENAI_HINT in agent._build_system_prompt()
agent.tools = [lookup]
assert OPENAI_HINT not in agent._build_system_prompt()
def test_explicit_harness_override_is_always_used(make):
agent = make(llm="gpt-4o-mini")
agent.harness_base_prompt = "House rules."
assert agent._build_system_prompt().startswith("House rules.")
def test_instructions_appear_once(make):
agent = make(llm="gpt-4o-mini", instructions="Answer in French.")
prompt = agent._build_system_prompt()
assert prompt.count("Answer in French.") == 1
assert "Your Goal:" not in prompt
def test_a_distinct_goal_is_still_stated(make):
agent = make(llm="gpt-4o-mini", instructions=None, role="Editor",
goal="Tighten the copy", backstory="A veteran sub-editor")
prompt = agent._build_system_prompt()
assert "Your Goal: Tighten the copy" in prompt
assert "A veteran sub-editor" in prompt
assert "Your Role: Editor" in prompt
def test_a_tool_renamed_to_edit_file_counts(make):
from praisonaiagents import tool
@tool(name="edit_file")
def my_editor(filepath: str, old_string: str, new_string: str) -> str:
"""Edit a file."""
return "ok"
agent = make(llm="gpt-4o-mini", tools=[my_editor])
assert OPENAI_HINT in agent._build_system_prompt()
def test_nested_tool_list_offers_the_edit_hint(make):
# _format_tools_for_completion flattens nested lists of tool dicts.
agent = make(llm="gpt-4o-mini", tools=[[EDIT_SCHEMA]])
assert OPENAI_HINT in agent._build_system_prompt()
def test_nested_names_the_formatter_drops_get_no_hint(make):
# A nested string is not sent to the model, so neither is the hint.
agent = make(llm="gpt-4o-mini", tools=[["edit_file"]])
assert OPENAI_HINT not in agent._build_system_prompt()
def test_nested_bare_name_dict_gets_no_hint(make):
# A nested dict without a function schema is dropped by the formatter.
agent = make(llm="gpt-4o-mini", tools=[[{"name": "edit_file"}]])
assert agent._format_tools_for_completion() == []
assert OPENAI_HINT not in agent._build_system_prompt()
EDIT_SCHEMA = {"type": "function", "function": {
"name": "edit_file", "description": "Edit a file.",
"parameters": {"type": "object", "properties": {}}}}
def _system_sent(agent, monkeypatch, run):
"""Run ``run(agent)`` on the native path; return the system prompt and tool names sent."""
from types import SimpleNamespace
sent = []
sent_tools = []
def fake(messages=None, temperature=None, tools=None, *a, **k):
sent.append(messages if messages is not None else k.get("messages"))
# Record the payload the real request builds (inherit, flatten, prune).
formatted = agent._format_tools_for_completion(tools)
sent_tools.append([t.get("function", {}).get("name") for t in formatted])
message = SimpleNamespace(content="ok", tool_calls=None, role="assistant", reasoning_content=None)
return SimpleNamespace(choices=[SimpleNamespace(message=message, finish_reason="stop", index=0)],
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=1, total_tokens=2),
model="gpt-4o-mini")
async def afake(*a, **k):
return fake(*a, **k)
monkeypatch.setattr(agent, "_chat_completion", fake)
monkeypatch.setattr(agent, "_achat_completion_with_retry", afake)
run(agent)
return next(m["content"] for m in sent[-1] if m["role"] == "system"), sent_tools[-1]
def test_native_chat_follows_per_call_edit_tools(make, monkeypatch):
agent = make(llm="gpt-4o-mini")
system, tool_names = _system_sent(agent, monkeypatch, lambda a: a.chat("hi", tools=[EDIT_SCHEMA]))
assert OPENAI_HINT in system
assert "edit_file" in tool_names
def test_nested_edit_schema_is_sent_with_the_hint(make, monkeypatch):
agent = make(llm="gpt-4o-mini", tools=[[EDIT_SCHEMA]])
system, tool_names = _system_sent(agent, monkeypatch, lambda a: a.chat("hi"))
assert OPENAI_HINT in system
assert "edit_file" in tool_names
def test_permission_denied_edit_tool_gets_no_hint(make, monkeypatch):
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
agent._perm_deny = frozenset({"edit_file"})
system, tool_names = _system_sent(agent, monkeypatch, lambda a: a.chat("hi"))
assert "edit_file" not in tool_names
assert OPENAI_HINT not in system
def test_deny_added_after_caching_prunes_cached_tools(make):
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
assert "edit_file" in [t["function"]["name"] for t in agent._format_tools_for_completion()]
agent._perm_deny = frozenset({"edit_file"})
assert "edit_file" not in [t["function"]["name"] for t in agent._format_tools_for_completion()]
assert OPENAI_HINT not in agent._build_system_prompt()
# Lifting the denial restores the tool from the same cache entry.
agent._perm_deny = frozenset()
assert "edit_file" in [t["function"]["name"] for t in agent._format_tools_for_completion()]
def test_deny_before_first_format_restores_after_lift(make):
agent = make(llm="gpt-4o-mini", tools=[edit_file, lookup])
agent._perm_deny = frozenset({"edit_file"})
assert "edit_file" not in [t["function"]["name"] for t in agent._format_tools_for_completion()]
agent._perm_deny = frozenset()
assert "edit_file" in [t["function"]["name"] for t in agent._format_tools_for_completion()]
def test_bridge_tool_search_hides_denied_tools(make):
agent = make(llm="gpt-4o-mini")
agent._tool_search_config = None
agent._tool_search_metadata = {"bridge_mode": True, "deferrable_tools": [EDIT_SCHEMA]}
agent._perm_deny = frozenset({"edit_file"})
out = agent._handle_bridge_tool_call("tool_describe", {"tool_name": "edit_file"})
assert "not found" in out
agent._perm_deny = frozenset()
assert "not found" not in agent._handle_bridge_tool_call("tool_describe", {"tool_name": "edit_file"})
def test_native_achat_follows_per_call_edit_tools(make, monkeypatch):
import asyncio
agent = make(llm="gpt-4o-mini")
system, tool_names = _system_sent(agent, monkeypatch, lambda a: asyncio.run(a.achat("hi", tools=[EDIT_SCHEMA])))
assert OPENAI_HINT in system
assert "edit_file" in tool_names
def test_system_prompt_attribute_states_instructions_once(make):
agent = make(llm="gpt-4o-mini", instructions="Answer in French.")
assert agent.system_prompt.count("Answer in French.") == 1
def test_cached_prompt_follows_a_backstory_change(make):
agent = make(llm="gpt-4o-mini", instructions="Answer in French.")
assert "Your Goal:" not in agent._build_system_prompt()
agent.backstory = "A translator"
prompt = agent._build_system_prompt()
assert "A translator" in prompt
assert "Your Goal: Answer in French." in prompt
@@ -32,6 +32,7 @@ def _agent_with_history(messages: int, words: int):
def fake_stream(**kwargs):
sent["history"] = list(kwargs.get("chat_history") or [])
sent["prompt"] = kwargs.get("prompt")
yield "ok"
agent._using_custom_llm = True
@@ -60,7 +61,10 @@ class TestCustomLlmStreamCompaction:
"""Compaction must not fire when it is not needed."""
agent, sent = _agent_with_history(6, 5)
history = _drain(agent, sent)
assert len(history) == 7, "a short history was compacted unnecessarily"
# The six earlier turns go as history; this turn goes once, as the prompt.
assert len(history) == 6, "a short history was compacted unnecessarily"
assert sent["prompt"] == "hello"
assert all(m["content"] != "hello" for m in history)
def test_the_context_manager_is_actually_configured_here(self):
"""Guards the premise: tools present auto-enable context management."""
@@ -0,0 +1,73 @@
"""Automatic promotion must honor the same capacity as explicit long-term writes."""
import json
import pytest
from praisonaiagents.memory.file_memory import FileMemory
@pytest.mark.parametrize("limit", [1, 2, 3])
def test_promotion_enforces_persisted_long_term_limit(tmp_path, limit):
memory = FileMemory(
base_path=str(tmp_path),
config={"short_term_limit": 2, "long_term_limit": limit},
)
for index in range(6):
memory.add_short_term(f"Important fact {index}", importance=0.8)
assert len(memory.get_long_term()) == limit
assert len(FileMemory(base_path=str(tmp_path)).get_long_term()) == limit
assert len(memory.get_short_term()) == 2
def test_promotion_keeps_highest_importance_and_newest_ties(tmp_path):
memory = FileMemory(
base_path=str(tmp_path),
config={"short_term_limit": 3, "long_term_limit": 2},
)
memory.add_long_term("Keep strongest", importance=1.0)
memory.add_long_term("Older tie", importance=0.8)
memory.add_short_term("Lower priority", importance=0.7)
memory.add_short_term("Newer tie", importance=0.8, metadata={"source": "fixture"})
memory.add_short_term("Below promotion threshold", importance=0.2)
for path, timestamp in [(memory.long_term_file, 100), (memory.short_term_file, 200)]:
records = json.loads(path.read_text())
for record in records:
record["created_at"] = timestamp
path.write_text(json.dumps(records))
promoted_before = next(
item for item in json.loads(memory.short_term_file.read_text())
if item["content"] == "Newer tie"
)
memory.add_short_term("Trigger overflow", importance=0.2)
contents = {item.content for item in memory.get_long_term()}
assert contents == {"Keep strongest", "Newer tie"}
promoted_after = next(item for item in memory.get_long_term() if item.content == "Newer tie")
assert promoted_after.to_dict() == promoted_before
assert {item.content for item in memory.get_short_term()} == {
"Newer tie", "Below promotion threshold", "Trigger overflow",
}
assert {item.content for item in FileMemory(base_path=str(tmp_path)).get_long_term()} == contents
def test_disabled_promotion_preserves_existing_long_term(tmp_path):
memory = FileMemory(
base_path=str(tmp_path),
config={"short_term_limit": 1, "long_term_limit": 1, "auto_promote": False},
)
memory.add_long_term("Existing fact")
for index in range(3):
memory.add_short_term(f"Unpromoted {index}", importance=1.0)
assert [item.content for item in memory.get_long_term()] == ["Existing fact"]
def test_promotion_zero_capacity_matches_explicit_long_term(tmp_path):
memory = FileMemory(
base_path=str(tmp_path),
config={"short_term_limit": 1, "long_term_limit": 0},
)
memory.add_long_term("Explicit fact", importance=1.0)
assert memory.get_long_term() == []
memory.add_short_term("Promoted one", importance=1.0)
memory.add_short_term("Promoted two", importance=1.0)
assert memory.get_long_term() == []
assert FileMemory(base_path=str(tmp_path)).get_long_term() == []
@@ -34,11 +34,11 @@ def _restore_approval_state():
from praisonaiagents.approval import get_approval_registry, clear_approval_context
registry = get_approval_registry()
previous = registry.get_backend()
previous = registry._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
try:
yield
finally:
registry.set_backend(previous)
registry._global_backend = previous
clear_approval_context()
@@ -154,7 +154,7 @@ def test_code_mode_changes_what_the_tool_can_do():
_probe_tool_name = "code_mode_probe"
get_registry().register(code_mode_probe)
previous = get_approval_registry().get_backend()
previous = get_approval_registry()._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
get_approval_registry().set_backend(AutoApproveBackend())
try:
safe = Agent(
@@ -182,7 +182,7 @@ def test_code_mode_changes_what_the_tool_can_do():
safe_result = safe_exec(code)
unsafe_result = unsafe_exec(code)
finally:
get_approval_registry().set_backend(previous)
get_approval_registry()._global_backend = previous
clear_approval_context()
# The tool registry is process-global. Leaving these probe tools in it
# changed what later tests saw when they enumerated or dispatched
@@ -221,7 +221,7 @@ def test_unsafe_code_mode_cannot_reach_tools_the_agent_was_not_granted():
_probe_tool_name = "ungranted_global_tool"
get_registry().register(ungranted_global_tool)
previous = get_approval_registry().get_backend()
previous = get_approval_registry()._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
get_approval_registry().set_backend(AutoApproveBackend())
try:
agent = Agent(
@@ -240,7 +240,7 @@ def test_unsafe_code_mode_cannot_reach_tools_the_agent_was_not_granted():
)
result = execute("print(ungranted_global_tool())")
finally:
get_approval_registry().set_backend(previous)
get_approval_registry()._global_backend = previous
clear_approval_context()
# The tool registry is process-global. Leaving these probe tools in it
# changed what later tests saw when they enumerated or dispatched
@@ -279,13 +279,13 @@ def test_code_execution_runs_a_trailing_expression_exactly_once():
from praisonaiagents.approval import clear_approval_context
registry = get_approval_registry()
previous = registry.get_backend()
previous = registry._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
registry.set_backend(AutoApproveBackend())
try:
assert execute_code("print(7)")["stdout"] == "7\n"
assert execute_code_with_tools("print(7)")["stdout"] == "7\n"
finally:
registry.set_backend(previous)
registry._global_backend = previous
clear_approval_context()
@@ -79,7 +79,7 @@ def test_no_handrolled_async_bridges_outside_helper():
for path in root.rglob("*.py"):
if path in allowed:
continue
tree = ast.parse(path.read_text(), filename=str(path))
tree = ast.parse(path.read_bytes(), filename=str(path))
for func in ast.walk(tree):
if not isinstance(func, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
@@ -93,6 +93,15 @@ def test_no_handrolled_async_bridges_outside_helper():
)
def test_scan_accepts_python_source_encoding_cookie(tmp_path, monkeypatch):
"""Scan the same Latin-1 source that Python's parser accepts."""
(tmp_path / "example.py").write_bytes(
b"# coding: latin-1; caf\xe9\nmessage = 'caf\xe9'\n"
)
monkeypatch.setitem(globals(), "_package_root", lambda: tmp_path)
test_no_handrolled_async_bridges_outside_helper()
def test_tool_timeout_uses_daemon_worker_and_does_not_leak():
before = {t.ident for t in threading.enumerate()}
@@ -0,0 +1,85 @@
"""Counting rules of benchmarks/execution_benchmark.py, with fake runners.
No framework is imported and nothing leaves the machine: each runner is a
``build()`` that returns the call to time, as the script's real runners do.
"""
import importlib.util
from pathlib import Path
import pytest
SCRIPT = Path(__file__).resolve().parents[2] / "benchmarks" / "execution_benchmark.py"
@pytest.fixture(scope="module")
def bench():
spec = importlib.util.spec_from_file_location("execution_benchmark", SCRIPT)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def runner(answers, log, name):
"""A build() whose calls return ``answers`` in order (an Exception is raised)."""
answers = iter(answers)
def build():
log.append(("build", name))
def call():
log.append(("call", name))
answer = next(answers)
if isinstance(answer, Exception):
raise answer
return answer
return call
return build
def test_warm_up_calls_are_not_timed_and_rounds_interleave(bench):
log = []
runners = {"A": runner(["w", "a1", "a2"], log, "A"), "B": runner(["w", "b1", "b2"], log, "B")}
results = bench.run_benchmark(runners, iterations=2, warmup=1)
assert [len(r["times"]) for r in results.values()] == [2, 2]
calls = [name for kind, name in log if kind == "call"]
assert calls == ["A", "B", "A", "B", "B", "A"] # warm-ups, round 1, round 2 rotated
def test_errors_and_empty_answers_are_failures_not_timings(bench):
log = []
runners = {"A": runner(["w", RuntimeError("401"), None, "None", " ", "4"], log, "A")}
results = bench.run_benchmark(runners, iterations=5, warmup=1)
assert len(results["A"]["times"]) == 1
assert len(results["A"]["failures"]) == 4
assert "RuntimeError: 401" in results["A"]["failures"][0]
def test_a_failed_warm_up_does_not_drop_the_framework(bench):
log = []
runners = {"A": runner([RuntimeError("timeout"), "a1", "a2"], log, "A")}
results = bench.run_benchmark(runners, iterations=2, warmup=1)
assert len(results["A"]["times"]) == 2
def test_summary_ranks_by_median_and_skips_frameworks_without_a_success(bench):
results = {
"slow": {"times": [1.0, 1.1, 9.0], "failures": []},
"fast": {"times": [0.5, 0.6, 0.7], "failures": ["x"]},
"broken": {"times": [], "failures": ["401"]},
}
rows = bench.summarise(results)
assert [r[0] for r in rows] == ["fast", "slow"]
name, median, mean, ok, failed = rows[1]
assert (median, ok, failed) == (1.1, 3, 0)
assert mean == pytest.approx(11.1 / 3)
def test_only_the_call_is_timed(bench, monkeypatch):
clock = iter([10.0, 10.5]) # start / end of the one timed call; build() reads no clock
def build():
return lambda: "4"
monkeypatch.setattr(bench.time, "perf_counter", lambda: next(clock))
results = bench.run_benchmark({"A": build}, iterations=1, warmup=0)
assert results["A"]["times"] == [0.5]
@@ -328,7 +328,7 @@ class TestMultiChannelGatewayConfig:
from praisonaiagents.gateway.config import ChannelRouteConfig, MultiChannelGatewayConfig
module = sys.modules["praisonaiagents.gateway.config"]
source_file = module.__file__
with open(source_file, "r") as f:
with open(source_file, "r", encoding="utf-8") as f:
content = f.read()
for dep in ["chromadb", "fastapi", "uvicorn", "pydantic", "litellm"]:
assert f"import {dep}" not in content, f"Found heavy dep '{dep}' in config.py"
@@ -376,7 +376,7 @@ class TestChatCommandInfo:
from praisonaiagents.bots.protocols import BotProtocol
module = sys.modules["praisonaiagents.bots.protocols"]
source_file = module.__file__
with open(source_file, "r") as f:
with open(source_file, "r", encoding="utf-8") as f:
content = f.read()
for dep in ["chromadb", "fastapi", "uvicorn", "litellm"]:
assert f"import {dep}" not in content, f"Found heavy dep '{dep}' in protocols.py"
@@ -0,0 +1,107 @@
"""Regression tests for Issue #6238.
* Caller-supplied names must never escape the storage directory.
* Sync entry points must work from inside a running event loop.
"""
import asyncio
import os
from praisonaiagents.planning.storage import PlanStorage
from praisonaiagents.planning.todo import TodoList
from praisonaiagents.utils.atomic_io import safe_join
def test_safe_join_keeps_legacy_names_and_confines_others(tmp_path):
base = str(tmp_path)
assert safe_join(base, "current", ".json") == os.path.join(base, "current.json")
for name in ("../../x", "/etc/passwd", "..", "", "a/b", "a\\b", ".hidden"):
path = safe_join(base, name, ".json")
assert os.path.dirname(path) == base, name
# Distinct inputs never collide on one file.
assert safe_join(base, "a/b") != safe_join(base, "a_b")
def test_plan_storage_todo_name_cannot_traverse(tmp_path):
storage = PlanStorage(base_path=str(tmp_path / "store"))
path = storage.save_todo(TodoList(), name="../../escaped")
assert os.path.dirname(path) == storage.todos_dir
assert not (tmp_path / "escaped.json").exists()
assert storage.load_todo("../../escaped") is not None
assert storage.delete_todo("../../escaped") is True
def test_generate_title_works_inside_running_loop(monkeypatch):
from praisonaiagents.session import title
async def fake(*args, **kwargs):
return "LLM Title"
monkeypatch.setattr(title, "generate_title_async", fake)
async def main():
return title.generate_title("hello", "world")
assert asyncio.run(main()) == "LLM Title"
def test_listed_todo_names_round_trip(tmp_path):
storage = PlanStorage(base_path=str(tmp_path / "store"))
storage.save_todo(TodoList(), name="my tasks")
(listed,) = storage.list_todos()
assert storage.load_todo(listed) is not None
assert storage.load_todo("my tasks") is not None
assert storage.delete_todo(listed) is True
assert storage.list_todos() == []
def test_legacy_todo_file_still_found(tmp_path):
storage = PlanStorage(base_path=str(tmp_path / "store"))
legacy = os.path.join(storage.todos_dir, "my tasks.json")
with open(legacy, "w", encoding="utf-8") as f:
f.write(TodoList().to_json())
assert storage.load_todo("my tasks") is not None
assert storage.save_todo(TodoList(), name="my tasks") == legacy
assert storage.list_todos() == ["my tasks"]
assert storage.delete_todo("my tasks") is True
def test_custom_plan_filename_keeps_md_suffix(tmp_path):
from praisonaiagents.planning.plan import Plan
storage = PlanStorage(base_path=str(tmp_path / "store"))
plan = Plan(name="demo")
path = storage.save_plan(plan, filename="my plan.md")
assert path.endswith(".md") and os.path.dirname(path) == storage.plans_dir
assert len(storage.list_plans()) == 1
def test_encoded_looking_names_do_not_share_a_file(tmp_path):
storage = PlanStorage(base_path=str(tmp_path / "store"))
names = ["a/b", "a_b~c14cddc033f6", "a%2Fb", "a b", "a%20b", "~x", ".."]
paths = {storage.save_todo(TodoList(name=n), name=n) for n in names}
assert len(paths) == len(names)
assert all(os.path.dirname(p) == storage.todos_dir for p in paths)
assert sorted(storage.list_todos()) == sorted(names)
for n in names:
assert storage.load_todo(n).name == n
assert storage.delete_todo("a/b") is True
assert storage.load_todo("a%2Fb") is not None
def test_long_unicode_names_are_bounded_and_round_trip(tmp_path):
storage = PlanStorage(base_path=str(tmp_path / "store"))
long_a, long_b = "我的任务" * 40, "我的任务" * 40 + "x"
pa = storage.save_todo(TodoList(name=long_a), name=long_a)
pb = storage.save_todo(TodoList(name=long_b), name=long_b)
assert pa != pb
for p in (pa, pb):
assert os.path.dirname(p) == storage.todos_dir
assert len(os.path.basename(p).encode("utf-8")) <= 200
# Short non-ASCII names keep their plain (pre-#6238) filename.
assert storage.save_todo(TodoList(), name="我的任务").endswith("我的任务.json")
for listed in storage.list_todos():
assert storage.load_todo(listed) is not None
assert storage.delete_todo(listed) is True
assert storage.list_todos() == []
assert storage.load_todo(long_a) is None
@@ -64,6 +64,35 @@ def test_grep_glob_filters_by_pattern(sample_tree):
assert "src" not in out
@pytest.mark.parametrize("pattern", ["**/*.py", "src/**/*.py", "src/**/**/test_*.py"])
def test_grep_recursive_glob_matches_zero_directories(sample_tree, pattern):
from praisonaiagents.tools import grep
direct = sample_tree / "src" / "test_direct.py"
direct.write_text("needle\n")
nested = sample_tree / "src" / "nested"
nested.mkdir()
(nested / "test_nested.py").write_text("needle\n")
with patch("shutil.which", return_value=None):
out = grep("needle", glob=pattern)
assert "src/test_direct.py:1:" in out.replace(os.sep, "/")
assert "src/nested/test_nested.py:1:" in out.replace(os.sep, "/")
if pattern == "**/*.py":
assert "a.py:2:" in out
else:
assert "a.py:2:" not in out
if "test_" in pattern:
assert "src/d.py:1:" not in out.replace(os.sep, "/")
def test_grep_recursive_glob_single_file(sample_tree):
from praisonaiagents.tools import grep
with patch("shutil.which", return_value=None):
out = grep("needle", path="a.py", glob="**/*.py")
assert "a.py:2:" in out
def test_grep_case_insensitive(sample_tree):
from praisonaiagents.tools import grep
with patch("shutil.which", return_value=None):
@@ -491,7 +491,7 @@ class TestTeamTasks:
return ApprovalDecision(approved=True, reason="looks good", approver="ana")
registry = get_approval_registry()
previous = registry.get_backend()
previous = registry._global_backend # not get_backend(): its ConsoleBackend fallback would be installed globally on restore
registry.set_backend(Reviewer())
try:
writer = Agent(name="Writer", instructions="write")
@@ -500,7 +500,7 @@ class TestTeamTasks:
out = TaskOutput(description="Draft", raw="api_key=sk-secret123456 text", agent="Writer")
team._apply_human_review(task, 0, out)
finally:
registry.set_backend(previous)
registry._global_backend = previous
requested, decided = _of(recorder, "approval_requested"), _of(recorder, "approval_decided")
assert len(requested) == len(decided) == 1
assert requested[0].data["kind"] == "task_output_review"
@@ -0,0 +1,57 @@
"""An agent's YAML `execution:` block reaches ExecutionConfig (Issue #6206)."""
from praisonaiagents.workflows.yaml_parser import YAMLWorkflowParser
def test_execution_block_passes_budget_and_durable_keys():
agent = YAMLWorkflowParser()._create_agent("worker", {
"role": "Worker",
"execution": {
"max_budget": 1.5,
"on_budget_exceeded": "warn",
"durable": True,
"resume_run_id": "run-123",
},
"max_iter": 7,
})
assert agent._max_budget == 1.5
assert agent._on_budget_exceeded == "warn"
assert agent.execution.durable is True
assert agent.execution.resume_run_id == "run-123"
assert agent.execution.max_iter == 7
def test_flat_max_iter_overrides_execution_block():
agent = YAMLWorkflowParser()._create_agent("worker", {
"execution": {"max_iter": 3},
"max_iter": 9,
})
assert agent.execution.max_iter == 9
def test_no_execution_block_keeps_defaults():
agent = YAMLWorkflowParser()._create_agent("worker", {"role": "Worker"})
assert not agent._max_budget
def test_roles_format_keeps_execution_block():
"""The agents.yaml `roles:` format must not drop `execution:` in conversion."""
parser = YAMLWorkflowParser()
parser.parse_string(
"framework: praisonai\n"
"topic: t\n"
"roles:\n"
" worker:\n"
" role: Worker\n"
" goal: g\n"
" backstory: b\n"
" execution:\n"
" max_budget: 2.0\n"
" on_budget_exceeded: warn\n"
" tasks:\n"
" t1:\n"
" description: do it\n"
" expected_output: done\n"
)
agent = parser._agents["worker"]
assert agent._max_budget == 2.0
assert agent._on_budget_exceeded == "warn"
@@ -28,7 +28,9 @@ from __future__ import annotations
import asyncio
import itertools
import logging
import math
import sys
import time
from contextlib import asynccontextmanager
from typing import Dict, Optional
@@ -106,9 +108,14 @@ class AdmissionRejected(Exception):
busy acknowledgement to the user instead of an opaque error.
"""
def __init__(self, message: str = "Busy right now — please resend in a moment."):
def __init__(
self,
message: str = "Busy right now — please resend in a moment.",
retry_after: Optional[float] = None,
):
super().__init__(message)
self.message = message
self.retry_after = retry_after
class _TurnSlot:
@@ -249,8 +256,16 @@ class AdmissionGate:
*,
resource_policy: Optional[object] = None,
resource_sampler: Optional[object] = None,
rate_limit_policy: Optional[object] = None,
):
self._policy = policy
# Issue #6215: optional per-identity request-rate ceiling (a core
# ``RateLimitPolicyProtocol``, e.g. ``SlidingWindowRateLimitPolicy``)
# checked before a turn is admitted, keyed by the caller's rate key
# (defaults to the session id).
self._rate_limit_policy = rate_limit_policy
self.rate_limited = 0
self._rate_prune_after = 0.0
# Optional memory/resource-pressure policy (Issue #3445). When set, a
# lightweight RSS sample is folded into each admit decision so the gate
# queues under soft pressure and sheds under hard pressure before the
@@ -303,6 +318,16 @@ class AdmissionGate:
# otherwise assume active (it was explicitly supplied).
return bool(getattr(pol, "enabled", True))
def set_rate_limit_policy(self, policy: Optional[object]) -> None:
"""Swap the per-identity rate budget in place (config hot-reload)."""
self._rate_limit_policy = policy
self._rate_prune_after = 0.0
@property
def _rate_limit_enabled(self) -> bool:
pol = self._rate_limit_policy
return pol is not None and bool(getattr(pol, "enabled", True))
@property
def enabled(self) -> bool:
"""Whether the gate is active.
@@ -313,7 +338,7 @@ class AdmissionGate:
"""
if self._policy is not None and self._max > 0:
return True
return self._resource_enabled
return self._resource_enabled or self._rate_limit_enabled
@property
def in_flight(self) -> int:
@@ -337,6 +362,10 @@ class AdmissionGate:
"admitted": self.admitted,
"rejected": self.rejected,
"shed": self.shed,
"per_user_per_minute": int(
getattr(self._rate_limit_policy, "max_requests", 0) or 0
),
"rate_limited": self.rate_limited,
"max_rss_mb": float(
getattr(self._resource_policy, "hard_rss_mb", 0.0) or 0.0
),
@@ -362,9 +391,14 @@ class AdmissionGate:
return self._sem
@asynccontextmanager
async def admit(self, *, session_id: str = ""):
async def admit(self, *, session_id: str = "", rate_key: Optional[str] = None):
"""Admit (or shed) an inbound turn, holding a slot for its duration.
``rate_key`` is the identity charged against the per-user request
budget. It defaults to ``session_id``; surfaces where the session id is
client-chosen (API headers, WebSocket sessions) pass a stable caller
identity instead so a new conversation cannot reset the budget.
Yields when capacity is available (immediately or after waiting in the
bounded queue). Raises :class:`AdmissionRejected` when the policy sheds
the turn. A no-op pass-through when the gate is disabled.
@@ -374,6 +408,8 @@ class AdmissionGate:
yield
return
self._check_rate_limit(rate_key or session_id)
# Issue #5168: the scope key is the caller-supplied session_id (the
# per-user/per-tenant storage key at the call sites). Only tracked when
# a per-scope sub-limit is active, so the unscoped path is unchanged.
@@ -473,6 +509,54 @@ class AdmissionGate:
pass
slot.close()
def _check_rate_limit(self, session_id: str) -> None:
"""Shed the turn if *session_id* is over its request-rate budget (#6215)."""
if not self._rate_limit_enabled:
return
pol = self._rate_limit_policy
now = time.monotonic()
try:
decision = pol.check(identity=session_id or "-", scope="ingress", now=now)
except Exception as e: # pragma: no cover — never block on policy error
logger.warning("AdmissionGate: rate-limit policy failed (%s); admitting", e)
return
finally:
# Bounded cadence: a burst of distinct callers must not trigger a
# full scan on every request while their windows are still live.
if now >= self._rate_prune_after:
self._rate_prune_after = now + _RATE_PRUNE_INTERVAL
self._prune_rate_limit_state(pol, now)
if decision.allowed:
return
self.rate_limited += 1
retry = decision.retry_after_seconds
wait = max(1, math.ceil(retry)) if retry is not None else None
logger.info("AdmissionGate: rate-limited session=%s retry_after=%s", session_id or "-", wait)
if wait is None:
raise AdmissionRejected("Rate limit reached — please try again later.")
raise AdmissionRejected(
f"Rate limit reached — please try again in {wait}s.", retry_after=float(wait)
)
@staticmethod
def _prune_rate_limit_state(pol: object, now: float, cap: int = 4096) -> None:
"""Drop expired per-identity entries so an open bot cannot grow them forever.
``SlidingWindowRateLimitPolicy`` reclaims a key only when that key is
checked again and leaves reclamation of an unbounded identity space to
the wrapper; inbound users are such a space.
"""
windows = getattr(pol, "_windows", None)
lockouts = getattr(pol, "_lockouts", None)
if not isinstance(windows, dict) or len(windows) + len(lockouts or {}) <= cap:
return
span = float(getattr(pol, "window_seconds", 0) or 0)
for key in [k for k, (start, _) in windows.items() if now - start >= span]:
windows.pop(key, None)
if isinstance(lockouts, dict):
for key in [k for k, until in lockouts.items() if now >= until]:
lockouts.pop(key, None)
def _take_slot(self, scope: str) -> None:
self._in_flight += 1
if scope:
@@ -596,6 +680,46 @@ def build_memory_pressure_policy(
return MemoryPressurePolicy(soft_rss_mb=soft, hard_rss_mb=hard)
_RATE_PRUNE_INTERVAL = 5.0
def build_rate_limit_policy(per_user_per_minute: object) -> Optional[object]:
"""Return a per-user request budget policy, or ``None`` when unlimited.
Accepts a non-negative whole number (``0``/``None`` = unlimited). Fractional,
boolean, negative or non-numeric values raise ``ValueError`` so a typo never
silently disables the budget the operator asked for.
"""
value = per_user_per_minute
if value is None:
return None
if isinstance(value, bool) or not isinstance(value, (int, float, str)):
raise ValueError(
f"per_user_per_minute must be a non-negative integer, got {value!r}"
)
try:
if isinstance(value, float):
if not value.is_integer():
raise ValueError
value = int(value)
else:
value = int(value)
except (TypeError, ValueError) as err:
raise ValueError(
f"per_user_per_minute must be a non-negative integer, "
f"got {per_user_per_minute!r}"
) from err
if value < 0:
raise ValueError(
f"per_user_per_minute must be >= 0, got {per_user_per_minute!r}"
)
if value == 0:
return None
from praisonaiagents.gateway import SlidingWindowRateLimitPolicy
return SlidingWindowRateLimitPolicy(max_requests=value, window_seconds=60.0)
def build_admission_gate(
max_concurrent_runs: int = 0,
queue_depth: int = 0,
@@ -603,6 +727,7 @@ def build_admission_gate(
policy: Optional[object] = None,
resource_policy: Optional[object] = None,
max_concurrent_runs_per_scope: int = 0,
per_user_per_minute: int = 0,
) -> Optional[AdmissionGate]:
"""Construct an :class:`AdmissionGate` from config, or return ``None``.
@@ -620,7 +745,13 @@ def build_admission_gate(
gate so it queues under soft RSS pressure and sheds under hard pressure. A
gate is built when *either* a concurrency ceiling *or* a resource policy is
configured, so a memory-only deployment still gets backpressure.
``per_user_per_minute`` (Issue #6215) caps how many turns one identity may
start per minute; ``0`` (the default) leaves ingress unlimited. Over the cap
the turn is refused with a ``retry_after`` hint before any work starts.
"""
# Raises ValueError on a negative / non-integer value (fail fast).
rate_limit_policy = build_rate_limit_policy(per_user_per_minute)
if policy is None:
# Only an explicit ``0`` (or ``None``) disables admission control. A
# *negative* ceiling is a misconfiguration, not "disabled": fall through
@@ -634,8 +765,12 @@ def build_admission_gate(
if no_ceiling:
# No concurrency ceiling: build a resource-only gate when a
# resource policy is supplied, else no gate (legacy behaviour).
if resource_policy is not None:
return AdmissionGate(None, resource_policy=resource_policy)
if resource_policy is not None or rate_limit_policy is not None:
return AdmissionGate(
None,
resource_policy=resource_policy,
rate_limit_policy=rate_limit_policy,
)
return None
try:
from praisonaiagents.gateway import ConcurrencyLimitPolicy
@@ -647,4 +782,6 @@ def build_admission_gate(
overflow_policy=overflow_policy,
max_concurrent_runs_per_scope=max_concurrent_runs_per_scope,
)
return AdmissionGate(policy, resource_policy=resource_policy)
return AdmissionGate(
policy, resource_policy=resource_policy, rate_limit_policy=rate_limit_policy
)
@@ -912,6 +912,8 @@ class GatewayServerSchema(BaseModel):
max_concurrent_runs: Optional[int] = Field(None, ge=0)
queue_depth: Optional[int] = Field(None, ge=0)
overflow_policy: Optional[str] = None
# Per-user inbound request budget (#6215). 0 = unlimited (default).
per_user_per_minute: Optional[int] = Field(None, ge=0)
preauth_max_connections_per_ip: Optional[int] = Field(None, ge=0)
max_unauthorized_frames: Optional[int] = Field(None, ge=0)
# Graceful-drain windows (#2375 / #2533)
@@ -106,6 +106,49 @@ def _coerce_drain_timeout(value: Optional[Any]) -> Optional[float]:
return timeout
# Issue #6210: keys ``BotOS.from_config`` actually reads. Anything else in a
# BotOS YAML used to be dropped silently.
_BOTOS_TOP_KEYS = frozenset({
"name", "agent", "platforms", "health", "supervision", "gateway",
"drain_timeout", "max_concurrent_runs", "queue_depth", "overflow_policy",
"max_concurrent_runs_per_scope", "max_rss_mb", "reliability",
"per_user_per_minute",
})
_BOTOS_GATEWAY_KEYS = frozenset({
"drain_timeout", "max_concurrent_runs", "queue_depth", "overflow_policy",
"max_concurrent_runs_per_scope", "max_rss_mb", "reliability",
"per_user_per_minute",
})
# Sections that only the gateway runtime (``praisonai gateway start``) honours.
_GATEWAY_ONLY_KEYS = frozenset({
"channels", "agents", "routing", "schedules", "rooms", "hooks",
"access_groups", "mcp_servers", "identity",
})
def _check_botos_config_shape(raw: Dict[str, Any], path: str) -> None:
"""Fail loudly on gateway-only YAML and warn on keys BotOS will not read."""
gateway_only = sorted(k for k in raw if k in _GATEWAY_ONLY_KEYS)
if gateway_only:
raise ValueError(
f"{path} is a gateway config (sections: {', '.join(gateway_only)}) "
"which BotOS.from_config() cannot honour. Run it the same way as "
"`praisonai gateway start --config`: "
"asyncio.run(WebSocketGateway().start_with_config(path)) "
"(from praisonai_bot import WebSocketGateway)."
)
ignored = [str(k) for k in raw if k not in _BOTOS_TOP_KEYS]
gateway_cfg = raw.get("gateway")
if isinstance(gateway_cfg, dict):
ignored += [f"gateway.{k}" for k in gateway_cfg if k not in _BOTOS_GATEWAY_KEYS]
if ignored:
logger.warning(
"BotOS.from_config(%s): ignoring unsupported keys %s "
"(use WebSocketGateway().start_with_config() for full gateway configs)",
path, ", ".join(sorted(ignored)),
)
class BotOS:
"""Multi-platform bot orchestrator.
@@ -150,6 +193,7 @@ class BotOS:
admission_policy: Optional[Any] = None,
max_rss_mb: float = 0.0,
reliability: Optional[str] = None,
per_user_per_minute: int = 0,
turn_lock: Optional[Any] = None,
redis_config: Optional[Any] = None,
degraded_registry: Optional[Any] = None,
@@ -224,6 +268,8 @@ class BotOS:
max_concurrent_runs_per_scope=max_concurrent_runs_per_scope,
policy=admission_policy,
resource_policy=build_memory_pressure_policy(max_rss_mb),
# Issue #6215: per-user inbound request budget (0 = unlimited).
per_user_per_minute=per_user_per_minute,
)
self._on_quiesce = None # optional callable(): host-suspend driver
# W1: shared identity resolver applied to every managed bot —
@@ -1600,11 +1646,22 @@ class BotOS:
startup_grace: 60
max_restarts_per_hour: 10
This is the lightweight BotOS dialect. A full ``gateway.yaml`` (with
``channels:``/``agents:``/``routing:``/``schedules:`` …) is *not*
accepted here — it raises ``ValueError`` instead of being silently
degraded. Run such a file exactly like ``praisonai gateway start``::
from praisonai_bot import WebSocketGateway
asyncio.run(WebSocketGateway().start_with_config("gateway.yaml"))
Args:
path: Path to YAML config file.
Returns:
Configured BotOS instance.
Raises:
ValueError: If the file uses gateway-only sections.
"""
import os
import re
@@ -1620,6 +1677,8 @@ class BotOS:
if not raw or not isinstance(raw, dict):
raise ValueError(f"Invalid BotOS config: {path}")
_check_botos_config_shape(raw, path)
def _resolve_env(val):
if isinstance(val, str):
return re.sub(
@@ -1788,6 +1847,11 @@ class BotOS:
# ``reliability`` or ``gateway.reliability``; the preset composes drain
# + admission while any explicit field above still overrides it.
reliability = raw.get("reliability", gateway_cfg.get("reliability"))
# Issue #6215: per-user inbound request budget. Default 0 = unlimited.
# Validated (not truncated) by ``build_rate_limit_policy``.
per_user_per_minute = raw.get(
"per_user_per_minute", gateway_cfg.get("per_user_per_minute", 0)
)
return cls(
bots=bots,
@@ -1800,6 +1864,7 @@ class BotOS:
max_concurrent_runs_per_scope=max_concurrent_runs_per_scope,
max_rss_mb=max_rss_mb,
reliability=reliability,
per_user_per_minute=per_user_per_minute,
)
async def probe_all(self, timeout: float = 15.0) -> Dict[str, Any]:
@@ -1739,6 +1739,19 @@
"default": null,
"title": "Overflow Policy"
},
"per_user_per_minute": {
"anyOf": [
{
"minimum": 0,
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"title": "Per User Per Minute"
},
"preauth_max_connections_per_ip": {
"anyOf": [
{
@@ -16,6 +16,8 @@ from __future__ import annotations
import asyncio
import contextvars
import hashlib
import hmac
import json
import time
import uuid
@@ -373,7 +375,13 @@ class GatewayApiEndpoints:
if gate is not None and getattr(gate, "enabled", False):
from ..bots._admission import AdmissionRejected
try:
async with gate.admit(session_id=session.session_id):
# Issue #6215: charge a stable caller identity, only when a
# budget is set (keeps duck-typed gates' signature intact).
rk = (
{"rate_key": self._rate_key_var.get() or "ip:unknown"}
if getattr(gate, "_rate_limit_enabled", False) else {}
)
async with gate.admit(session_id=session.session_id, **rk):
result = await self._gw._dispatch_agent_turn(
agent, content, on_complete=_capture, **extra
)
@@ -554,6 +562,41 @@ class GatewayApiEndpoints:
session_id = f"api:{key}:{agent_id}"
return self._gw.create_session(agent_id=agent_id, session_id=session_id)
# Per-request stable identity for the per-user request budget (#6215);
# inherited by the turn's task/stream context like ``_stream_turn_var``.
_rate_key_var: "contextvars.ContextVar[Optional[str]]" = (
contextvars.ContextVar("praisonai_gw_rate_key", default=None)
)
@staticmethod
def _rate_key(request: Any, auth_token: Optional[str] = None) -> str:
"""Stable caller identity that a client-chosen value cannot reset.
A bearer token is used only when it *is* the gateway's configured
``auth_token`` (so it was validated); any other bearer value is
attacker-chosen and would mint a fresh budget per request, so such
callers are charged to their peer address. Never returns ``None``, so the
budget never falls back to the client-chosen session id.
"""
auth = request.headers.get("authorization", "")
bearer = auth[7:] if auth.startswith("Bearer ") else ""
if bearer and auth_token and hmac.compare_digest(
bearer.encode("utf-8"), str(auth_token).encode("utf-8")
):
return "token:" + hashlib.sha256(bearer.encode("utf-8")).hexdigest()[:16]
client = getattr(request, "client", None)
host = getattr(client, "host", None)
return f"ip:{host}" if host else "ip:unknown"
def _bind_rate_key(self, request: Any) -> None:
"""Bind this request's per-user budget key for its turn (#6215).
Called by the auth-guarded route wrapper after auth has passed.
"""
cfg = getattr(self._gw, "config", None)
token = getattr(cfg, "auth_token", None)
self._rate_key_var.set(self._rate_key(request, token))
@staticmethod
def _caller_key(request: Any) -> str:
"""Derive a stable per-caller session key from the request.
@@ -9,6 +9,7 @@ from __future__ import annotations
import os
import socket
import time
from contextlib import contextmanager
from pathlib import Path
from typing import Optional, Tuple
@@ -86,19 +87,21 @@ def _process_create_time(pid: int) -> Optional[float]:
class GatewayPIDLock:
"""Manages PID lock file for single-instance gateway enforcement."""
def __init__(self, lock_dir: Optional[Path] = None, host: str = "127.0.0.1", port: int = 8765):
def __init__(self, lock_dir: Optional[Path] = None, host: str = "127.0.0.1", port: int = 8765,
lock_name: Optional[str] = None):
"""Initialize PID lock manager.
Args:
lock_dir: Directory for lock file (default: ~/.praisonai/)
host: Gateway host
port: Gateway port
lock_name: Override the lock file name (used by the state lock)
"""
self.lock_dir = lock_dir or Path.home() / ".praisonai"
# Make lock file specific to host and port to allow multiple instances
safe_host = host.replace(":", "_").replace(".", "_")
self.lock_file = self.lock_dir / f"gateway-{safe_host}-{port}.pid"
self.lock_dir.mkdir(exist_ok=True)
self.lock_file = self.lock_dir / (lock_name or f"gateway-{safe_host}-{port}.pid")
self.lock_dir.mkdir(parents=True, exist_ok=True)
def acquire_lock(self, host: str, port: int) -> bool:
"""Acquire the PID lock for the gateway.
@@ -110,6 +113,27 @@ class GatewayPIDLock:
Returns:
True if lock acquired successfully, False if conflict
"""
# Serialise check-then-claim across processes so two simultaneous
# starts can never both observe "no live holder" and both win.
with self._claim_guard():
return self._acquire_locked(host, port)
@contextmanager
def _claim_guard(self):
"""Hold an OS advisory lock on a sidecar file for the claim (POSIX)."""
try:
import fcntl
except ImportError: # Windows: exclusive link() below still applies
yield
return
guard = open(self.lock_file.with_name(self.lock_file.name + ".guard"), "a")
try:
fcntl.flock(guard, fcntl.LOCK_EX)
yield
finally:
guard.close() # closing drops the flock
def _acquire_locked(self, host: str, port: int) -> bool:
current_pid = os.getpid()
# Check if lock file exists
@@ -120,6 +144,9 @@ class GatewayPIDLock:
lines = lock_content.split('\n')
if lines:
existing_pid = int(lines[0])
if existing_pid == current_pid:
# Re-acquire by the same process (in-process restart).
return True
existing_create_time = None
if len(lines) >= 5 and lines[4]:
try:
@@ -145,14 +172,29 @@ class GatewayPIDLock:
create_time = _process_create_time(current_pid)
create_time_line = "" if create_time is None else repr(create_time)
lock_content = f"{current_pid}\n{host}\n{port}\n{int(time.time())}\n{create_time_line}\n"
# Exclusive claim: write a private temp file, then hard-link it into
# place. ``link`` fails if the lock already exists, so a racing claimant
# can never overwrite a live holder (unlike rename/replace).
temp_lock_file = self.lock_file.with_name(f"{self.lock_file.name}.{current_pid}.tmp")
try:
# Use a temporary file and atomic rename to prevent race conditions
temp_lock_file = self.lock_file.with_suffix(".tmp")
temp_lock_file.write_text(lock_content)
temp_lock_file.replace(self.lock_file)
try:
os.link(temp_lock_file, self.lock_file)
except FileExistsError:
return False
except OSError:
# Filesystem without hard links: fall back to O_EXCL create.
fd = os.open(self.lock_file, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644)
with os.fdopen(fd, "w") as f:
f.write(lock_content)
return True
except OSError:
return False
finally:
try:
temp_lock_file.unlink()
except OSError:
pass
def release_lock(self) -> None:
"""Release the PID lock."""
@@ -253,6 +295,43 @@ class GatewayPIDLock:
pass
def resolve_state_dir() -> Path:
"""Return the gateway's shared mutable state root (``~/.praisonai``).
The durable outbox, DLQ, pairing and hook stores are hard-wired under
``~/.praisonai`` (they do not honour ``PRAISONAI_HOME``), so the home root -
not the TCP port or ``PRAISONAI_HOME`` - is what identifies a gateway.
"""
return Path.home() / ".praisonai"
def state_lock(state_dir: Optional[Path] = None, host: str = "127.0.0.1",
port: int = 8765) -> GatewayPIDLock:
"""Single-instance lock keyed on the shared state directory, regardless of port.
Two gateways sharing one state root would poll the same bot tokens and write
the same SQLite stores concurrently (split-brain), so only one may own it.
"""
return GatewayPIDLock(lock_dir=state_dir or resolve_state_dir(), host=host,
port=port, lock_name="gateway.state.pid")
def format_state_collision_error(state_dir: Path, lock_info: Optional[dict] = None) -> str:
"""Error shown when another live gateway already owns the shared state."""
holder = ""
if lock_info:
holder = f" (PID {lock_info['pid']} on {lock_info['host']}:{lock_info['port']})"
return "\n".join([
f"Error: another gateway already owns the state at {state_dir}{holder}.",
"",
" Two gateways sharing state poll the same bot tokens and write the same",
" queues, causing 409 Conflict errors and duplicate delivery.",
" Stop it: praisonai gateway stop",
" Or run an isolated gateway with its own state and bot tokens:",
" HOME=/srv/gateway-b praisonai gateway start",
])
def format_collision_error(host: str, port: int, lock_info: Optional[dict] = None) -> str:
"""Format a user-friendly error message for port collisions.
@@ -275,10 +354,11 @@ def format_collision_error(host: str, port: int, lock_info: Optional[dict] = Non
pid = lock_info['pid']
lines.extend([
f" Another gateway may be running (PID {pid}).",
f" Stop it: praisonai gateway stop",
f" Or use a different port: GATEWAY_PORT={port + 1} praisonai gateway start",
" Stop it: praisonai gateway stop",
"",
" Only ONE gateway process should poll each Telegram bot token.",
" Only ONE gateway process may poll each bot token. To run a second,",
" isolated gateway give it its own state, bot tokens and port:",
f" HOME=/srv/gateway-b GATEWAY_PORT={port + 1} praisonai gateway start",
])
else:
lines.extend([
+110 -16
View File
@@ -1412,6 +1412,7 @@ class WebSocketGateway:
# PID lock for single-instance enforcement
self._pid_lock: Optional[Any] = None
self._state_lock: Optional[Any] = None
# Channel supervisor for resilient bot management. Share the gateway's
# degraded-capability registry so the fleet crash-loop breaker (Issue
@@ -1544,9 +1545,46 @@ class WebSocketGateway:
logger.error("Gateway startup failed - another instance is running")
raise RuntimeError(error_msg)
# Store PID lock for cleanup
self._pid_lock = pid_lock
# When ``start_with_config`` already claimed the state root, its channel
# bots are still polling if we fail here; only it may release that lock
# (after channels stop), or a second gateway could claim the state.
owns_state_lock = self._state_lock is None
try:
self._acquire_state_lock()
await self._start_locked()
except BaseException:
# Any failure after claiming (bad config, bind error, cancel) must
# not leave a live process holding the locks and blocking restarts.
self._release_locks(state=owns_state_lock)
raise
def _acquire_state_lock(self) -> None:
"""Claim the shared state root (idempotent).
The port lock alone admits a second gateway on another port sharing the
same state + bot tokens (split-brain), so the state root is locked too.
"""
if self._state_lock is not None:
return
from .port_utils import state_lock, resolve_state_dir, format_state_collision_error
state_dir = resolve_state_dir()
st_lock = state_lock(state_dir, host=self._host, port=self._port)
if not st_lock.acquire_lock(self._host, self._port):
logger.error("Gateway startup failed - another instance owns the shared state")
raise RuntimeError(format_state_collision_error(state_dir, st_lock.get_lock_info()))
self._state_lock = st_lock
def _release_locks(self, state: bool = True) -> None:
"""Release the port lock and (if ``state``) the state lock (idempotent)."""
for attr in ("_pid_lock", "_state_lock") if state else ("_pid_lock",):
lock = getattr(self, attr, None)
if lock:
lock.release_lock()
setattr(self, attr, None)
async def _start_locked(self) -> None:
"""Body of :meth:`start` once the single-instance locks are held."""
# Validate bind-aware auth configuration before starting
from .auth import assert_external_bind_safe
self.config.host = self._host
@@ -2631,6 +2669,8 @@ class WebSocketGateway:
auth_err = _check_auth(request)
if auth_err:
return auth_err
# Issue #6215: bind the per-user budget key post-auth.
self._api_endpoints._bind_rate_key(request)
return await handler(request)
return _wrapped
@@ -2768,14 +2808,8 @@ class WebSocketGateway:
self._ledger_heartbeat_loop(), name="gateway-lifecycle-ledger"
)
try:
# Lock cleanup on failure is handled by ``start()``.
await self._server.serve()
except Exception as e:
# Clean up PID lock on any startup failure
if hasattr(self, '_pid_lock') and self._pid_lock:
self._pid_lock.release_lock()
self._pid_lock = None
# Re-raise the original exception
raise
finally:
self._disarm_watchdog()
await self._cancel_liveness_task()
@@ -3993,9 +4027,7 @@ class WebSocketGateway:
logger.debug("Failed to shut down pooled MCP client")
self._mcp_pool.clear()
# Release PID lock
if hasattr(self, '_pid_lock') and self._pid_lock:
self._pid_lock.release_lock()
self._release_locks()
logger.info("Gateway stopped")
@@ -4614,7 +4646,7 @@ class WebSocketGateway:
elif msg_type == "server_reply":
# Issue #5351: a client's correlated answer to a server→client
# interactive request (approval / choice / input).
# interactive request (approval / choice / input / secret).
request_id = data.get("request_id")
value = data.get("value")
if not request_id or not isinstance(value, str):
@@ -4648,7 +4680,7 @@ class WebSocketGateway:
return True
# Answering an approval on behalf of the turn is an approval decision
# and requires the APPROVALS scope (parity with approvals.resolve);
# choice/input answers require WRITE like sending a message.
# choice/input/secret answers require WRITE like sending a message.
required_scope = (
OperatorScope.APPROVALS if req_kind == "approval" else OperatorScope.WRITE
)
@@ -5089,7 +5121,14 @@ class WebSocketGateway:
# so enforce the shared gate here too.
from ..bots._admission import AdmissionRejected
try:
async with gate.admit(session_id=session.session_id):
# Issue #6215: stable per-user budget key (only
# when a budget is set; duck-typed gates unchanged).
_rk = (
{"rate_key": self._client_rate_key(client_id)}
if getattr(gate, "_rate_limit_enabled", False)
else {}
)
async with gate.admit(session_id=session.session_id, **_rk):
if getattr(session, "_stop_epoch", 0) != dequeued_stop_epoch:
response = _stop_sentinel
else:
@@ -5812,7 +5851,9 @@ class WebSocketGateway:
* ``approval`` → ``"allow"`` or ``"deny"``.
* ``choice`` → one of the offered ``options`` (any value if none given).
* ``input`` → any string.
* ``input`` / ``secret`` → any string. A ``secret`` value is handed
only to the waiting caller — never logged, echoed, or journalled
(Issue #6223).
"""
kind = getattr(req, "kind", "input")
if kind == "approval":
@@ -5894,6 +5935,18 @@ class WebSocketGateway:
# ── Client Management (Public API) ────────────────────────────────
def _client_rate_key(self, client_id: Optional[str]) -> str:
"""Stable per-user budget key for a WebSocket turn (#6215).
Session ids are client-chosen, so charge the peer address instead.
With no known peer, callers share one ``ip:unknown`` budget rather
than falling back to the client-chosen session id.
"""
ws = self._clients.get(client_id) if client_id else None
peer = getattr(ws, "client", None)
host = getattr(peer, "host", None)
return f"ip:{host}" if host else "ip:unknown"
def add_client(self, client_id: str, websocket: Any) -> None:
"""Register a client WebSocket connection.
@@ -11281,6 +11334,29 @@ class WebSocketGateway:
self._shell_routed_agents.clear()
self._shell_routed_agents.update(candidate.get("shell", {}))
@staticmethod
def _rate_policy_from_config(cfg: Any) -> Any:
"""Build the ``gateway.per_user_per_minute`` policy (#6215); may raise."""
from ..bots._admission import build_rate_limit_policy
gw_cfg = (cfg or {}).get("gateway") or {}
return build_rate_limit_policy(gw_cfg.get("per_user_per_minute", 0))
def _apply_rate_policy(self, policy: Any) -> None:
"""Install a reloaded per-user budget on the live admission gate (#6215).
An existing gate is updated in place (bots/API/WS already share it); a
budget added to a gateway that had no gate builds a rate-only gate,
which channels started afterwards pick up via ``_stamp_admission_gate``.
"""
gate = getattr(self, "_admission_gate", None)
if gate is not None:
gate.set_rate_limit_policy(policy)
elif policy is not None:
from ..bots._admission import AdmissionGate
self._admission_gate = AdmissionGate(None, rate_limit_policy=policy)
async def _restore_previous_config(
self, error: str, changed_paths: Set[str]
) -> None:
@@ -11302,6 +11378,10 @@ class WebSocketGateway:
)
return
try:
try:
self._apply_rate_policy(self._rate_policy_from_config(prev_cfg))
except ValueError:
pass # previous config was validated when it went live
prev_agents_cfg = prev_cfg.get("agents", {})
if prev_agents_cfg:
provider_cfg = prev_cfg.get("provider", {})
@@ -11431,6 +11511,13 @@ class WebSocketGateway:
# drain window keep resolving the previous agents until the cutover
# commits.
report = self._validate_candidate(new_cfg)
# Issue #6215: validate the per-user budget with the candidate so a
# bad value rejects the reload instead of failing mid-cutover.
try:
rate_policy = self._rate_policy_from_config(new_cfg)
except ValueError as e:
report.ok = False
report.failures.append(f"gateway.per_user_per_minute: {e}")
if not report.ok:
error = "; ".join(report.failures) or "candidate validation failed"
logger.error(f"Reload rejected — {error}; keeping previous config")
@@ -11453,6 +11540,7 @@ class WebSocketGateway:
# channel is accepting turns (greptile #1 — never expose the
# candidate before the cutover commits).
self._activate_candidate(report.candidate)
self._apply_rate_policy(rate_policy)
channels_cfg = new_cfg.get("channels", {})
try:
@@ -12181,6 +12269,8 @@ class WebSocketGateway:
queue_depth=_queue_depth,
overflow_policy=_overflow,
max_concurrent_runs_per_scope=_max_runs_per_scope,
# Issue #6215: per-user inbound request budget (0 = unlimited).
per_user_per_minute=gw_cfg.get("per_user_per_minute", 0),
)
if self._admission_gate is not None:
logger.info(
@@ -12436,6 +12526,9 @@ class WebSocketGateway:
except (NotImplementedError, OSError, ValueError):
pass
# Claim the state root BEFORE channels start: channel setup replays the
# inbound journal, which a rejected second gateway must never touch.
self._acquire_state_lock()
try:
await _run_all()
finally:
@@ -12486,3 +12579,4 @@ class WebSocketGateway:
logger.warning("Error draining websocket sessions: %s", e)
finally:
self._cancel_shutdown_deadline()
self._release_locks()
@@ -67,6 +67,37 @@ def test_server_reply_as_dict():
}
def test_server_reply_repr_hides_value():
reply = GatewayServerReply(request_id="r1", value="hunter2")
assert "hunter2" not in repr(reply)
assert reply.value == "hunter2"
def test_secret_request_resolves_only_to_caller():
"""Issue #6223: a secret reply reaches the waiting caller and nowhere else."""
async def go():
gw = WebSocketGateway()
sent = _connect(gw, "c1", "sess-1")
req = GatewayServerRequest(
request_id="req-s", kind="secret", prompt="sudo password?",
session_id="sess-1",
)
task = asyncio.create_task(gw.request_channel.request(req, timeout_s=5))
await asyncio.sleep(0)
assert sent[0]["request"]["kind"] == "secret"
await gw._handle_client_message(
"c1", {"type": "server_reply", "request_id": "req-s", "value": "hunter2"}
)
reply = await asyncio.wait_for(task, timeout=1)
assert reply.value == "hunter2"
# The value is never echoed back over the wire (ack carries no value).
assert all("hunter2" not in repr(frame) for frame in sent)
assert gw.request_channel.open_requests("sess-1") == []
asyncio.run(go())
# ---------------------------------------------------------------------------
# Test harness: a gateway with a connected, session-bound client that records
# every frame the server sends it.
@@ -0,0 +1,125 @@
"""A gateway's single-instance identity is its shared state, not its port (#6208)."""
import asyncio
import multiprocessing
import os
from pathlib import Path
import pytest
from praisonai_bot.gateway.port_utils import (
format_collision_error,
format_state_collision_error,
resolve_state_dir,
state_lock,
)
def _claim(d, start, done, results):
start.wait() # release every worker at once so the claims genuinely race
results.put(state_lock(Path(d)).acquire_lock("127.0.0.1", 8765))
done.wait() # stay alive until all claimed, so no lock is seen as stale
def _seed_foreign_holder(lock, pid):
lock.lock_file.write_text(f"{pid}\n127.0.0.1\n8765\n0\n\n")
def test_second_gateway_on_another_port_is_refused(tmp_path, monkeypatch):
first = state_lock(tmp_path, port=8765)
_seed_foreign_holder(first, os.getppid()) # a live process other than us
second = state_lock(tmp_path, port=8766)
assert second.lock_file == first.lock_file
assert second.acquire_lock("127.0.0.1", 8766) is False
msg = format_state_collision_error(tmp_path, second.get_lock_info())
assert str(tmp_path) in msg and "HOME=" in msg and "PRAISONAI_HOME" not in msg
def test_distinct_state_dirs_are_independent(tmp_path):
a = state_lock(tmp_path / "a")
_seed_foreign_holder(a, os.getppid())
assert state_lock(tmp_path / "b").acquire_lock("127.0.0.1", 8766) is True
def test_stale_state_lock_is_reclaimed(tmp_path, monkeypatch):
lock = state_lock(tmp_path)
_seed_foreign_holder(lock, 999999)
monkeypatch.setattr(lock, "_is_process_running", lambda pid: False)
assert lock.acquire_lock("127.0.0.1", 8765) is True
def test_state_dir_ignores_praisonai_home(tmp_path, monkeypatch):
# The stores live under ~/.praisonai regardless of PRAISONAI_HOME, so a
# different PRAISONAI_HOME must not escape the lock.
monkeypatch.setenv("PRAISONAI_HOME", str(tmp_path))
assert resolve_state_dir() == Path.home() / ".praisonai"
def test_concurrent_claims_have_exactly_one_winner(tmp_path):
ctx = multiprocessing.get_context("spawn")
n = 8
start, done, results = ctx.Barrier(n), ctx.Barrier(n), ctx.Queue()
procs = [ctx.Process(target=_claim, args=(str(tmp_path), start, done, results)) for _ in range(n)]
for p in procs:
p.start()
outcomes = [results.get(timeout=60) for _ in range(n)]
for p in procs:
p.join(timeout=60)
assert sum(outcomes) == 1
def test_claim_never_overwrites_existing_lock(tmp_path):
lock = state_lock(tmp_path)
_seed_foreign_holder(lock, os.getppid())
assert lock.acquire_lock("127.0.0.1", 8766) is False
assert lock.lock_file.read_text().startswith(str(os.getppid()))
def test_failed_startup_releases_both_locks(tmp_path, monkeypatch):
from praisonai_bot.gateway import server as srv
monkeypatch.setenv("HOME", str(tmp_path)) # isolate both lock files
gw = srv.WebSocketGateway.__new__(srv.WebSocketGateway)
gw._is_running, gw._host, gw._port = False, "127.0.0.1", 0
gw._pid_lock = gw._state_lock = None
async def boom():
raise RuntimeError("bad config")
gw._start_locked = boom
monkeypatch.setattr("praisonai_bot.gateway.port_utils.check_port_available", lambda h, p: (True, None))
with pytest.raises(RuntimeError, match="bad config"):
asyncio.run(gw.start())
assert gw._pid_lock is None and gw._state_lock is None
assert not list((tmp_path / ".praisonai").glob("*.pid"))
def test_start_failure_keeps_state_lock_owned_by_start_with_config(tmp_path, monkeypatch):
# start_with_config claims the state lock before channels start; if start()
# then fails, channels are still polling, so start() must not drop it.
from praisonai_bot.gateway import server as srv
monkeypatch.setenv("HOME", str(tmp_path))
gw = srv.WebSocketGateway.__new__(srv.WebSocketGateway)
gw._is_running, gw._host, gw._port = False, "127.0.0.1", 0
gw._pid_lock = gw._state_lock = None
gw._acquire_state_lock()
held = gw._state_lock
async def boom():
raise RuntimeError("bind error")
gw._start_locked = boom
monkeypatch.setattr("praisonai_bot.gateway.port_utils.check_port_available", lambda h, p: (True, None))
with pytest.raises(RuntimeError, match="bind error"):
asyncio.run(gw.start())
assert gw._pid_lock is None
assert gw._state_lock is held and held.lock_file.exists()
gw._release_locks() # start_with_config's finally, after channels stop
assert gw._state_lock is None and not held.lock_file.exists()
def test_collision_error_no_longer_suggests_bare_port_change():
msg = format_collision_error("127.0.0.1", 8765, {"pid": 1, "is_running": True})
assert "Or use a different port" not in msg
assert "HOME=" in msg and "PRAISONAI_HOME" not in msg
@@ -0,0 +1,240 @@
"""Per-user inbound request budget in the admission gate (Issue #6215)."""
import asyncio
import pytest
from praisonai_bot.bots._admission import (
AdmissionGate,
AdmissionRejected,
build_admission_gate,
)
async def _turn(gate, user):
async with gate.admit(session_id=user):
return "ran"
def test_unconfigured_means_no_gate():
assert build_admission_gate(per_user_per_minute=0) is None
def test_negative_budget_fails_fast():
with pytest.raises(ValueError):
build_admission_gate(per_user_per_minute=-1)
def test_over_budget_user_refused_with_retry_after_others_unaffected():
gate = build_admission_gate(per_user_per_minute=2)
assert gate is not None and gate.enabled
async def run():
assert await _turn(gate, "alice") == "ran"
assert await _turn(gate, "alice") == "ran"
with pytest.raises(AdmissionRejected) as exc:
await _turn(gate, "alice")
# A different user still has their own budget.
assert await _turn(gate, "bob") == "ran"
return exc.value
rej = asyncio.run(run())
assert rej.retry_after is not None and 1 <= rej.retry_after <= 60
assert "try again in" in rej.message
stats = gate.stats()
assert stats["rate_limited"] == 1
assert stats["per_user_per_minute"] == 2
assert stats["in_flight"] == 0
def test_rate_limit_composes_with_concurrency_gate():
gate = build_admission_gate(max_concurrent_runs=4, per_user_per_minute=1)
assert gate.stats()["max_concurrent_runs"] == 4
async def run():
await _turn(gate, "u")
with pytest.raises(AdmissionRejected):
await _turn(gate, "u")
asyncio.run(run())
assert gate.stats()["in_flight"] == 0
def test_stale_identities_are_pruned():
from praisonaiagents.gateway import SlidingWindowRateLimitPolicy
pol = SlidingWindowRateLimitPolicy(max_requests=1, window_seconds=60)
for i in range(10):
pol.check(identity=f"u{i}", scope="ingress", now=0.0)
AdmissionGate._prune_rate_limit_state(pol, now=120.0, cap=5)
assert pol._windows == {}
def test_botos_kwarg_and_yaml(tmp_path):
try:
from praisonai_bot.bots.botos import BotOS
except ModuleNotFoundError: # pragma: no cover - optional deps
pytest.skip("BotOS optional dependency not installed")
assert BotOS(per_user_per_minute=5).admission_stats["per_user_per_minute"] == 5
pytest.importorskip("yaml")
cfg = tmp_path / "botos.yaml"
cfg.write_text("gateway:\n per_user_per_minute: 7\n")
assert BotOS.from_config(str(cfg)).admission_stats["per_user_per_minute"] == 7
@pytest.mark.parametrize("bad", [0.5, -0.5, 1.5, "0.5", "abc", True, -1])
def test_fractional_or_invalid_budget_rejected(bad):
with pytest.raises(ValueError):
build_admission_gate(per_user_per_minute=bad)
def test_botos_yaml_fractional_budget_rejected(tmp_path):
try:
from praisonai_bot.bots.botos import BotOS
except ModuleNotFoundError: # pragma: no cover - optional deps
pytest.skip("BotOS optional dependency not installed")
pytest.importorskip("yaml")
cfg = tmp_path / "botos.yaml"
cfg.write_text("gateway:\n per_user_per_minute: 0.5\n")
with pytest.raises(ValueError):
BotOS.from_config(str(cfg))
with pytest.raises(ValueError):
BotOS(per_user_per_minute=0.5)
def test_rate_key_shared_across_new_sessions():
"""A caller cannot reset the budget by switching conversation ids."""
gate = build_admission_gate(per_user_per_minute=1)
async def run():
async with gate.admit(session_id="conv-1", rate_key="ip:1.2.3.4"):
pass
with pytest.raises(AdmissionRejected):
async with gate.admit(session_id="conv-2", rate_key="ip:1.2.3.4"):
pass
asyncio.run(run())
def test_prune_runs_at_bounded_interval(monkeypatch):
gate = build_admission_gate(per_user_per_minute=5)
calls = []
monkeypatch.setattr(
AdmissionGate, "_prune_rate_limit_state",
staticmethod(lambda pol, now, cap=4096: calls.append(now)),
)
for i in range(50):
gate._check_rate_limit(f"user-{i}")
assert len(calls) == 1
def test_set_rate_limit_policy_swaps_budget_in_place():
from praisonai_bot.bots._admission import build_rate_limit_policy
gate = build_admission_gate(max_concurrent_runs=4)
gate.set_rate_limit_policy(build_rate_limit_policy(1))
assert gate.stats()["per_user_per_minute"] == 1
async def run():
await _turn(gate, "u")
with pytest.raises(AdmissionRejected):
await _turn(gate, "u")
asyncio.run(run())
gate.set_rate_limit_policy(build_rate_limit_policy(0))
assert gate.stats()["per_user_per_minute"] == 0
assert asyncio.run(_turn(gate, "u")) == "ran"
def test_gateway_reload_applies_budget():
try:
from praisonai_bot.gateway.server import WebSocketGateway
except ModuleNotFoundError: # pragma: no cover - optional deps
pytest.skip("gateway optional dependency not installed")
gw = WebSocketGateway.__new__(WebSocketGateway)
gw._admission_gate = None
gw._apply_rate_policy(
gw._rate_policy_from_config({"gateway": {"per_user_per_minute": 3}})
)
assert gw._admission_gate.stats()["per_user_per_minute"] == 3
gw._apply_rate_policy(gw._rate_policy_from_config({"gateway": {}}))
assert gw._admission_gate.stats()["per_user_per_minute"] == 0
with pytest.raises(ValueError):
gw._rate_policy_from_config({"gateway": {"per_user_per_minute": 0.5}})
def test_api_rate_key_ignores_session_header():
try:
from praisonai_bot.gateway.api_endpoints import GatewayApiEndpoints
except ModuleNotFoundError: # pragma: no cover - optional deps
pytest.skip("gateway optional dependency not installed")
class _Req:
def __init__(self, headers, host="10.0.0.1"):
self.headers = headers
self.client = type("C", (), {"host": host})()
a = GatewayApiEndpoints._rate_key(_Req({"x-session-id": "one"}))
b = GatewayApiEndpoints._rate_key(_Req({"x-session-id": "two"}))
assert a == b == "ip:10.0.0.1"
tok = GatewayApiEndpoints._rate_key(
_Req({"authorization": "Bearer secret-token"}), "secret-token"
)
assert tok.startswith("token:") and "secret" not in tok
def test_api_rate_key_rotating_unvalidated_bearer_shares_ip_budget():
"""An unvalidated bearer (auth open, or wrong token) cannot mint budgets."""
try:
from praisonai_bot.gateway.api_endpoints import GatewayApiEndpoints
except ModuleNotFoundError: # pragma: no cover - optional deps
pytest.skip("gateway optional dependency not installed")
class _Req:
def __init__(self, headers, host="10.0.0.1"):
self.headers = headers
self.client = type("C", (), {"host": host})()
for configured in (None, "", "real-token"):
keys = {
GatewayApiEndpoints._rate_key(
_Req({"authorization": f"Bearer rotate-{i}"}), configured
)
for i in range(5)
}
assert keys == {"ip:10.0.0.1"}
def test_api_rate_key_missing_peer_never_falls_back_to_session():
try:
from praisonai_bot.gateway.api_endpoints import GatewayApiEndpoints
except ModuleNotFoundError: # pragma: no cover - optional deps
pytest.skip("gateway optional dependency not installed")
class _Req:
headers = {"x-session-id": "chosen"}
client = None
assert GatewayApiEndpoints._rate_key(_Req()) == "ip:unknown"
gate = build_admission_gate(per_user_per_minute=1)
async def run():
async with gate.admit(session_id="conv-1", rate_key="ip:unknown"):
pass
with pytest.raises(AdmissionRejected):
async with gate.admit(session_id="conv-2", rate_key="ip:unknown"):
pass
asyncio.run(run())
def test_ws_rate_key_missing_peer_is_shared():
try:
from praisonai_bot.gateway.server import WebSocketGateway
except ModuleNotFoundError: # pragma: no cover - optional deps
pytest.skip("gateway optional dependency not installed")
gw = WebSocketGateway.__new__(WebSocketGateway)
gw._clients = {}
assert gw._client_rate_key(None) == "ip:unknown"
assert gw._client_rate_key("gone") == "ip:unknown"
@@ -239,3 +239,60 @@ class TestBotOSFromConfigEnhanced:
assert botos.list_bots() == []
finally:
os.unlink(path)
class TestBotOSFromConfigGatewayDrift:
"""Issue #6210: gateway.yaml must not be silently degraded by BotOS."""
def _write_yaml(self, data):
f = tempfile.NamedTemporaryFile(mode="w", suffix=".yaml", delete=False)
yaml.dump(data, f)
f.close()
return f.name
@pytest.mark.parametrize("section", ["channels", "agents", "routing", "schedules"])
def test_gateway_sections_raise(self, section):
from praisonai_bot.bots.botos import BotOS
path = self._write_yaml({
"platforms": {"telegram": {"token": "fake"}},
section: {"x": {}},
})
try:
with pytest.raises(ValueError, match="start_with_config"):
BotOS.from_config(path)
finally:
os.unlink(path)
def test_unknown_keys_warn(self, caplog):
from praisonai_bot.bots.botos import BotOS
path = self._write_yaml({
"platforms": {"telegram": {"token": "fake"}},
"helth": {"interval": 5},
"gateway": {"port": 9999, "drain_timeout": 5},
})
try:
with caplog.at_level("WARNING"):
botos = BotOS.from_config(path)
assert "telegram" in botos.list_bots()
assert "helth" in caplog.text and "gateway.port" in caplog.text
assert "gateway.drain_timeout" not in caplog.text
finally:
os.unlink(path)
def test_non_string_unknown_key_warns_not_crashes(self, caplog):
from praisonai_bot.bots.botos import BotOS
path = self._write_yaml({
"platforms": {"telegram": {"token": "fake"}},
123: "ignored",
"zzz": 1,
})
try:
with caplog.at_level("WARNING"):
botos = BotOS.from_config(path)
assert "telegram" in botos.list_bots()
assert "123" in caplog.text and "zzz" in caplog.text
finally:
os.unlink(path)
@@ -7,6 +7,7 @@ Provides CLI interface for running agents in autonomous loops with:
- Iteration limits and timeouts
"""
import math
from typing import Optional
import typer
@@ -31,6 +32,8 @@ def loop_main(
observe: bool = typer.Option(False, "--observe", "-o", help="Enable observability event logging"),
recovery: bool = typer.Option(True, "--recovery/--no-recovery", help="Enable graduated doom loop recovery (default: on)"),
level: Optional[str] = typer.Option(None, "--level", "-l", help="Autonomy level: suggest, auto_edit, full_auto"),
max_budget: Optional[float] = typer.Option(None, "--max-budget", help="Hard USD spend cap for the whole run"),
on_budget_exceeded: str = typer.Option("stop", "--on-budget-exceeded", help="When --max-budget is hit: stop or warn"),
):
"""Run an agent in an autonomous loop.
@@ -66,6 +69,8 @@ def loop_main(
observe=observe,
recovery=recovery,
level=level,
max_budget=max_budget,
on_budget_exceeded=on_budget_exceeded,
)
@@ -80,8 +85,17 @@ def _run_loop(
observe: bool = False,
recovery: bool = True,
level: Optional[str] = None,
max_budget: Optional[float] = None,
on_budget_exceeded: str = "stop",
):
"""Execute the autonomous loop."""
if on_budget_exceeded not in ("stop", "warn"):
typer.echo("Error: --on-budget-exceeded must be 'stop' or 'warn'", err=True)
raise typer.Exit(2)
# 0 would mean "no cap" downstream; negatives and NaN/inf are never a real cap.
if max_budget is not None and not (math.isfinite(max_budget) and max_budget > 0):
typer.echo("Error: --max-budget must be a positive number", err=True)
raise typer.Exit(2)
try:
from praisonaiagents import Agent
except ImportError:
@@ -117,6 +131,13 @@ def _run_loop(
if not verbose:
agent_kwargs["output"] = "silent"
# Only forward a budget when the user set one (None = no cap, zero overhead).
if max_budget is not None:
agent_kwargs["execution"] = {
"max_budget": max_budget,
"on_budget_exceeded": on_budget_exceeded,
}
# Add completion promise instructions if provided
if completion_promise:
@@ -138,6 +159,8 @@ def _run_loop(
typer.echo(" Context clearing: enabled")
if timeout:
typer.echo(f" Timeout: {timeout}s")
if max_budget is not None:
typer.echo(f" Max budget: ${max_budget} ({on_budget_exceeded})")
typer.echo("")
# Run autonomous loop using unified API
@@ -200,6 +223,8 @@ OPTIONS:
-p, --completion-promise STR Promise text to signal completion
-c, --clear-context Clear chat history between iterations
-t, --timeout FLOAT Timeout in seconds
--max-budget FLOAT Hard USD spend cap for the whole run
--on-budget-exceeded STR stop (default) or warn when the cap is hit
-m, --model STR LLM model to use
-v, --verbose Show verbose output
@@ -1712,6 +1712,7 @@ def run_main(
# execution path does not discover and execute the same definitions twice.
preloaded_agent_config = None
custom_agent_model = None
custom_agent_base_url = None
if target or agent or command: # Any execution surface may need model/budget resolution
import sys
from praisonai_code.llm.credentials import ensure_configured_or_onboard
@@ -1731,6 +1732,10 @@ def run_main(
_preview_llm = (_preview_config or {}).get("llm")
if isinstance(_preview_llm, dict):
custom_agent_model = _preview_llm.get("model")
# Gate an Ollama agent on the server it will really call.
_base = _preview_llm.get("base_url")
if isinstance(_base, str) and _base.strip():
custom_agent_base_url = _base.strip()
else:
custom_agent_model = _preview_llm
if isinstance(custom_agent_model, str):
@@ -1747,7 +1752,9 @@ def run_main(
# endpoint, credentials, sampling, and retry options.
credential_model = model or custom_agent_model
resolved_model = ensure_configured_or_onboard(
model=credential_model, interactive=not _headless
model=credential_model,
interactive=not _headless,
base_url=None if model else custom_agent_base_url,
)
if not (agent and model is None and custom_agent_model):
model = resolved_model
@@ -123,6 +123,43 @@ def detect_local_endpoint():
return None
_OLLAMA_PREFIXES = ("ollama/", "ollama_chat/")
def _ollama_host_for_model(
model: Optional[str], base_url: Optional[str] = None
) -> Optional[str]:
"""Return the Ollama host to probe for an ``ollama/`` model, else ``None``.
Ollama needs no key, so an explicit Ollama model is gated on the server
being reachable rather than on a credential. The endpoint the caller will
actually use (e.g. an agent's ``llm.base_url``) wins, then
``OLLAMA_API_BASE`` (what the runtime honours), then ``OLLAMA_HOST``, then
the default port.
"""
import os
if not model or not model.lower().startswith(_OLLAMA_PREFIXES):
return None
from praisonai_code.llm.local_detect import _DEFAULT_OLLAMA_HOST
return (
base_url
or os.environ.get("OLLAMA_API_BASE")
or os.environ.get("OLLAMA_HOST")
or _DEFAULT_OLLAMA_HOST
)
def _ollama_reachable(host: str) -> bool:
try:
from praisonai_code.llm.local_detect import ollama_reachable
return ollama_reachable(host)
except Exception:
return False
def _provider_key_vars_for_model(model: str) -> tuple[str, ...]:
"""Map a model id to the environment variable(s) for its provider.
@@ -258,6 +295,10 @@ def is_configured(model: Optional[str] = None) -> bool:
if model is None:
model = default_model_for_available_provider()
ollama_host = _ollama_host_for_model(model) if explicit_model else None
if ollama_host is not None:
return _ollama_reachable(ollama_host)
required_vars = _provider_key_vars_for_model(model)
if required_vars:
@@ -303,6 +344,7 @@ def ensure_configured_or_onboard(
*,
model: Optional[str] = None,
interactive: bool = True,
base_url: Optional[str] = None,
) -> Optional[str]:
"""Shared first-run credential gate for LLM-bound interactive commands.
@@ -331,6 +373,9 @@ def ensure_configured_or_onboard(
keyless local-first fallback (its own provider gate applies).
interactive: Whether a wizard prompt may be shown. Callers pass ``False``
for headless (``-p``, piped, ``--output json``) paths.
base_url: The endpoint the caller will actually use (e.g. a custom
agent's ``llm.base_url``). For an explicit Ollama model this is the
server probed, instead of ``OLLAMA_API_BASE``/``OLLAMA_HOST``.
Returns:
The model id to proceed with. May be a detected local model id when the
@@ -343,6 +388,21 @@ def ensure_configured_or_onboard(
import typer
inject_credentials_into_env()
# An explicit Ollama model needs a running server, not a key: probe the
# endpoint the caller will really use and, if it is down, say so instead
# of sending the user to an API-key wizard.
ollama_host = _ollama_host_for_model(model, base_url)
if ollama_host is not None:
if _ollama_reachable(ollama_host):
return model
typer.echo(
f"Error: Ollama isn't reachable at {ollama_host} — start it with "
"`ollama serve` (or set OLLAMA_API_BASE).",
err=True,
)
raise typer.Exit(1)
if is_configured(model):
return model
@@ -360,11 +420,16 @@ def ensure_configured_or_onboard(
return local.model
if not interactive:
# Only mention auto-detection when it actually ran and found nothing.
hint = f" for model {model}" if model else ""
tail = (
"" if model else
"\n(no running local endpoint such as Ollama was found; one "
"would be used automatically)"
)
typer.echo(
"Error: No API key configured. Run: praisonai setup\n"
"or set environment variables like OPENAI_API_KEY\n"
"(a running local endpoint such as Ollama would be used "
"automatically)",
f"Error: No API key configured{hint}. Run: praisonai setup\n"
f"or set environment variables like OPENAI_API_KEY{tail}",
err=True,
)
raise typer.Exit(1)
@@ -19,9 +19,10 @@ from dataclasses import dataclass
from typing import Optional
from urllib.parse import urlsplit
# Total time budget for a probe so the credential/first-run hot path is never
# stalled when nothing is listening. Kept small deliberately.
_PROBE_TIMEOUT_S = 0.15
# Per-request probe timeout. Only paid on the keyless path; a closed port is
# refused instantly, while a running Ollama can take ~0.35 s on its first call,
# so a tighter budget made detection flaky (issue #6228).
_PROBE_TIMEOUT_S = 1.0
# How long a negative probe is remembered so repeated invocations in the same
# process (e.g. bare TUI then run) don't re-pay the connection latency.
@@ -187,6 +188,11 @@ def _probe_ollama_tags(host: str) -> Optional[str]:
return None
def ollama_reachable(host: str) -> bool:
"""Return ``True`` if an Ollama server answers ``/api/tags`` at ``host``."""
return _get_json(_root_host(_with_scheme(host)) + "/api/tags") is not None
def detect_local_model(*, use_cache: bool = True) -> Optional[LocalModel]:
"""Detect a reachable local OpenAI-compatible endpoint.
@@ -0,0 +1,76 @@
"""`praisonai loop --max-budget` reaches the agent's ExecutionConfig (Issue #6206)."""
import pytest
pytest.importorskip("typer")
import typer
from praisonai_code.cli.commands import loop as loop_module
class _Result:
success = True
completion_reason = "promise"
iterations = 1
duration_seconds = 0.1
started_at = None
output = "done"
error = None
def _capture(monkeypatch):
captured = {}
class FakeAgent:
def __init__(self, **kwargs):
captured["agent_kwargs"] = kwargs
def start(self, prompt, **kwargs):
return _Result()
import praisonaiagents
monkeypatch.setattr(praisonaiagents, "Agent", FakeAgent)
return captured
def _run(**extra):
loop_module._run_loop(
prompt="build it", max_iterations=2, completion_promise=None,
clear_context=False, timeout=None, model=None, verbose=False, **extra,
)
def test_max_budget_forwarded_to_execution(monkeypatch):
captured = _capture(monkeypatch)
_run(max_budget=0.5, on_budget_exceeded="warn")
assert captured["agent_kwargs"]["execution"] == {
"max_budget": 0.5, "on_budget_exceeded": "warn",
}
def test_no_budget_leaves_execution_unset(monkeypatch):
captured = _capture(monkeypatch)
_run()
assert "execution" not in captured["agent_kwargs"]
def test_invalid_on_budget_exceeded_rejected(monkeypatch):
_capture(monkeypatch)
with pytest.raises(typer.Exit):
_run(max_budget=1.0, on_budget_exceeded="explode")
def test_budget_dict_builds_real_execution_config():
pytest.importorskip("praisonaiagents")
from praisonaiagents import Agent
agent = Agent(name="a", execution={"max_budget": 0.25, "on_budget_exceeded": "stop"})
assert agent._max_budget == 0.25
@pytest.mark.parametrize("bad", [0.0, -1.0, float("nan"), float("inf")])
def test_invalid_max_budget_rejected(monkeypatch, bad):
captured = _capture(monkeypatch)
with pytest.raises(typer.Exit):
_run(max_budget=bad)
assert "agent_kwargs" not in captured
@@ -0,0 +1,144 @@
"""Tests for issue #6228 — explicit ``ollama/`` models need no API key.
An explicit Ollama model is gated on the server being reachable, not on
``OLLAMA_HOST`` being set; the probe tolerates a slow first response; and the
error text for an explicit model does not claim auto-detection would kick in.
"""
import io
import json
import pytest
import typer
from praisonai_code.llm import credentials, local_detect
_ENV = (
"OPENAI_API_KEY", "ANTHROPIC_API_KEY", "GEMINI_API_KEY", "GOOGLE_API_KEY",
"GROQ_API_KEY", "COHERE_API_KEY", "OPENROUTER_API_KEY", "OLLAMA_HOST",
"OLLAMA_API_BASE", "OPENAI_BASE_URL", "MODEL_NAME", "OPENAI_MODEL_NAME",
"PRAISONAI_LOCAL_ENDPOINTS",
)
@pytest.fixture
def clean_env(monkeypatch):
for var in _ENV:
monkeypatch.delenv(var, raising=False)
monkeypatch.setattr(credentials, "inject_credentials_into_env", lambda: False)
local_detect.reset_cache()
yield monkeypatch
local_detect.reset_cache()
class _Resp(io.BytesIO):
status = 200
def __enter__(self):
return self
def __exit__(self, *a):
return False
def _fake_urlopen(seen, delay=0.0):
def _urlopen(url, timeout):
seen.append((url, timeout))
if delay > timeout:
raise TimeoutError("probe timed out")
return _Resp(json.dumps({"models": [{"name": "llama3.2:latest"}]}).encode())
return _urlopen
def _refuse(url, timeout):
raise ConnectionRefusedError()
def test_explicit_ollama_configured_when_reachable(clean_env):
seen = []
clean_env.setattr(local_detect.urllib.request, "urlopen", _fake_urlopen(seen))
assert credentials.is_configured("ollama/llama3.2") is True
assert seen[0][0] == "http://127.0.0.1:11434/api/tags"
def test_explicit_ollama_honours_ollama_api_base(clean_env):
seen = []
clean_env.setenv("OLLAMA_API_BASE", "http://gpu-box:11434")
clean_env.setattr(local_detect.urllib.request, "urlopen", _fake_urlopen(seen))
assert credentials.is_configured("ollama_chat/llama3.2") is True
assert seen[0][0] == "http://gpu-box:11434/api/tags"
def test_explicit_ollama_not_configured_when_unreachable(clean_env):
clean_env.setattr(local_detect.urllib.request, "urlopen", _refuse)
assert credentials.is_configured("ollama/llama3.2") is False
def test_unreachable_ollama_error_is_actionable(clean_env, capsys):
clean_env.setattr(local_detect.urllib.request, "urlopen", _refuse)
with pytest.raises(typer.Exit):
credentials.ensure_configured_or_onboard(
model="ollama/llama3.2", interactive=False
)
err = capsys.readouterr().err
assert "Ollama isn't reachable at http://127.0.0.1:11434" in err
assert "ollama serve" in err
assert "API key" not in err
def test_explicit_ollama_passes_gate_when_reachable(clean_env):
clean_env.setattr(local_detect.urllib.request, "urlopen", _fake_urlopen([]))
assert credentials.ensure_configured_or_onboard(
model="ollama/llama3.2", interactive=False
) == "ollama/llama3.2"
def test_probe_tolerates_slow_first_response(clean_env):
"""A ~0.4 s first /api/tags response must still be detected."""
clean_env.setattr(
local_detect.urllib.request, "urlopen", _fake_urlopen([], delay=0.4)
)
result = local_detect.detect_local_model(use_cache=False)
assert result is not None
assert result.model == "ollama/llama3.2:latest"
def test_explicit_cloud_model_error_omits_auto_detect_hint(clean_env, capsys):
clean_env.setattr(credentials, "detect_local_endpoint", lambda: None)
clean_env.setattr(credentials, "is_configured", lambda model=None: False)
with pytest.raises(typer.Exit):
credentials.ensure_configured_or_onboard(
model="gpt-4o-mini", interactive=False
)
err = capsys.readouterr().err
assert "for model gpt-4o-mini" in err
assert "automatically" not in err
def test_keyless_no_model_error_mentions_auto_detect(clean_env, capsys):
clean_env.setattr(credentials, "detect_local_endpoint", lambda: None)
clean_env.setattr(credentials, "is_configured", lambda model=None: False)
with pytest.raises(typer.Exit):
credentials.ensure_configured_or_onboard(model=None, interactive=False)
assert "automatically" in capsys.readouterr().err
def test_gate_probes_agent_base_url_not_env_host(clean_env):
"""A custom agent's ``llm.base_url`` is the server gated, not OLLAMA_HOST."""
seen = []
clean_env.setenv("OLLAMA_HOST", "http://down-box:11434")
def _urlopen(url, timeout):
seen.append(url)
if "down-box" in url:
raise ConnectionRefusedError()
return _Resp(json.dumps({"models": []}).encode())
clean_env.setattr(local_detect.urllib.request, "urlopen", _urlopen)
assert credentials.ensure_configured_or_onboard(
model="ollama/llama3.2",
interactive=False,
base_url="http://gpu-box:11434/v1",
) == "ollama/llama3.2"
assert seen == ["http://gpu-box:11434/api/tags"]
@@ -717,6 +717,9 @@ def train_export(
mtp_draft: bool = typer.Option(
False, "--mtp-draft/--no-mtp-draft",
help="Also download the stock MTP drafter (Gemma-4 only) for fast inference"),
push: Optional[bool] = typer.Option(
None, "--push/--no-push",
help="ollama: push to ollama.com after `ollama create` (--no-push keeps it local)"),
):
"""
Export / publish an ALREADY-trained model without re-running training.
@@ -726,6 +729,7 @@ def train_export(
praisonai-train export gguf --model-dir lora_model --quant q4_k_m # local .gguf
praisonai-train export gguf --model-dir lora_model --hf me/my-model # local + push
praisonai-train export ollama --model-dir lora_model --ollama me/my-model --quant q4_k_m
praisonai-train export ollama --model-dir lora_model --ollama my-model --no-push # local only
"""
from ..output.console import get_output_controller
@@ -765,6 +769,9 @@ def train_export(
cfg["hf_model_name"] = hf
if ollama:
cfg["ollama_model"] = ollama
# Either explicit flag overrides the config; unset keeps config/default (push).
if isinstance(push, bool):
cfg["ollama_push"] = push
# Recover the model name (drives chat-template selection) from --base-model or
# the trained model's config.json (_name_or_path), falling back to the dir name.
@@ -1,4 +1,5 @@
ollama_save: "true"
ollama_push: "true" # set "false" to keep the Ollama model local (no upload to ollama.com)
huggingface_save: "true"
train: "true"
@@ -220,8 +220,9 @@ def create_and_push_ollama_model(
model_parameters: str,
modelfile_content: str,
quantization: Optional[str] = None,
push: bool = True,
) -> None:
"""Create and push an Ollama model with proper daemon management.
"""Create (and by default push) an Ollama model with proper daemon management.
Args:
ollama_model: Name of the Ollama model
@@ -229,6 +230,8 @@ def create_and_push_ollama_model(
modelfile_content: Content for the Modelfile
quantization: Optional quantization (e.g. "q4_k_m") passed to
``ollama create --quantize`` so the model isn't stored as huge f16.
push: When False, stop after a successful ``ollama create`` and keep the
model local-only (nothing is uploaded to ollama.com).
Raises:
RuntimeError: If ollama create/push operations fail (with actionable guidance)
@@ -259,6 +262,10 @@ def create_and_push_ollama_model(
f"{(create.stderr or create.stdout or '').strip()}"
)
if not push:
print(f"Created local Ollama model `{tag}` (not pushed). Run it with: ollama run {tag}")
return
# Push with captured output so a 401/403 becomes an actionable message rather
# than a raw non-zero exit.
result = subprocess.run(["ollama", "push", tag], capture_output=True, text=True)
@@ -545,6 +545,7 @@ class TrainModel:
"optim", "weight_decay", "lr_scheduler_type", "seed", "output_dir",
"assistant_only_loss", "train_on_responses_only", "save_steps",
"train", "huggingface_save", "huggingface_save_gguf", "ollama_save",
"ollama_push",
"method", "beta", "max_prompt_length", "desirable_weight", "undesirable_weight",
"reward_funcs", "num_generations", "max_completion_length",
"embedding_learning_rate",
@@ -951,6 +952,10 @@ class TrainModel:
if self._flag(dataset_info.get("shuffle"), default=False):
dataset = dataset.shuffle(
seed=int(dataset_info.get("seed", self.config.get("seed", 3407))))
# OpenAI-style chat rows ({"messages": [...]}, as written by
# praisonaiagents.eval.export_sft) are the ShareGPT shape under another name.
if "messages" in dataset.column_names and "conversations" not in dataset.column_names:
dataset = dataset.rename_column("messages", "conversations")
if "conversations" in dataset.column_names:
print("DEBUG: Standardizing dataset (ShareGPT style)...")
dataset = standardize_sharegpt(dataset)
@@ -970,8 +975,8 @@ class TrainModel:
if len(dataset) == 0:
raise ValueError(
"All examples formatted to empty text. The dataset columns are "
f"{columns_before}; this trainer understands ShareGPT "
"('conversations'), Alpaca ('instruction'/'input'/'output') and "
f"{columns_before}; this trainer understands chat "
"('conversations' or 'messages'), Alpaca ('instruction'/'input'/'output') and "
"raw text ('text'). Rename your columns to one of those shapes, "
"or set chat_template to match your data.")
return dataset
@@ -1784,6 +1789,7 @@ class TrainModel:
self.config.get('model_parameters', 'latest'),
modelfile_content,
quantization=self.config.get('quantization_method'),
push=self._flag(self.config.get('ollama_push'), default=True),
)
def run(self):
@@ -313,7 +313,8 @@ PARAMETER top_p 0.9
create_and_push_ollama_model(
self.config['ollama_model'],
self.config['model_parameters'],
modelfile_content
modelfile_content,
push=self._flag(self.config.get('ollama_push'), default=True),
)
def run(self):
@@ -0,0 +1,58 @@
"""An OpenAI-style `messages` chat dataset formatted to empty text.
`praisonai-train validate` and `praisonaiagents.eval.export_sft` both use the
`{"messages": [...]}` shape, but `process_dataset` only recognised
`conversations`, so every row fell into the Alpaca branch, formatted to "",
and the run died after the base model had already been loaded.
"""
from praisonai_train.train.llm import trainer as trainer_mod
class _FakeDataset:
def __init__(self, rows):
self.rows = rows
@property
def column_names(self):
return list(self.rows[0].keys()) if self.rows else []
def __len__(self):
return len(self.rows)
def rename_column(self, src, dst):
return _FakeDataset([{(dst if k == src else k): v for k, v in r.items()}
for r in self.rows])
def map(self, func, batched, remove_columns):
batch = {c: [r[c] for r in self.rows] for c in self.column_names}
out = func(batch)
return _FakeDataset([{"text": t} for t in out["text"]])
def filter(self, pred):
return _FakeDataset([r for r in self.rows if pred(r)])
class _Tokenizer:
def apply_chat_template(self, convo, tokenize, add_generation_prompt):
return "".join(f"<{m['role']}>{m['content']}" for m in convo)
def test_messages_column_is_treated_as_conversations(monkeypatch):
rows = [{"messages": [
{"role": "system", "content": "s"},
{"role": "user", "content": f"q{i}"},
{"role": "assistant", "content": f"a{i}"},
]} for i in range(3)]
monkeypatch.setattr(trainer_mod, "load_dataset",
lambda *a, **k: _FakeDataset(rows), raising=False)
monkeypatch.setattr(trainer_mod, "standardize_sharegpt", lambda d: d,
raising=False)
tm = trainer_mod.TrainModel.__new__(trainer_mod.TrainModel)
tm.config = {"method": "sft"}
tm.chat_tokenizer = _Tokenizer()
out = tm.process_dataset({"name": "hub/some-dataset"})
assert len(out) == 3
assert out.rows[0]["text"] == "<system>s<user>q0<assistant>a0"
@@ -412,3 +412,88 @@ def test_export_command_bad_target_exits(monkeypatch, tmp_path):
quant=None,
base_model=None,
)
def test_ollama_no_push_skips_push(monkeypatch, tmp_path):
"""push=False stops after `ollama create`; nothing is sent to ollama.com."""
monkeypatch.chdir(tmp_path)
monkeypatch.setattr(ollama_mod, "_check_ollama_disk", lambda *_a, **_k: None)
monkeypatch.setattr(ollama_mod, "ensure_ollama_running", lambda *_a, **_k: None)
calls = []
def _fake_run(cmd, *a, **k):
calls.append(cmd)
return types.SimpleNamespace(returncode=0, stdout="", stderr="")
monkeypatch.setattr(ollama_mod.subprocess, "run", _fake_run)
ollama_mod.create_and_push_ollama_model(
"my-model", "latest", "FROM ./model\n", push=False)
assert [c[1] for c in calls] == ["create"]
assert not any("push" in c for c in calls)
def _ollama_export_cfg(monkeypatch, tmp_path, push, config_push=None):
"""Run `export ollama` with a fake trainer; return the config it received."""
import praisonai_train.cli.commands.train as cli
seen = {}
class _FakeTrainer:
@classmethod
def for_export(cls, cfg):
obj = cls()
obj.config = cfg
obj.model = None
obj.hf_tokenizer = None
return obj
def load_model(self):
return ("MODEL", "TOK")
def create_and_push_ollama_model(self):
seen.update(self.config)
monkeypatch.setattr(trainer_mod, "TrainModel", _FakeTrainer)
config = None
if config_push is not None:
config = tmp_path / "cfg.yaml"
config.write_text(f"ollama_push: {config_push}\n")
cli.train_export(
target="ollama", model_dir=str(tmp_path), config=str(config) if config else None,
ollama="my-model", hf=None, quant=None, base_model="unsloth/gemma-2-2b",
mtp_draft=False, push=push,
)
return seen
def test_export_ollama_no_push_flag_sets_config(monkeypatch, tmp_path):
assert _ollama_export_cfg(monkeypatch, tmp_path, push=False)["ollama_push"] is False
def test_export_ollama_explicit_push_overrides_config(monkeypatch, tmp_path):
cfg = _ollama_export_cfg(monkeypatch, tmp_path, push=True, config_push="false")
assert cfg["ollama_push"] is True
def test_export_ollama_unset_push_keeps_config(monkeypatch, tmp_path):
cfg = _ollama_export_cfg(monkeypatch, tmp_path, push=None, config_push="false")
assert cfg["ollama_push"] is False # YAML bool preserved, not overridden
assert "ollama_push" not in _ollama_export_cfg(monkeypatch, tmp_path, push=None)
@pytest.mark.parametrize("value, expected", [
(False, False), ("false", False), (True, True), ("true", True), (None, True),
])
def test_llm_trainer_forwards_ollama_push(monkeypatch, value, expected):
"""TrainModel forwards `ollama_push` (bool or string) to the helper."""
captured = {}
monkeypatch.setattr(
ollama_mod, "create_and_push_ollama_model",
lambda *a, **k: captured.update(k))
fake = types.SimpleNamespace(
config={"ollama_model": "m", "ollama_push": value},
prepare_modelfile_content=lambda: "FROM ./model\n",
_flag=trainer_mod.TrainModel._flag,
)
trainer_mod.TrainModel.create_and_push_ollama_model(fake)
assert captured["push"] is expected
+3 -1
View File
@@ -1,6 +1,8 @@
# PraisonAI TypeScript Node.js AI Agents Framework
PraisonAI is a production-ready Multi AI Agents framework, designed to create AI Agents to automate and solve problems ranging from simple tasks to complex challenges. It provides a low-code solution to streamline the building and management of multi-agent LLM systems, emphasising simplicity, customisation, and effective human-agent collaboration.
PraisonAI runs AI agents on your own computers, even fully offline, so private data never has to leave the building. It is built for highly regulated teams, and free and open source for everyone.
This package is the TypeScript and JavaScript SDK of the open-source PraisonAI agent framework. Build single agents, multi-agent workflows, tools and memory in Node.js, on hosted models or on local models such as Ollama. Data stays on your own machines only when the model runs locally too; with a hosted model such as the default OpenAI one, prompts are sent to that provider. The Python framework and the full docs are at [github.com/MervinPraison/PraisonAI](https://github.com/MervinPraison/PraisonAI) and [praison.ai/docs](https://praison.ai/docs).
## Installation
+2 -2
View File
@@ -219,7 +219,7 @@ TS-only members: `previousResult`?, `onToken`?, `signal`?, `onEvent`?, `options`
### `Agent.chat`
- Python: `src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3657`
- Python: `src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3714`
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:3592`
- Counts: 17 python params: 7 exact, 9 camelCase, 1 alias, 0 flattened, 0 missing; 0 mismatches; 0 waived; 2 TS-only of 19
@@ -316,7 +316,7 @@ TS-only members: `config`
### `Session.__init__`
- Python: `src/praisonai-agents/praisonaiagents/session/api.py:67`
- TypeScript: `src/praisonai-ts/src/session/session.ts:29` (ctor `src/praisonai-ts/src/session/session.ts:132`)
- TypeScript: `src/praisonai-ts/src/session/session.ts:29` (ctor `src/praisonai-ts/src/session/session.ts:133`)
- Counts: 7 python params: 1 exact, 6 camelCase, 0 alias, 0 flattened, 0 missing; 1 mismatches; 1 waived; 5 TS-only of 12
| Python param | Kind | Py default | Py type | Match | TS name | TS default | TS type | Status |
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "praisonai",
"version": "1.7.4",
"description": "PraisonAI TypeScript AI Agents Framework - Node.js, npm, and Javascript AI Agents Framework",
"description": "Open-source TypeScript and JavaScript AI agent framework for Node.js, on hosted or local models. The JavaScript SDK of PraisonAI.",
"main": "dist/index.js",
"module": "dist/esm/index.js",
"types": "dist/index.d.ts",
+2 -2
View File
@@ -1797,7 +1797,7 @@
}
],
"python": {
"location": "src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3657",
"location": "src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3714",
"resolved_class": "ChatMixin"
},
"ts_only": [
@@ -5142,7 +5142,7 @@
],
"ts_only_required": [],
"typescript": {
"ctor_location": "src/praisonai-ts/src/session/session.ts:132",
"ctor_location": "src/praisonai-ts/src/session/session.ts:133",
"location": "src/praisonai-ts/src/session/session.ts:29",
"options_parameters": [
"config"
+20 -5
View File
@@ -72,7 +72,7 @@ export class MemoryCache extends BaseCache {
if (this.maxSize && this.cache.size >= this.maxSize && !this.cache.has(key)) {
// Remove oldest entry
const firstKey = this.cache.keys().next().value;
if (firstKey) this.cache.delete(firstKey);
if (firstKey !== undefined) this.cache.delete(firstKey);
}
const effectiveTtl = ttl ?? this.ttl;
@@ -137,12 +137,15 @@ export class MemoryCache extends BaseCache {
*/
export class FileCache extends BaseCache {
private cacheDir: string;
private ttl?: number;
private fs: any;
private path: any;
private listQueues: Map<string, Promise<void>> = new Map();
constructor(config: CacheConfig & { cacheDir?: string } = {}) {
super(config);
this.cacheDir = config.cacheDir || '.cache';
this.ttl = config.ttl;
}
private async ensureDir(): Promise<void> {
@@ -177,10 +180,11 @@ export class FileCache extends BaseCache {
async set<T = any>(key: string, value: T, ttl?: number): Promise<void> {
await this.ensureDir();
const effectiveTtl = ttl ?? this.ttl;
const entry: CacheEntry<T> = {
value,
createdAt: Date.now(),
expiresAt: ttl ? Date.now() + ttl : undefined
expiresAt: effectiveTtl ? Date.now() + effectiveTtl : undefined
};
await this.fs.writeFile(this.getFilePath(key), JSON.stringify(entry));
}
@@ -227,9 +231,20 @@ export class FileCache extends BaseCache {
}
async listPush<T = any>(key: string, value: T): Promise<void> {
const list = (await this.get<T[]>(`list_${key}`)) || [];
list.push(value);
await this.set(`list_${key}`, list);
// Serialize the full read/modify/write operation for this instance and key.
// A failed append must not block subsequent appends.
const previous = this.listQueues.get(key) ?? Promise.resolve();
const next = previous.catch(() => {}).then(async () => {
const list = (await this.get<T[]>(`list_${key}`)) || [];
list.push(value);
await this.set(`list_${key}`, list);
});
this.listQueues.set(key, next);
try {
await next;
} finally {
if (this.listQueues.get(key) === next) this.listQueues.delete(key);
}
}
async listLength(key: string): Promise<number> {
+12 -7
View File
@@ -231,12 +231,17 @@ export class HooksManager {
return promise;
}
return Promise.race([
promise,
new Promise<T>((_, reject) =>
setTimeout(() => reject(new Error(`Hook timeout after ${timeout}ms`)), timeout)
),
]);
let timer: ReturnType<typeof setTimeout> | undefined;
try {
return await Promise.race([
promise,
new Promise<T>((_, reject) => {
timer = setTimeout(() => reject(new Error(`Hook timeout after ${timeout}ms`)), timeout);
}),
]);
} finally {
if (timer !== undefined) clearTimeout(timer);
}
}
/**
@@ -251,7 +256,7 @@ export class HooksManager {
* Get all registered hooks for an event
*/
getHooks(event: HookEvent): readonly HookConfig[] {
return Object.freeze(this.hooks.get(event) ?? []);
return Object.freeze([...(this.hooks.get(event) ?? [])]);
}
/**
+12 -8
View File
@@ -82,16 +82,20 @@ export class Chunking {
* Chunk by sentences
*/
chunkBySentence(text: string): Chunk[] {
const sentences = text.match(/[^.!?]+[.!?]+/g) || [text];
return sentences.map((content, index) => {
const startOffset = text.indexOf(content);
return {
content: content.trim(),
index,
const chunks: Chunk[] = [];
// Include the unterminated tail and punctuation-only spans as well.
for (const match of text.matchAll(/[^.!?]+[.!?]*|[.!?]+/g)) {
const content = match[0].trim();
if (!content) continue;
const startOffset = match.index + match[0].indexOf(content);
chunks.push({
content,
index: chunks.length,
startOffset,
endOffset: startOffset + content.length
};
});
});
}
return chunks;
}
/**
@@ -60,7 +60,23 @@ export class GraphStore {
* Add an edge to the graph
*/
addEdge(edge: GraphEdge): void {
const previous = this.edges.get(edge.id);
this.edges.set(edge.id, edge);
if (previous && (previous.source !== edge.source || previous.target !== edge.target)) {
// Another edge may still connect the old endpoints.
let hasParallelEdge = false;
for (const candidate of this.edges.values()) {
if (candidate.source === previous.source && candidate.target === previous.target) {
hasParallelEdge = true;
break;
}
}
if (!hasParallelEdge) {
this.adjacencyList.get(previous.source)?.delete(previous.target);
this.reverseAdjacencyList.get(previous.target)?.delete(previous.source);
}
}
// Update adjacency lists
if (!this.adjacencyList.has(edge.source)) {
+27 -12
View File
@@ -70,22 +70,24 @@ export class WebSocketTransport implements MCPTransport {
}
const WebSocket = wsModule.default;
this.ws = new WebSocket(this.url);
const socket = new WebSocket(this.url);
this.ws = socket;
await new Promise<void>((resolve, reject) => {
this.ws.on('open', () => {
socket.on('open', () => {
this.connected = true;
resolve();
});
this.ws.on('error', (err: Error) => reject(err));
socket.on('error', (err: Error) => reject(err));
});
this.ws.on('message', (data: Buffer | string) => {
socket.on('message', (data: Buffer | string) => {
if (this.ws !== socket) return;
try {
const message = JSON.parse(data.toString()) as TransportMessage;
// Handle response
if (message.id && this.pendingRequests.has(String(message.id))) {
if (message.id !== undefined && this.pendingRequests.has(String(message.id))) {
const { resolve, reject } = this.pendingRequests.get(String(message.id))!;
this.pendingRequests.delete(String(message.id));
if (message.error) {
@@ -102,16 +104,29 @@ export class WebSocketTransport implements MCPTransport {
}
});
this.ws.on('error', (err: Error) => this.errorHandler?.(err));
this.ws.on('close', () => { this.connected = false; });
socket.on('error', (err: Error) => {
if (this.ws === socket) this.errorHandler?.(err);
});
socket.on('close', () => {
if (this.ws !== socket) return;
this.connected = false;
this.ws = null;
this.rejectPendingRequests();
});
}
async disconnect(): Promise<void> {
if (this.ws) {
this.ws.close();
this.ws = null;
this.connected = false;
}
const socket = this.ws;
this.ws = null;
this.connected = false;
this.rejectPendingRequests();
socket?.close();
}
private rejectPendingRequests(): void {
const error = new Error('WebSocket connection closed');
for (const { reject } of this.pendingRequests.values()) reject(error);
this.pendingRequests.clear();
}
async send(message: TransportMessage): Promise<TransportMessage> {
+14 -4
View File
@@ -28,6 +28,7 @@ export class FileMemory {
private initialized: boolean = false;
private initPromise?: Promise<void>;
private writeQueue: Promise<void> = Promise.resolve();
private appendNeedsSeparator: boolean = false;
constructor(config: FileMemoryConfig) {
this.filePath = config.filePath;
@@ -67,6 +68,8 @@ export class FileMemory {
try {
const fs = await import('fs/promises');
const content = await fs.readFile(this.filePath, 'utf-8').catch(() => '');
// A previous instance may have left a record without its final newline.
this.appendNeedsSeparator = content.length > 0 && !content.endsWith('\n');
if (content) {
const lines = content.trim().split('\n').filter(Boolean);
@@ -155,6 +158,7 @@ export class FileMemory {
*/
async getRecent(count: number): Promise<MemoryEntry[]> {
const all = await this.getAll();
if (count === 0) return [];
return all.slice(-count);
}
@@ -265,7 +269,7 @@ export class FileMemory {
* Build context string from recent memory
*/
async buildContext(count?: number): Promise<string> {
const entries = count ? await this.getRecent(count) : await this.getAll();
const entries = count !== undefined ? await this.getRecent(count) : await this.getAll();
return entries
.map(e => `${e.role}: ${e.content}`)
.join('\n');
@@ -273,11 +277,17 @@ export class FileMemory {
private async appendEntry(entry: FileMemoryEntry): Promise<void> {
// Queue writes to ensure order
this.writeQueue = this.writeQueue.then(async () => {
const write = this.writeQueue.then(async () => {
const fs = await import('fs/promises');
await fs.appendFile(this.filePath, JSON.stringify(entry) + '\n');
// A failed append may have left an unfinished JSONL line.
const separator = this.appendNeedsSeparator ? '\n' : '';
this.appendNeedsSeparator = true;
await fs.appendFile(this.filePath, separator + JSON.stringify(entry) + '\n');
this.appendNeedsSeparator = false;
});
await this.writeQueue;
// Report this write's error to its caller without blocking later writes.
this.writeQueue = write.catch(() => {});
await write;
}
private textSearch(query: string, limit: number): SearchResult[] {
+2 -1
View File
@@ -87,6 +87,7 @@ export class Memory {
* Get recent entries
*/
getRecent(count: number): MemoryEntry[] {
if (count === 0) return [];
return this.getAll().slice(-count);
}
@@ -179,7 +180,7 @@ export class Memory {
* Build context string from recent memory
*/
buildContext(count?: number): string {
const entries = count ? this.getRecent(count) : this.getAll();
const entries = count !== undefined ? this.getRecent(count) : this.getAll();
return entries
.map(e => `${e.role}: ${e.content}`)
.join('\n');
+7 -6
View File
@@ -123,7 +123,8 @@ export class Session {
private readonly timeout: number;
private readonly isRemote: boolean;
private state: Record<string, any> = {};
// State keys are user data, including names inherited by ordinary objects.
private state: Record<string, any> = Object.create(null);
private messages: SessionMessage[] = [];
private createdAt: number;
private updatedAt: number;
@@ -264,7 +265,7 @@ export class Session {
* Set entire state object.
*/
setState(state: Record<string, any>): void {
this.state = { ...state };
this.state = Object.assign(Object.create(null), state);
this.updatedAt = Date.now();
}
@@ -272,7 +273,7 @@ export class Session {
* Clear all state.
*/
clearState(): void {
this.state = {};
this.state = Object.create(null);
this.updatedAt = Date.now();
}
@@ -364,7 +365,7 @@ export class Session {
const messages = await this.db.getMessages(this.id, 1);
if (messages.length > 0) {
const sessionState: SessionState = JSON.parse(messages[0].content || '{}');
this.state = sessionState.data || {};
this.setState(sessionState.data || {});
this.messages = sessionState.messages || [];
this.createdAt = sessionState.createdAt || Date.now();
this.updatedAt = sessionState.updatedAt || Date.now();
@@ -382,7 +383,7 @@ export class Session {
return {
id: this.id,
userId: this.userId,
data: this.state,
data: this.getState(),
messages: this.messages,
createdAt: this.createdAt,
updatedAt: this.updatedAt,
@@ -399,7 +400,7 @@ export class Session {
userId: json.userId,
...config,
});
session.state = json.data || {};
session.setState(json.data || {});
session.messages = json.messages || [];
session.createdAt = json.createdAt;
session.updatedAt = json.updatedAt;
+24 -5
View File
@@ -111,6 +111,8 @@ export class FileSessionStore implements ISessionStore {
private autoSaveInterval?: number;
private saveTimer?: NodeJS.Timeout;
private dirty: boolean = false;
private saveQueue: Promise<void> = Promise.resolve();
private pendingSaves: number = 0;
constructor(filePath: string, config?: { defaultTTL?: number; autoSaveInterval?: number }) {
this.filePath = filePath;
@@ -120,7 +122,7 @@ export class FileSessionStore implements ISessionStore {
if (this.autoSaveInterval) {
this.saveTimer = setInterval(() => {
if (this.dirty) this.save();
if (this.dirty && this.pendingSaves === 0) this.save();
}, this.autoSaveInterval);
}
}
@@ -146,10 +148,26 @@ export class FileSessionStore implements ISessionStore {
}
async save(): Promise<void> {
const fs = await import('fs').then(m => m.promises);
const sessions = Array.from(this.sessions.values());
await fs.writeFile(this.filePath, JSON.stringify(sessions, null, 2));
this.dirty = false;
this.pendingSaves++;
const saving = this.saveQueue.then(async () => {
const fs = await import('fs').then(m => m.promises);
const data = JSON.stringify(Array.from(this.sessions.values()), null, 2);
// Only this snapshot is clean; mutations during the write stay dirty.
this.dirty = false;
try {
await fs.writeFile(this.filePath, data);
} catch (error) {
this.dirty = true;
throw error;
}
});
// Keep the queue usable after a failure, while this caller still sees it.
this.saveQueue = saving.catch(() => {});
try {
await saving;
} finally {
this.pendingSaves--;
}
}
async get(id: string): Promise<StoredSession | null> {
@@ -205,6 +223,7 @@ export class FileSessionStore implements ISessionStore {
if (this.saveTimer) {
clearInterval(this.saveTimer);
}
await this.saveQueue;
if (this.dirty) {
await this.save();
}
+15 -1
View File
@@ -420,7 +420,21 @@ export class FunctionTool<TParams = any, TResult = any> {
}
const delay = retryDelayMs(policy, attempt - 1);
if (delay > 0) {
await new Promise<void>(resolve => setTimeout(resolve, delay));
const signal = context?.signal;
const completed = await new Promise<boolean>(resolve => {
let timer: ReturnType<typeof setTimeout> | undefined;
const finish = (value: boolean) => {
if (timer !== undefined) clearTimeout(timer);
signal?.removeEventListener('abort', onAbort);
resolve(value);
};
const onAbort = () => finish(false);
timer = setTimeout(() => finish(true), delay);
signal?.addEventListener('abort', onAbort, { once: true });
if (signal?.aborted) finish(false);
});
// Preserve the existing last-error behavior when cancellation stops retries.
if (!completed || signal?.aborted) throw error;
}
}
}
@@ -147,6 +147,9 @@ export function createRetryMiddleware(
let lastError: Error | undefined;
for (let attempt = 0; attempt <= maxRetries; attempt++) {
if (context.signal?.aborted) {
throw context.signal.reason ?? new Error('The operation was aborted');
}
try {
return await next();
} catch (error) {
@@ -212,7 +215,9 @@ export function createValidationMiddleware(): ToolMiddleware {
const limits = context.limits;
if (limits?.maxPayloadBytes) {
const size = JSON.stringify(input).length;
const serialized = JSON.stringify(input);
if (serialized === undefined) throw new TypeError('Input payload is not JSON serializable');
const size = new TextEncoder().encode(serialized).byteLength;
if (size > limits.maxPayloadBytes) {
throw new Error(`Input payload size (${size} bytes) exceeds limit (${limits.maxPayloadBytes} bytes)`);
}
@@ -227,17 +232,16 @@ export function createValidationMiddleware(): ToolMiddleware {
*/
export function composeMiddleware(...middlewares: ToolMiddleware[]): ToolMiddleware {
return async (input, context, next) => {
let index = 0;
const composedNext = async (): Promise<unknown> => {
// Each continuation keeps its own downstream position so retry middleware
// can invoke next() again without skipping the remaining layers.
const dispatch = async (index: number): Promise<unknown> => {
if (index < middlewares.length) {
const mw = middlewares[index++];
return mw(input, context, composedNext);
return middlewares[index](input, context, () => dispatch(index + 1));
}
return next();
};
return composedNext();
return dispatch(0);
};
}
+18 -25
View File
@@ -5,6 +5,8 @@
* Supports lazy loading, middleware, and multi-agent safety.
*/
import { composeMiddleware } from './middleware';
import type {
ToolMetadata,
ToolFactory,
@@ -61,6 +63,7 @@ export class ToolsRegistry {
*/
register(metadata: ToolMetadata, factory: ToolFactory): this {
this.tools.set(metadata.id, { metadata, factory });
this.installCache.delete(metadata.id);
return this;
}
@@ -68,6 +71,7 @@ export class ToolsRegistry {
* Unregister a tool
*/
unregister(id: string): boolean {
this.installCache.delete(id);
return this.tools.delete(id);
}
@@ -158,16 +162,7 @@ export class ToolsRegistry {
if (this.middleware.length === 0) {
result = await executeWithMiddleware();
} else {
// Chain middleware
let index = 0;
const next = async (): Promise<unknown> => {
if (index < this.middleware.length) {
const mw = this.middleware[index++];
return mw(input, ctx, next);
}
return executeWithMiddleware();
};
result = await next();
result = await composeMiddleware(...this.middleware)(input, ctx, executeWithMiddleware);
}
// Call afterToolCall hook
@@ -222,27 +217,25 @@ export class ToolsRegistry {
* Check if a tool's optional dependency is installed
*/
async checkInstalled(id: string): Promise<boolean> {
// Check cache first
const registered = this.tools.get(id);
if (!registered) return false;
if (this.installCache.has(id)) {
return this.installCache.get(id)!;
}
const registered = this.tools.get(id);
if (!registered) {
return false;
const { packageName } = registered.metadata;
let installed: boolean;
try {
await import(packageName);
installed = true;
} catch {
installed = false;
}
const { packageName } = registered.metadata;
try {
// Try to resolve the package
await import(packageName);
this.installCache.set(id, true);
return true;
} catch {
this.installCache.set(id, false);
return false;
}
// An earlier import must not repopulate the cache after replacement/removal.
if (this.tools.get(id) === registered) this.installCache.set(id, installed);
return installed;
}
/**
+86
View File
@@ -3,6 +3,9 @@
*/
import { MemoryCache, FileCache, createMemoryCache, createFileCache } from '../../../src/cache';
import { mkdtemp, rm } from 'fs/promises';
import { tmpdir } from 'os';
import { join } from 'path';
describe('MemoryCache', () => {
let cache: MemoryCache;
@@ -158,3 +161,86 @@ describe('Factory functions', () => {
expect(cache).toBeInstanceOf(FileCache);
});
});
describe('FileCache default TTL', () => {
let cacheDir: string;
let now: number;
beforeEach(async () => {
cacheDir = await mkdtemp(join(tmpdir(), 'praisonai-file-cache-ttl-'));
now = 1000;
jest.spyOn(Date, 'now').mockImplementation(() => now);
});
afterEach(async () => {
jest.restoreAllMocks();
await rm(cacheDir, { recursive: true, force: true });
});
it.each(['get', 'has'] as const)('honors the configured TTL through %s after reopening', async (method) => {
const cache = createFileCache({ cacheDir, ttl: 50 });
await cache.set('answer', 'old answer');
expect(await cache.get('answer')).toBe('old answer');
const reopened = new FileCache({ cacheDir });
now += 51;
if (method === 'get') {
expect(await reopened.get('answer')).toBeUndefined();
} else {
expect(await reopened.has('answer')).toBe(false);
}
expect(await reopened.keys()).not.toContain('answer');
});
it('allows a per-entry TTL to override the configured TTL', async () => {
const cache = new FileCache({ cacheDir, ttl: 50 });
await cache.set('answer', 'value', 100);
now += 51;
expect(await cache.get('answer')).toBe('value');
now += 50;
expect(await cache.get('answer')).toBeUndefined();
});
it('allows an explicit zero TTL to disable the configured expiry', async () => {
const cache = new FileCache({ cacheDir, ttl: 50 });
await cache.set('answer', 'value', 0);
now += 1000;
expect(await cache.get('answer')).toBe('value');
});
it('keeps entries without a default or explicit TTL', async () => {
const cache = new FileCache({ cacheDir });
await cache.set('answer', 'value');
now += 1000;
expect(await cache.get('answer')).toBe('value');
});
});
describe('MemoryCache capacity with empty keys', () => {
it.each([1, 2])('keeps capacity %i bounded when the oldest key is empty', async (maxSize) => {
const cache = new MemoryCache({ maxSize });
await cache.set('', 'oldest');
for (let i = 0; i <= maxSize; i++) {
await cache.set(`key-${i}`, i);
}
expect(await cache.size()).toBe(maxSize);
expect(await cache.has('')).toBe(false);
expect(await cache.keys()).toEqual(
Array.from({ length: maxSize }, (_, i) => `key-${i + 1}`)
);
expect(await cache.get(`key-${maxSize}`)).toBe(maxSize);
});
it('updates an existing empty key without evicting another entry', async () => {
const cache = new MemoryCache({ maxSize: 2 });
await cache.set('', 'old value');
await cache.set('other', 'retained');
await cache.set('', 'new value');
expect(await cache.size()).toBe(2);
expect(await cache.keys()).toEqual(['', 'other']);
expect(await cache.get('')).toBe('new value');
expect(await cache.get('other')).toBe('retained');
});
});
@@ -0,0 +1,70 @@
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
import { mkdtemp, mkdir, rm } from 'fs/promises';
import { tmpdir } from 'os';
import { join } from 'path';
import { createFileCache } from '../../../src/cache';
describe('FileCache concurrent list appends', () => {
let directory: string;
beforeEach(async () => { directory = await mkdtemp(join(tmpdir(), 'praison-list-')); });
afterEach(async () => { await rm(directory, { recursive: true, force: true }); });
it('persists every successful concurrent append in call order', async () => {
const cache = createFileCache({ cacheDir: directory });
await cache.listPush('events', 'existing');
const values = Array.from({ length: 25 }, (_, index) => `event-${index}`);
await Promise.all(values.map(value => cache.listPush('events', value)));
const reopened = createFileCache({ cacheDir: directory });
expect(await reopened.listRange('events', 0)).toEqual(['existing', ...values]);
expect(await reopened.listLength('events')).toBe(26);
});
it('retains each list when different keys receive concurrent appends', async () => {
const cache = createFileCache({ cacheDir: directory });
await cache.keys();
const values = Array.from({ length: 12 }, (_, index) => index + 1);
await Promise.all(values.flatMap(value => [cache.listPush('left', value), cache.listPush('right', -value)]));
const reopened = createFileCache({ cacheDir: directory });
expect(await reopened.listRange('left', 0)).toEqual(values);
expect(await reopened.listRange('right', 0)).toEqual(values.map(value => -value));
});
it('recovers after a failed append without persisting that value', async () => {
const cache = createFileCache({ cacheDir: directory });
const blockedFile = join(directory, 'list_events.json');
await mkdir(blockedFile);
await expect(cache.listPush('events', 'failed')).rejects.toThrow();
await rm(blockedFile, { recursive: true });
await cache.listPush('events', 'retry');
expect(await cache.listRange('events', 0)).toEqual(['retry']);
});
it('continues an already queued append after a transient write failure', async () => {
const cache = createFileCache({ cacheDir: directory });
let releaseFailure!: () => void;
let signalWrite!: () => void;
const blocked = new Promise<void>(resolve => { releaseFailure = resolve; });
const writing = new Promise<void>(resolve => { signalWrite = resolve; });
// Inject one failed public write; the queued retry uses the real filesystem.
jest.spyOn(cache, 'set').mockImplementationOnce(async () => {
signalWrite();
await blocked;
throw new Error('transient write failure');
});
const failed = expect(cache.listPush('events', 'failed')).rejects.toThrow('transient write failure');
const retry = cache.listPush('events', 'retry');
await writing;
releaseFailure();
await failed;
await retry;
const reopened = createFileCache({ cacheDir: directory });
expect(await reopened.listRange('events', 0)).toEqual(['retry']);
});
it('preserves sequential appends and list slicing', async () => {
const cache = createFileCache({ cacheDir: directory });
for (const value of [1, 2, 3]) await cache.listPush('events', value);
expect(await cache.listRange('events', 1, 2)).toEqual([2]);
expect(await cache.listRange('events', -1)).toEqual([3]);
});
});
@@ -0,0 +1,107 @@
import { promises as fs } from 'fs';
import * as os from 'os';
import * as path from 'path';
import { FileMemory } from '../../src/memory/file-memory';
describe('FileMemory write recovery', () => {
let tempDir: string;
beforeEach(async () => {
tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'praisonai-write-recovery-'));
});
afterEach(async () => {
jest.restoreAllMocks();
await fs.rm(tempDir, { recursive: true, force: true });
});
test('persists later writes after the filesystem becomes available again', async () => {
const directory = path.join(tempDir, 'store');
const unavailableDirectory = path.join(tempDir, 'unavailable');
await fs.mkdir(directory);
const filePath = path.join(directory, 'memory.jsonl');
const memory = new FileMemory({ filePath });
const initial = await memory.add('Before interruption', 'user');
await fs.rename(directory, unavailableDirectory);
await expect(memory.add('During interruption', 'user')).rejects.toMatchObject({ code: 'ENOENT' });
await fs.rename(unavailableDirectory, directory);
const recovered = await memory.add('After recovery', 'assistant');
const reopened = new FileMemory({ filePath });
expect(await reopened.getAll()).toEqual([initial, recovered]);
});
test('preserves each write result and continues queued writes after a rejection', async () => {
const filePath = path.join(tempDir, 'memory.jsonl');
const memory = new FileMemory({ filePath });
const initial = await memory.add('Before interruption', 'user');
const appendFile = fs.appendFile;
const failure = new Error('Temporary append failure');
let releaseFailure!: () => void;
let notifyAppend!: () => void;
const blocked = new Promise<void>(resolve => { releaseFailure = resolve; });
const started = new Promise<void>(resolve => { notifyAppend = resolve; });
jest.spyOn(fs, 'appendFile').mockImplementationOnce(async () => {
notifyAppend();
await blocked;
throw failure;
}).mockImplementation(appendFile);
const failed = memory.add('Failed write', 'user');
const queued = memory.add('Queued write', 'assistant');
const settled = Promise.allSettled([failed, queued]);
await started;
releaseFailure();
const [first, second] = await settled;
expect(first).toEqual({ status: 'rejected', reason: failure });
expect(second.status).toBe('fulfilled');
if (second.status !== 'fulfilled') throw second.reason;
const later = await memory.add('Later write', 'user');
const reopened = new FileMemory({ filePath });
expect(await reopened.getAll()).toEqual([initial, second.value, later]);
});
test.each([1, 2])('preserves later records after %i partial append failures', async (failures) => {
const filePath = path.join(tempDir, 'memory.jsonl');
const memory = new FileMemory({ filePath });
const initial = await memory.add('Before interruption', 'user');
const appendFile = fs.appendFile;
const failure = Object.assign(new Error('Disk full after a partial write'), { code: 'ENOSPC' });
let attempts = 0;
jest.spyOn(fs, 'appendFile').mockImplementation(async (file, data, options) => {
if (attempts++ < failures) {
// Write an actual unterminated JSON prefix before reporting failure.
await appendFile(file, String(data).slice(0, 20), options);
throw failure;
}
await appendFile(file, data, options);
});
for (let attempt = 0; attempt < failures; attempt++) {
await expect(memory.add('Interrupted write', 'user')).rejects.toBe(failure);
}
const recovered = await memory.add('Recovered write', 'assistant');
const later = await memory.add('Later write', 'user');
const reopened = new FileMemory({ filePath });
expect(await reopened.getAll()).toEqual([initial, recovered, later]);
});
test.each([
['incomplete JSON', false],
['complete JSON', true],
] as const)('preserves new records after loading an unterminated %s line', async (_, complete) => {
const filePath = path.join(tempDir, 'memory.jsonl');
const original = new FileMemory({ filePath });
const initial = await original.add('Before restart', 'user');
const last = { id: 'last-record', content: 'Last record', role: 'user', timestamp: initial.timestamp };
const tail = complete ? JSON.stringify(last) : '{"id":"interrupted","content":';
await fs.appendFile(filePath, tail);
const restarted = new FileMemory({ filePath });
const recovered = await restarted.add('After restart', 'assistant');
const reopened = new FileMemory({ filePath });
expect(await reopened.getAll()).toEqual(complete ? [initial, last, recovered] : [initial, recovered]);
});
});
@@ -0,0 +1,47 @@
import { describe, expect, it } from '@jest/globals';
import { HooksManager } from '../../src/hooks/manager';
describe('HooksManager inspection', () => {
it('allows registering and executing hooks after inspection', async () => {
const manager = new HooksManager();
const called: string[] = [];
const first = manager.register('pre_tool_call', () => { called.push('first'); });
const snapshot = manager.getHooks('pre_tool_call');
const second = manager.register('pre_tool_call', () => { called.push('second'); }, { priority: 1 });
expect(snapshot.map(hook => hook.id)).toEqual([first]);
expect(manager.getHooks('pre_tool_call').map(hook => hook.id)).toEqual([second, first]);
expect((await manager.execute('pre_tool_call', {})).success).toBe(true);
expect(called).toEqual(['second', 'first']);
});
it('allows unregistering a hook after inspection', async () => {
const manager = new HooksManager();
const called: string[] = [];
const first = manager.register('pre_tool_call', () => { called.push('first'); });
const second = manager.register('pre_tool_call', () => { called.push('second'); });
const snapshot = manager.getHooks('pre_tool_call');
expect(manager.unregister(first)).toBe(true);
expect(snapshot.map(hook => hook.id)).toEqual([first, second]);
expect(manager.getStats().totalHooks).toBe(1);
await manager.execute('pre_tool_call', {});
expect(called).toEqual(['second']);
});
it('keeps returned arrays frozen against caller mutations', () => {
const manager = new HooksManager();
const id = manager.register('pre_tool_call', () => {});
const snapshot = manager.getHooks('pre_tool_call');
expect(Object.isFrozen(snapshot)).toBe(true);
expect(() => (snapshot as unknown[]).pop()).toThrow(TypeError);
expect(manager.getHooks('pre_tool_call').map(hook => hook.id)).toEqual([id]);
});
it('allows first registration after inspecting an empty event', () => {
const manager = new HooksManager();
const snapshot = manager.getHooks('pre_tool_call');
const id = manager.register('pre_tool_call', () => {});
expect(snapshot).toEqual([]);
expect(Object.isFrozen(snapshot)).toBe(true);
expect(manager.getHooks('pre_tool_call').map(hook => hook.id)).toEqual([id]);
});
});
@@ -0,0 +1,59 @@
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
import { spawnSync } from 'child_process';
import { resolve } from 'path';
import { HooksManager } from '../../src/hooks/manager';
describe('HooksManager timeout lifecycle', () => {
beforeEach(() => { jest.useFakeTimers(); });
afterEach(() => { jest.clearAllTimers(); jest.useRealTimers(); });
it('releases the deadline after a successful asynchronous hook', async () => {
const hooks = new HooksManager({ timeout: 30000 });
hooks.register('pre_tool_call', async (context) => ({ ...context, checked: true }));
const result = await hooks.execute('pre_tool_call', { input: 'test' });
expect(result.modifiedContext).toEqual({ input: 'test', checked: true });
expect(result.blocked).toBe(false);
expect(jest.getTimerCount()).toBe(0);
});
it('releases the deadline after an asynchronous hook rejects', async () => {
const hooks = new HooksManager({ timeout: 30000 });
hooks.register('pre_tool_call', async () => { throw new Error('validation failed'); }, { blockOnFailure: true });
const result = await hooks.execute('pre_tool_call', {});
expect(result).toMatchObject({ success: false, blocked: true, error: 'validation failed' });
expect(jest.getTimerCount()).toBe(0);
});
it('still times out a pending hook and preserves blocking policy', async () => {
const hooks = new HooksManager({ timeout: 10 });
hooks.register('pre_tool_call', () => new Promise(() => {}), { blockOnFailure: true });
const execution = hooks.execute('pre_tool_call', {});
expect(jest.getTimerCount()).toBe(1);
await jest.advanceTimersByTimeAsync(10);
expect(await execution).toMatchObject({ success: false, blocked: true, error: 'Hook timeout after 10ms' });
expect(jest.getTimerCount()).toBe(0);
});
it('does not allocate a deadline for a synchronous hook', async () => {
const hooks = new HooksManager();
hooks.register('pre_tool_call', () => ({ checked: true }));
expect((await hooks.execute('pre_tool_call', {})).modifiedContext).toEqual({ checked: true });
expect(jest.getTimerCount()).toBe(0);
});
it('lets a real process exit naturally after its async hook settles', () => {
const script = `
const { HooksManager } = require(${JSON.stringify(resolve(__dirname, '../../src/hooks/manager'))});
const hooks = new HooksManager({ timeout: 30000 });
hooks.register('pre_tool_call', async context => context);
hooks.execute('pre_tool_call', {}).then(() => console.log('hook settled')).catch(error => { console.error(error); process.exitCode = 1; });
`;
const result = spawnSync(process.execPath, ['-r', 'ts-node/register/transpile-only', '-e', script], {
timeout: 10000,
encoding: 'utf-8',
});
expect(result.stdout).toContain('hook settled');
expect(result.error).toBeUndefined();
expect({ status: result.status, stderr: result.stderr }).toEqual({ status: 0, stderr: '' });
}, 15000);
});
@@ -30,6 +30,34 @@ describe('Chunking', () => {
expect(chunks.length).toBe(3);
});
it.each([
'First sentence. Final important fact without punctuation',
'Echo.Echo.Echo.',
' \nFirst. Second! \nTail \t',
'... Leading punctuation. Tail',
'No punctuation here',
'Really?! Next.',
'你好。完整段落',
])('preserves sentence content and its source span: %j', async text => {
const { Chunking } = await import('../../../src/knowledge/chunking');
const chunks = new Chunking({ strategy: 'sentence' }).chunk(text);
let cursor = 0;
for (const [index, chunk] of chunks.entries()) {
expect(chunk.index).toBe(index);
expect(chunk.content.length).toBeGreaterThan(0);
expect(chunk.startOffset).toBeGreaterThanOrEqual(cursor);
expect(text.slice(cursor, chunk.startOffset).trim()).toBe('');
expect(text.slice(chunk.startOffset, chunk.endOffset)).toBe(chunk.content);
cursor = chunk.endOffset;
}
expect(text.slice(cursor).trim()).toBe('');
});
it.each(['', ' \n\t '])('does not emit empty sentence chunks for %j', async text => {
const { Chunking } = await import('../../../src/knowledge/chunking');
expect(new Chunking({ strategy: 'sentence' }).chunk(text)).toEqual([]);
});
it('should chunk by paragraphs', async () => {
const { Chunking } = await import('../../../src/knowledge/chunking');
const chunker = new Chunking({ strategy: 'paragraph' });
@@ -0,0 +1,47 @@
import { describe, expect, it } from '@jest/globals';
import { GraphStore } from '../../../src/knowledge/graph-rag';
function graph(): GraphStore {
const store = new GraphStore();
for (const id of ['a', 'b', 'c', 'd']) store.addNode({ id, type: 'document', content: id });
return store;
}
describe('GraphStore edge replacement', () => {
it.each([
['a', 'c'], ['c', 'b'], ['c', 'd'], ['a', 'a'],
])('removes obsolete adjacency when an edge moves to %s -> %s', (source, target) => {
const store = graph();
store.addEdge({ id: 'edge', source: 'a', target: 'b', type: 'link' });
store.addEdge({ id: 'edge', source, target, type: 'link' });
expect(store.getNeighbors('a', 'outgoing').map(n => n.id)).not.toContain('b');
expect(store.getNeighbors('b', 'incoming').map(n => n.id)).not.toContain('a');
expect(store.getEdgesBetween('a', 'b')).toEqual([]);
expect(store.findPaths('a', 'b')).toEqual([]);
expect(store.getNeighbors(source, 'outgoing').map(n => n.id)).toContain(target);
expect(store.getNeighbors(target, 'incoming').map(n => n.id)).toContain(source);
expect(store.getStats().edgeCount).toBe(1);
});
it('retains adjacency backed by a parallel edge', () => {
const store = graph();
store.addEdge({ id: 'one', source: 'a', target: 'b', type: 'one' });
store.addEdge({ id: 'two', source: 'a', target: 'b', type: 'two' });
store.addEdge({ id: 'one', source: 'c', target: 'd', type: 'one' });
expect(store.getNeighbors('a', 'outgoing').map(n => n.id)).toEqual(['b']);
expect(store.getNeighbors('b', 'incoming').map(n => n.id)).toEqual(['a']);
expect(store.getEdgesBetween('a', 'b').map(e => e.id)).toEqual(['two']);
store.addEdge({ id: 'two', source: 'c', target: 'd', type: 'two' });
expect(store.getNeighbors('a', 'outgoing')).toEqual([]);
expect(store.getNeighbors('b', 'incoming')).toEqual([]);
expect(store.getStats().edgeCount).toBe(2);
});
it('preserves neighbors when only edge metadata changes', () => {
const store = graph();
store.addEdge({ id: 'edge', source: 'a', target: 'b', type: 'link', weight: 1 });
store.addEdge({ id: 'edge', source: 'a', target: 'b', type: 'link', weight: 2 });
expect(store.getNeighbors('a', 'outgoing').map(n => n.id)).toEqual(['b']);
expect(store.getEdgesBetween('a', 'b')[0].weight).toBe(2);
});
});
@@ -0,0 +1,93 @@
import { EventEmitter } from 'events';
import { WebSocketTransport } from '../../src/mcp/transports';
class MockWebSocket extends EventEmitter {
static last: MockWebSocket;
send = jest.fn();
// Real close handshakes finish asynchronously; tests emit close explicitly.
close = jest.fn();
constructor(_url: string) {
super();
MockWebSocket.last = this;
queueMicrotask(() => this.emit('open'));
}
}
jest.mock('ws', () => ({ __esModule: true, default: MockWebSocket }), { virtual: true });
async function boundedOutcome(pending: Promise<unknown>): Promise<unknown> {
let timer: ReturnType<typeof setTimeout> | undefined;
try {
return await Promise.race([
pending.then(() => 'resolved', error => error),
new Promise(resolve => { timer = setTimeout(() => resolve('pending'), 100); }),
]);
} finally {
if (timer) clearTimeout(timer);
}
}
const response = (source: string) => JSON.stringify({ jsonrpc: '2.0', id: 1, result: { source } });
describe('WebSocketTransport connection lifecycle', () => {
let transport: WebSocketTransport;
beforeEach(async () => {
transport = new WebSocketTransport('ws://localhost');
await transport.connect();
});
afterEach(async () => {
await transport.disconnect();
});
it.each(['remote', 'local'])('rejects every pending request on %s close', async (mode) => {
const outcomes = [1, 2].map(id => boundedOutcome(
transport.send({ jsonrpc: '2.0', id, method: 'tools/list' })
));
if (mode === 'remote') MockWebSocket.last.emit('close');
else await transport.disconnect();
expect(transport.isConnected()).toBe(false);
for (const outcome of await Promise.all(outcomes)) {
expect(outcome).toBeInstanceOf(Error);
expect(outcome).toEqual(expect.objectContaining({
message: expect.stringMatching(/closed|disconnect/i),
}));
}
});
it('keeps a replacement connection active when the old close event arrives', async () => {
const old = MockWebSocket.last;
await transport.disconnect();
await transport.connect();
old.emit('close');
expect(transport.isConnected()).toBe(true);
const pending = transport.send({ jsonrpc: '2.0', id: 1, method: 'tools/list' });
MockWebSocket.last.emit('message', response('new'));
await expect(pending).resolves.toMatchObject({ result: { source: 'new' } });
});
it('ignores a closing connection response when a new request reuses its ID', async () => {
const old = MockWebSocket.last;
await transport.disconnect();
await transport.connect();
const received = jest.fn();
transport.onMessage(received);
const pending = transport.send({ jsonrpc: '2.0', id: 1, method: 'tools/list' });
old.emit('message', response('old'));
MockWebSocket.last.emit('message', response('new'));
await expect(pending).resolves.toMatchObject({ result: { source: 'new' } });
expect(received).toHaveBeenCalledTimes(1);
});
it('keeps a completed response available after its connection closes', async () => {
const pending = transport.send({ jsonrpc: '2.0', id: 1, method: 'tools/list' });
MockWebSocket.last.emit('message', response('complete'));
MockWebSocket.last.emit('close');
await expect(pending).resolves.toMatchObject({ result: { source: 'complete' } });
});
});
@@ -0,0 +1,55 @@
import { EventEmitter } from 'events';
import { WebSocketTransport } from '../../src/mcp/transports';
class MockWebSocket extends EventEmitter {
static last: MockWebSocket;
send = jest.fn();
constructor(_url: string) {
super();
MockWebSocket.last = this;
queueMicrotask(() => this.emit('open'));
}
close(): void {
this.emit('close');
}
}
jest.mock('ws', () => ({ __esModule: true, default: MockWebSocket }), { virtual: true });
describe('WebSocketTransport response correlation', () => {
let transport: WebSocketTransport;
beforeEach(async () => {
transport = new WebSocketTransport('ws://localhost');
await transport.connect();
});
afterEach(async () => {
await transport.disconnect();
});
it.each([0, 1, 'request-1'])('resolves the request with id %p', async (id) => {
const response = { jsonrpc: '2.0' as const, id, result: { tools: [] } };
const received = jest.fn();
transport.onMessage(received);
const pending = transport.send({ jsonrpc: '2.0', id, method: 'tools/list' });
MockWebSocket.last.emit('message', Buffer.from(JSON.stringify(response)));
await expect(pending).resolves.toEqual(response);
expect(received).toHaveBeenCalledWith(response);
});
it('rejects an error response with id zero', async () => {
const pending = transport.send({ jsonrpc: '2.0', id: 0, method: 'tools/list' });
const rejected = expect(pending).rejects.toThrow('Tool listing failed');
MockWebSocket.last.emit('message', JSON.stringify({
jsonrpc: '2.0', id: 0, error: { code: -32603, message: 'Tool listing failed' },
}));
await rejected;
});
});
@@ -13,6 +13,31 @@ import {
import { promises as fs } from 'fs';
import * as path from 'path';
import * as os from 'os';
import { Memory } from '../../src/memory/memory';
describe('Memory history limits', () => {
test('an explicit zero returns no recent entries', async () => {
const memory = new Memory();
const first = await memory.add('First message', 'user');
const second = await memory.add('Second message', 'assistant');
expect(memory.getRecent(0)).toEqual([]);
expect(memory.getRecent(1)).toEqual([second]);
expect(memory.getRecent(3)).toEqual([first, second]);
expect(memory.getAll()).toEqual([first, second]);
});
test('zero context entries differs from an omitted limit', async () => {
const memory = new Memory();
await memory.add('First message', 'user');
await memory.add('Second message', 'assistant');
expect(memory.buildContext(0)).toBe('');
expect(memory.buildContext(1)).toBe('assistant: Second message');
expect(memory.buildContext()).toBe('user: First message\nassistant: Second message');
expect(memory.size).toBe(2);
});
});
describe('FileMemory', () => {
let tempDir: string;
@@ -76,6 +101,20 @@ describe('FileMemory', () => {
expect(recent[1].content).toBe('Message 3');
});
test('zero history limits survive a file-memory reload', async () => {
const memory = createFileMemory({ filePath: testFilePath });
const first = await memory.add('First message', 'user');
const second = await memory.add('Second message', 'assistant');
const reloaded = createFileMemory({ filePath: testFilePath });
expect(await reloaded.getRecent(0)).toEqual([]);
expect(await reloaded.buildContext(0)).toBe('');
expect(await reloaded.getRecent(1)).toEqual([second]);
expect(await reloaded.buildContext(1)).toBe('assistant: Second message');
expect(await reloaded.buildContext()).toBe('user: First message\nassistant: Second message');
expect(await reloaded.getAll()).toEqual([first, second]);
});
test('delete removes entry', async () => {
const memory = createFileMemory({ filePath: testFilePath });
const entry = await memory.add('Hello world', 'user');
@@ -268,6 +307,18 @@ describe('FileMemory', () => {
});
describe('AutoMemory', () => {
test('zero recent entries does not return stored automatic memories', async () => {
const memory = createAutoMemory({
policies: [{ name: 'store-all', condition: () => true, action: 'store', priority: 1000 }]
});
const first = await memory.add('First message', 'user');
const second = await memory.add('Second message', 'assistant');
expect(memory.getRecent(0)).toEqual([]);
expect(memory.getRecent(1)).toEqual([second]);
expect(memory.getAll()).toEqual([first, second]);
});
test('createAutoMemory creates instance', () => {
const memory = createAutoMemory();
expect(memory).toBeInstanceOf(AutoMemory);
@@ -0,0 +1,138 @@
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
import { promises as fs } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
import { FileSessionStore, createStoredSession } from '../../../src/session/store';
function gate(): { promise: Promise<void>; release: () => void } {
let release!: () => void;
const promise = new Promise<void>(resolve => { release = resolve; });
return { promise, release };
}
describe('FileSessionStore save ordering', () => {
let directory: string;
let filePath: string;
beforeEach(async () => {
directory = await fs.mkdtemp(join(tmpdir(), 'praison-session-save-'));
filePath = join(directory, 'sessions.json');
});
afterEach(async () => {
jest.restoreAllMocks();
jest.useRealTimers();
await fs.rm(directory, { recursive: true, force: true });
});
function pauseFirstWrite() {
const entered = gate();
const blocked = gate();
const writeFile = fs.writeFile.bind(fs);
let writes = 0;
jest.spyOn(fs, 'writeFile').mockImplementation(async (...args: Parameters<typeof fs.writeFile>) => {
if (++writes === 1) {
entered.release();
await blocked.promise;
}
return writeFile(...args);
});
return { entered: entered.promise, release: blocked.release, writes: () => writes };
}
it.each(['set', 'delete', 'clear'] as const)('persists %s during an in-flight save on close', async operation => {
const store = new FileSessionStore(filePath);
const first = createStoredSession({ turn: 1 });
const second = createStoredSession({ turn: 2 });
await store.set(first);
const paused = pauseFirstWrite();
const saving = store.save();
await paused.entered;
if (operation === 'set') await store.set(second);
else if (operation === 'delete') await store.delete(first.id);
else await store.clear();
paused.release();
await saving;
await store.close();
const reloaded = new FileSessionStore(filePath);
await reloaded.load();
expect((await reloaded.list()).map(session => session.id)).toEqual(operation === 'set' ? [first.id, second.id] : []);
});
it('serializes saves so an older snapshot cannot overwrite a newer one', async () => {
const store = new FileSessionStore(filePath);
const session = createStoredSession({ turn: 1 });
await store.set(session);
const paused = pauseFirstWrite();
const firstSave = store.save();
await paused.entered;
await store.set({ ...session, data: { turn: 2 } });
const secondSave = store.save();
await new Promise<void>(resolve => setImmediate(resolve));
const writesBeforeRelease = paused.writes();
paused.release();
await Promise.all([firstSave, secondSave]);
await store.close();
const reloaded = new FileSessionStore(filePath);
await reloaded.load();
expect(writesBeforeRelease).toBe(1);
expect((await reloaded.get(session.id))?.data).toEqual({ turn: 2 });
});
it('waits for an outstanding save before closing', async () => {
const store = new FileSessionStore(filePath);
await store.set(createStoredSession());
const paused = pauseFirstWrite();
const saving = store.save();
await paused.entered;
let closed = false;
const closing = store.close().then(() => { closed = true; });
await new Promise<void>(resolve => setImmediate(resolve));
const writesBeforeRelease = paused.writes();
const closedBeforeRelease = closed;
paused.release();
await Promise.all([saving, closing]);
expect(writesBeforeRelease).toBe(1);
expect(closedBeforeRelease).toBe(false);
});
it('does not queue repeated autosaves behind a blocked write', async () => {
jest.useFakeTimers();
const store = new FileSessionStore(filePath, { autoSaveInterval: 10 });
const save = jest.spyOn(store, 'save');
const first = createStoredSession({ turn: 1 });
const second = createStoredSession({ turn: 2 });
await store.set(first);
const paused = pauseFirstWrite();
const saving = store.save();
await paused.entered;
await store.set(second);
jest.advanceTimersByTime(100);
const saveCallsBeforeRelease = save.mock.calls.length;
paused.release();
await saving;
jest.advanceTimersByTime(10);
const saveCallsAfterRelease = save.mock.calls.length;
await store.close();
const reloaded = new FileSessionStore(filePath);
await reloaded.load();
expect(saveCallsBeforeRelease).toBe(1);
expect(saveCallsAfterRelease).toBe(2);
expect((await reloaded.list()).map(session => session.id)).toEqual([first.id, second.id]);
});
it('retries a failed save on close without poisoning later saves', async () => {
const store = new FileSessionStore(filePath);
const session = createStoredSession({ turn: 1 });
await store.set(session);
jest.spyOn(fs, 'writeFile').mockRejectedValueOnce(new Error('disk unavailable'));
await expect(store.save()).rejects.toThrow('disk unavailable');
await store.close();
const reloaded = new FileSessionStore(filePath);
await reloaded.load();
expect((await reloaded.get(session.id))?.data).toEqual({ turn: 1 });
});
});
@@ -0,0 +1,81 @@
import { describe, it, expect } from '@jest/globals';
import { Session } from '../../../src/session/session';
import { MemoryDbAdapter } from '../../../src/db/memory-adapter';
const factories: Array<[string, () => Session]> = [
['new', () => new Session()],
['setState', () => { const s = new Session(); s.setState({}); return s; }],
['clearState', () => { const s = new Session(); s.set('old', 1); s.clearState(); return s; }],
['fromJSON', () => Session.fromJSON(JSON.parse(JSON.stringify(new Session().toJSON())))],
];
describe.each(factories)('Session state keys after %s', (_name, create) => {
it.each(['toString', 'constructor', '__proto__'])('does not report an unstored %s key', key => {
const session = create();
expect(session.has(key)).toBe(false);
expect(session.get(key)).toBeUndefined();
expect(session.delete(key)).toBe(false);
});
it.each(['toString', 'constructor', '__proto__'])('stores and deletes %s as ordinary data', key => {
const session = create();
const value = { preference: 'window seat' };
session.set(key, value);
expect(session.has(key)).toBe(true);
expect(session.get(key)).toEqual(value);
expect(session.keys()).toEqual([key]);
expect(JSON.parse(JSON.stringify(session.toJSON())).data[key]).toEqual(value);
expect(session.delete(key)).toBe(true);
expect(session.has(key)).toBe(false);
expect(session.get(key)).toBeUndefined();
});
it('preserves an explicitly stored undefined value until deletion', () => {
const session = create();
session.set('optional', undefined);
expect(session.has('optional')).toBe(true);
expect(session.delete('optional')).toBe(true);
expect(session.has('optional')).toBe(false);
});
});
it('preserves special keys supplied through setState and a JSON round trip', () => {
const state = JSON.parse('{"__proto__":{"seat":"window"},"constructor":2,"toString":3}');
const session = new Session();
session.setState(state);
const restored = Session.fromJSON(JSON.parse(JSON.stringify(session.toJSON())));
expect(restored.getState()).toEqual(state);
expect(restored.delete('__proto__')).toBe(true);
expect(restored.has('__proto__')).toBe(false);
restored.set('__proto__', { seat: 'aisle' });
expect(restored.keys()).toContain('__proto__');
expect(restored.get('__proto__')).toEqual({ seat: 'aisle' });
});
it('preserves special keys across database save/load without inheriting unstored keys', async () => {
const db = new MemoryDbAdapter();
const session = new Session({ id: 'state-keys', db });
session.set('__proto__', { preference: 'window seat' });
await session.save();
const restored = new Session({ id: session.id, db });
await restored.load();
expect(restored.get('__proto__')).toEqual({ preference: 'window seat' });
expect(restored.keys()).toEqual(['__proto__']);
expect(restored.has('toString')).toBe(false);
restored.delete('__proto__');
restored.set('__proto__', { preference: 'aisle' });
expect(restored.keys()).toEqual(['__proto__']);
expect(restored.get('__proto__')).toEqual({ preference: 'aisle' });
});
it('exports state as an ordinary object while preserving special own keys', () => {
const session = new Session();
session.set('seat', 'window');
session.set('__proto__', { preference: 'quiet' });
const exported = session.toJSON().data;
expect(Object.getPrototypeOf(exported)).toBe(Object.prototype);
expect(exported.hasOwnProperty('seat')).toBe(true);
expect(exported.hasOwnProperty('__proto__')).toBe(true);
expect(JSON.parse(JSON.stringify(exported))['__proto__']).toEqual({ preference: 'quiet' });
expect(session.has('toString')).toBe(false);
});
@@ -0,0 +1,84 @@
import { describe, expect, it } from '@jest/globals';
import { ToolsRegistry } from '../../../src/tools/registry/registry';
import type { ToolMetadata } from '../../../src/tools/registry/types';
const missingPackage = '@praisonai-test/not-installed-registry-fixture';
function register(registry: ToolsRegistry, packageName: string) {
const metadata: ToolMetadata = {
id: 'dependency-test', displayName: 'Dependency test', description: 'Local status test',
tags: [], requiredEnv: [], optionalEnv: [], docsSlug: 'tools/dependency-test',
install: { npm: `npm install ${packageName}`, pnpm: '', yarn: '', bun: '' },
capabilities: {}, packageName,
};
registry.register(metadata, () => ({
name: 'dependency-test', description: 'Local status test',
parameters: { type: 'object', properties: {} }, execute: async () => 'ok',
}));
}
describe('ToolsRegistry installation cache lifecycle', () => {
it('does not report a removed cached tool as installed', async () => {
const registry = new ToolsRegistry();
register(registry, 'node:path');
expect(await registry.checkInstalled('dependency-test')).toBe(true);
expect(registry.unregister('dependency-test')).toBe(true);
expect(await registry.checkInstalled('dependency-test')).toBe(false);
expect(await registry.getInstallStatus('dependency-test')).toBeNull();
});
it('rechecks an installed dependency replaced by a missing one', async () => {
const registry = new ToolsRegistry();
register(registry, 'node:path');
expect(await registry.checkInstalled('dependency-test')).toBe(true);
register(registry, missingPackage);
expect(await registry.getInstallStatus('dependency-test')).toMatchObject({
installed: false, installCommand: `npm install ${missingPackage}`,
});
});
it('rechecks a missing dependency replaced by an installed one', async () => {
const registry = new ToolsRegistry();
register(registry, missingPackage);
expect(await registry.checkInstalled('dependency-test')).toBe(false);
register(registry, 'node:path');
expect(await registry.getInstallStatus('dependency-test')).toMatchObject({ installed: true, installCommand: undefined });
});
it('does not cache an old successful import after replacement', async () => {
const registry = new ToolsRegistry();
register(registry, 'node:path');
const oldCheck = registry.checkInstalled('dependency-test');
register(registry, missingPackage);
expect(await oldCheck).toBe(true);
expect(await registry.checkInstalled('dependency-test')).toBe(false);
});
it('does not cache an old failed import after replacement', async () => {
const registry = new ToolsRegistry();
register(registry, missingPackage);
const oldCheck = registry.checkInstalled('dependency-test');
register(registry, 'node:path');
expect(await oldCheck).toBe(false);
expect(await registry.checkInstalled('dependency-test')).toBe(true);
});
it('reports an unknown tool as unavailable', async () => {
const registry = new ToolsRegistry();
expect(await registry.checkInstalled('unknown')).toBe(false);
expect(await registry.getInstallStatus('unknown')).toBeNull();
});
it('keeps repeated installed checks consistent', async () => {
const registry = new ToolsRegistry();
register(registry, 'node:path');
expect(await registry.checkInstalled('dependency-test')).toBe(true);
expect(await registry.checkInstalled('dependency-test')).toBe(true);
});
it('keeps repeated missing checks consistent', async () => {
const registry = new ToolsRegistry();
register(registry, missingPackage);
expect(await registry.checkInstalled('dependency-test')).toBe(false);
expect(await registry.checkInstalled('dependency-test')).toBe(false);
});
});
@@ -0,0 +1,104 @@
import { describe, expect, it, jest } from '@jest/globals';
import { ToolsRegistry } from '../../../src/tools/registry/registry';
import { composeMiddleware, createRetryMiddleware } from '../../../src/tools/registry/middleware';
import type { ToolMetadata, ToolMiddleware, ToolExecutionContext } from '../../../src/tools/registry/types';
const metadata: ToolMetadata = {
id: 'retry-test', displayName: 'Retry test', description: 'Local retry test',
tags: [], requiredEnv: [], optionalEnv: [], docsSlug: 'tools/retry-test',
install: { npm: '', pnpm: '', yarn: '', bun: '' }, capabilities: {}, packageName: 'node:path',
};
function runPipeline(mode: 'composed' | 'registry', middleware: ToolMiddleware[], execute: () => Promise<unknown>, context: ToolExecutionContext = {}) {
if (mode === 'composed') return composeMiddleware(...middleware)({}, context, execute);
const registry = new ToolsRegistry();
registry.register(metadata, () => ({
name: 'retry-test', description: 'Local retry test',
parameters: { type: 'object', properties: {} }, execute,
}));
for (const layer of middleware) registry.use(layer);
return registry.create('retry-test').execute({}, context);
}
describe.each(['composed', 'registry'] as const)('%s middleware retry chain', mode => {
it('reapplies result shaping when the tool succeeds on retry', async () => {
let attempts = 0;
let shapingCalls = 0;
const shape: ToolMiddleware = async (_input, _context, next) => {
shapingCalls++;
return { wrapped: await next() };
};
const result = await runPipeline(mode, [createRetryMiddleware(1, 0), shape], async () => {
if (++attempts === 1) throw new Error('temporary tool failure');
return 'recovered';
});
expect(result).toEqual({ wrapped: 'recovered' });
expect(attempts).toBe(2);
expect(shapingCalls).toBe(2);
});
it('re-enters a downstream middleware that failed before the tool', async () => {
let layerAttempts = 0;
let toolCalls = 0;
const layer: ToolMiddleware = async (_input, _context, next) => {
if (++layerAttempts === 1) throw new Error('temporary middleware failure');
return { wrapped: await next() };
};
const result = await runPipeline(mode, [createRetryMiddleware(1, 0), layer], async () => {
toolCalls++;
return 'result';
});
expect(result).toEqual({ wrapped: 'result' });
expect(layerAttempts).toBe(2);
expect(toolCalls).toBe(1);
});
it('runs downstream failure handling on every exhausted attempt', async () => {
let attempts = 0;
let handled = 0;
const layer: ToolMiddleware = async (_input, _context, next) => {
try { return await next(); } finally { handled++; }
};
await expect(runPipeline(mode, [createRetryMiddleware(2, 0), layer], async () => {
throw new Error(`failure-${++attempts}`);
})).rejects.toThrow('failure-3');
expect(attempts).toBe(3);
expect(handled).toBe(3);
});
it('does not re-enter downstream middleware after cancellation during backoff', async () => {
jest.useFakeTimers();
try {
const controller = new AbortController();
let entries = 0;
let calls = 0;
const layer: ToolMiddleware = async (_input, _context, next) => { entries++; return next(); };
const done = runPipeline(mode, [createRetryMiddleware(1, 1000), layer], async () => {
if (++calls === 1) throw new Error('temporary');
return 'retried';
}, { signal: controller.signal }).then(value => ({ value, error: undefined }), error => ({ value: undefined, error }));
await jest.advanceTimersByTimeAsync(0);
expect(entries).toBe(1);
controller.abort();
await jest.advanceTimersByTimeAsync(1000);
expect(entries).toBe(1);
expect(calls).toBe(1);
expect((await done).error).toBe(controller.signal.reason);
} finally { jest.clearAllTimers(); jest.useRealTimers(); }
});
it('preserves middleware order for a successful first attempt', async () => {
const order: string[] = [];
const layer = (name: string): ToolMiddleware => async (_input, _context, next) => {
order.push(`${name}:before`);
const result = await next();
order.push(`${name}:after`);
return result;
};
expect(await runPipeline(mode, [createRetryMiddleware(1, 0), layer('a'), layer('b')], async () => {
order.push('tool');
return 'ok';
})).toBe('ok');
expect(order).toEqual(['a:before', 'b:before', 'tool', 'b:after', 'a:after']);
});
});
@@ -0,0 +1,54 @@
import { describe, expect, it, jest } from '@jest/globals';
import { createValidationMiddleware } from '../../../src/tools/registry/middleware';
const payloads: Array<[string, unknown]> = [
['CJK text', '你好'],
['emoji', '🙂'],
['nested JSON', { message: '中文🙂', values: ['é'] }],
];
describe('tool payload UTF-8 byte limits', () => {
it.each(payloads)('rejects %s above its actual byte budget', async (_label, input) => {
const bytes = Buffer.byteLength(JSON.stringify(input), 'utf8');
const next = jest.fn(async () => 'executed');
await expect(createValidationMiddleware()(input, { limits: { maxPayloadBytes: bytes - 1 } }, next))
.rejects.toThrow(`Input payload size (${bytes} bytes) exceeds limit (${bytes - 1} bytes)`);
expect(next).not.toHaveBeenCalled();
});
it.each(payloads)('accepts %s at the exact byte budget', async (_label, input) => {
const bytes = Buffer.byteLength(JSON.stringify(input), 'utf8');
const next = jest.fn(async () => 'executed');
expect(await createValidationMiddleware()(input, { limits: { maxPayloadBytes: bytes } }, next)).toBe('executed');
expect(next).toHaveBeenCalledTimes(1);
});
it('preserves ASCII boundaries', async () => {
const next = jest.fn(async () => 'executed');
const validate = createValidationMiddleware();
await expect(validate('hello', { limits: { maxPayloadBytes: 6 } }, next)).rejects.toThrow('7 bytes');
expect(await validate('hello', { limits: { maxPayloadBytes: 7 } }, next)).toBe('executed');
expect(next).toHaveBeenCalledTimes(1);
});
it('does not serialize inputs when no payload limit is configured', async () => {
const cyclic: { self?: unknown } = {};
cyclic.self = cyclic;
const next = jest.fn(async () => 'executed');
expect(await createValidationMiddleware()(cyclic, {}, next)).toBe('executed');
});
it('continues rejecting non-JSON undefined input when a limit is configured', async () => {
const next = jest.fn(async () => 'executed');
await expect(createValidationMiddleware()(undefined, { limits: { maxPayloadBytes: 10 } }, next)).rejects.toThrow(TypeError);
expect(next).not.toHaveBeenCalled();
});
it('preserves circular JSON errors when a limit is configured', async () => {
const cyclic: { self?: unknown } = {};
cyclic.self = cyclic;
const next = jest.fn(async () => 'executed');
await expect(createValidationMiddleware()(cyclic, { limits: { maxPayloadBytes: 100 } }, next)).rejects.toThrow(TypeError);
expect(next).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,94 @@
import { afterEach, beforeEach, describe, expect, it, jest } from '@jest/globals';
import { getEventListeners } from 'events';
import { FunctionTool } from '../../../src/tools/decorator';
function makeTool(execute: () => Promise<string>) {
return new FunctionTool({ name: 'retry_test', execute, retryPolicy: { maxAttempts: 2, initialDelayMs: 1000, maxDelayMs: 1000 } });
}
describe('FunctionTool retry backoff cancellation', () => {
beforeEach(() => { jest.useFakeTimers(); });
afterEach(() => { jest.clearAllTimers(); jest.useRealTimers(); });
it.each(['execute', 'executeRaw'] as const)('%s settles promptly when cancelled during backoff', async method => {
const controller = new AbortController();
const failure = new Error('connection dropped');
let calls = 0;
let settled = false;
const tool = makeTool(async () => { if (++calls === 1) throw failure; return 'retried'; });
const done = tool[method]({}, { signal: controller.signal }).then(
value => { settled = true; return { value, error: undefined }; },
error => { settled = true; return { value: undefined, error }; },
);
await jest.advanceTimersByTimeAsync(0);
expect(calls).toBe(1);
controller.abort();
await jest.advanceTimersByTimeAsync(0);
expect(settled).toBe(true);
expect((await done).error).toBe(failure);
expect(jest.getTimerCount()).toBe(0);
expect(getEventListeners(controller.signal, 'abort')).toHaveLength(0);
await jest.advanceTimersByTimeAsync(2000);
expect(calls).toBe(1);
});
it.each(['execute', 'executeRaw'] as const)('%s does not start another call after the original delay', async method => {
const controller = new AbortController();
const failure = new Error('connection dropped');
let calls = 0;
const tool = makeTool(async () => { if (++calls === 1) throw failure; return 'retried'; });
const done = tool[method]({}, { signal: controller.signal }).then(value => ({ value, error: undefined }), error => ({ value: undefined, error }));
await jest.advanceTimersByTimeAsync(0);
controller.abort();
await jest.advanceTimersByTimeAsync(1000);
expect(calls).toBe(1);
expect((await done).error).toBe(failure);
});
it('retains the configured delay without cancellation and releases its listener', async () => {
const controller = new AbortController();
let calls = 0;
const tool = makeTool(async () => { if (++calls === 1) throw new Error('connection dropped'); return 'recovered'; });
const done = tool.execute({}, { signal: controller.signal });
await jest.advanceTimersByTimeAsync(999);
expect(calls).toBe(1);
await jest.advanceTimersByTimeAsync(1);
expect(await done).toBe('recovered');
expect(calls).toBe(2);
expect(jest.getTimerCount()).toBe(0);
expect(getEventListeners(controller.signal, 'abort')).toHaveLength(0);
});
it('retains retries without a signal', async () => {
let calls = 0;
const tool = makeTool(async () => { if (++calls === 1) throw new Error('connection dropped'); return 'recovered'; });
const done = tool.execute({});
await jest.advanceTimersByTimeAsync(1000);
expect(await done).toBe('recovered');
expect(calls).toBe(2);
});
it('preserves the last error when already cancelled at failure', async () => {
const controller = new AbortController();
const failure = new Error('connection dropped');
const tool = makeTool(async () => { controller.abort(); throw failure; });
await expect(tool.execute({}, { signal: controller.signal })).rejects.toBe(failure);
expect(jest.getTimerCount()).toBe(0);
});
it('does not retry a nonretryable failure', async () => {
const failure = new Error('invalid argument');
let calls = 0;
const tool = makeTool(async () => { calls++; throw failure; });
await expect(tool.execute({})).rejects.toBe(failure);
expect(calls).toBe(1);
expect(jest.getTimerCount()).toBe(0);
});
it('does not allocate a backoff on immediate success', async () => {
const controller = new AbortController();
expect(await makeTool(async () => 'ok').execute({}, { signal: controller.signal })).toBe('ok');
expect(jest.getTimerCount()).toBe(0);
expect(getEventListeners(controller.signal, 'abort')).toHaveLength(0);
});
});
+340 -177
View File
@@ -14,27 +14,145 @@
<a href="https://github.com/MervinPraison/PraisonAI"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License" /></a>
</p>
<div align="center">
# PraisonAI
# PraisonAI 🦞
**Private AI that earns trust one task at a time.**
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
PraisonAI runs AI agents on your own computers, even fully offline, so private data never has to leave the building.
</div>
- **For highly regulated teams (early access):** each task moves to your own AI only after it passes a test. If quality slips, it moves back automatically, and every step is recorded for regulators. [How it works](#how-the-handover-works-early-access)
- **For everyone else:** a free, open-source Python agent framework. With a local model you need no API key and pay no cloud bill, and your files stay yours.
PraisonAI 🦞 — **Hire a 24/7 AI Workforce.** Stop writing boilerplate and start shipping autonomous, self-improving agents that research, plan, and execute tasks across your apps. From one agent to an entire organization, deployed in 5 lines of code.
[2M+ downloads](https://pepy.tech/projects/praisonai) · [9.3K stars](https://github.com/MervinPraison/PraisonAI) · [600+ releases in the last year](https://github.com/MervinPraison/PraisonAI/releases) · [80+ contributors](https://github.com/MervinPraison/PraisonAI/graphs/contributors) · Python
```bash
curl -fsSL https://praison.ai/install.sh | bash
pip install praisonai
```
Only need the Python library? `pip install praisonaiagents` installs just the lightweight core.
## Using PraisonAI for something else?
PraisonAI is still a general-purpose, open-source agent framework, and everything works as before: the same packages, imports, commands and examples. Go straight to what you came for:
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs)
- **Workflows:** [AgentFlow: route, parallel, loop, repeat](https://praison.ai/docs/concepts/agentflow)
- **Memory and knowledge:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge](https://praison.ai/docs/concepts/knowledge) · [RAG](https://praison.ai/docs/concepts/rag)
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
- **Bots (Telegram, Discord, Slack, WhatsApp):** [Bot CLI](https://praison.ai/docs/cli/bot) · [Bot gateway](https://praison.ai/docs/features/bot-gateway)
- **Terminal CLI:** [`praisonai code`](https://praison.ai/docs/cli/code) · [`praisonai run`](https://praison.ai/docs/cli/run) · [`praisonai chat`](https://praison.ai/docs/cli/chat) · [CLI reference](https://praison.ai/docs/cli/cli-reference)
- **Training and fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
- **Browser automation:** [Browser agent](https://praison.ai/docs/features/browser-agent) · [Browser CLI](https://praison.ai/docs/cli/browser)
- **Sandboxes:** [Sandboxed code execution](https://praison.ai/docs/features/sandbox)
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
- **Dashboard and UI:** [Claw dashboard](https://praison.ai/docs/concepts/claw) · [Chat UI](https://praison.ai/docs/ui)
- **YAML, no code:** [YAML agents](#-using-yaml-no-code) · [YAML reference](https://praison.ai/docs/features/yaml-configuration-reference)
- **JavaScript and TypeScript:** [JavaScript SDK](https://praison.ai/docs/js)
- **Desktop app:** [PraisonAI Desktop](https://praison.ai/docs/features/desktop/index) · [Downloads](#desktop-app)
- **Everything else:** the [full feature reference](#full-feature-reference) further down this page
## Quick start
Set a key for any supported provider (OpenAI shown here), or skip the key and use a local model as in the next section.
```bash
export OPENAI_API_KEY="your-api-key"
```
```python
from praisonaiagents import Agent
agent = Agent(instructions="You are a helpful assistant.")
print(agent.start("In two sentences, why do some companies keep their data on their own computers?"))
```
## Run fully offline
Run the same agent on a model on your own computer with [Ollama](https://ollama.com). You need no API key, and your prompts and answers stay on the machine.
Install the package and pull the model while you still have internet; after that, the agent runs with no network at all. If you installed only the core library, use `pip install "praisonaiagents[llm]"`, which adds the local-model support.
```bash
# Start Ollama first (the app, or "ollama serve" in another terminal)
ollama pull llama3.2
export DO_NOT_TRACK=true # keeps telemetry off (PraisonAI sends none unless you turn it on)
```
```python
from praisonaiagents import Agent
agent = Agent(instructions="You are a helpful assistant.", model="ollama/llama3.2")
print(agent.start("Why is the sky blue?"))
```
If Ollama runs on another computer on your network, pass `base_url` (or set `OLLAMA_API_BASE`):
```python
agent = Agent(instructions="...", model="ollama/llama3.2", base_url="http://192.168.1.10:11434")
```
vLLM and LM Studio work the same way: use `hosted_vllm/` (vLLM) or `lm_studio/` with your server's `base_url`. More in the [Ollama guide](https://praison.ai/docs/models/ollama) and [Models docs](https://praison.ai/docs/models).
## How the handover works (early access)
1. **Watch:** records how each task is done today.
2. **Test:** your own AI takes an exam made from real past work it never studied.
3. **Hand over:** a task moves to your own AI only when it passes.
4. **Roll back:** if quality drops, the task moves back automatically.
5. **Record:** every handover leaves a record your regulator can check.
**What is "your own AI"?** An open model, such as Llama, Qwen or Gemma, running on your own computers. Use it as it is, or train it on your own past work with [PraisonAI Train](https://praison.ai/docs/train) on your own GPU servers. Your team decides when to train; nothing trains automatically. Either way, a task moves to it only after it passes that exam.
Highly regulated teams, such as banks, insurers and hospitals, want AI but can't send customer data to outside models. From 6 January 2027, OpenAI also takes no new self-serve fine-tuning jobs ([notice](https://developers.openai.com/api/docs/deprecations)), so teams that customise models need models they own. The handover lets these teams move work to AI they own, one task at a time, with a record that shows why each step was trusted.
It is built on parts of this framework that are live today: tracing, evaluation, fine-tuning, model routing and local models. The handover itself is in early access, and we are taking on our first design partners. To take part, see [praison.ai/enterprise](https://praison.ai/enterprise/).
## Features
- **Agents and multi-agent teams:** [Agents](https://praison.ai/docs/concepts/agents) · [Handoffs](https://praison.ai/docs/concepts/handoffs) · [Workflows](https://praison.ai/docs/concepts/agentflow)
- **Memory:** [Memory](https://praison.ai/docs/concepts/memory) · [Knowledge and RAG](https://praison.ai/docs/concepts/rag) · [Sessions](https://praison.ai/docs/concepts/session-management)
- **Tools and MCP:** [Tools](https://praison.ai/docs/tools/tools) · [MCP](https://praison.ai/docs/mcp/transports)
- **Guardrails and approval:** [Guardrails](https://praison.ai/docs/concepts/guardrails) · [Human approval](https://praison.ai/docs/concepts/approval) · [Policy engine](https://praison.ai/docs/features/policy-engine)
- **Tracing and evaluation:** [Tracing](https://praison.ai/docs/observability/overview) · [Telemetry](https://praison.ai/docs/features/telemetry) · [Langfuse](https://praison.ai/docs/observability/langfuse) · [Evaluation](https://praison.ai/docs/concepts/evaluation)
- **Your choice of model:** [Models](https://praison.ai/docs/models) · [Model router](https://praison.ai/docs/features/model-router)
- **Fine-tuning:** [PraisonAI Train](https://praison.ai/docs/train)
- **Deploy:** [Deploy guide](https://praison.ai/docs/deploy)
## JavaScript and TypeScript
```bash
npm install praisonai
```
The TypeScript and JavaScript SDK has its own guide at [praison.ai/docs/js](https://praison.ai/docs/js). A short example is in [Using JavaScript Code](#-using-javascript-code) below.
## The company
- **Who:** Praison AI Limited in Cambridge, UK, building PraisonAI since September 2023. Co-founders Mervin Praison (CEO), Leebana Smith (COO) and Karthick Subramanian (CBO). [About us](https://praison.ai/about/)
- **What we're building:** the task handover above, as a self-hosted product on top of this framework. It is in early access, and we are taking on our first design partners.
- **Why now:** highly regulated teams want AI but can't send customer data to outside models, and from 6 January 2027 OpenAI takes no new self-serve fine-tuning jobs.
- **How it fits together:** the framework stays free and MIT-licensed, and brings developers in. Regulated teams license the handover to run inside their own walls.
- **Traction:** every number at the top of this page is public and linked to its source.
- **Get in touch:** design partners, investors and press: enterprise@praison.ai or [praison.ai/enterprise](https://praison.ai/enterprise/)
## Docs, community and licence
- **Documentation:** [praison.ai/docs](https://praison.ai/docs)
- **Questions and bugs:** [Discussions](https://github.com/MervinPraison/PraisonAI/discussions) · [Issues](https://github.com/MervinPraison/PraisonAI/issues)
- **Contributing:** [Contributing guide](https://github.com/MervinPraison/PraisonAI/blob/main/CONTRIBUTING.md)
- **Videos and news:** [YouTube](https://youtube.com/@MervinPraison) · [X](https://x.com/MervinPraison) · [LinkedIn](https://linkedin.com/in/mervinpraison)
- **Desktop app:** macOS, Windows and Linux builds are on the [latest release](https://github.com/MervinPraison/PraisonAI/releases/latest)
- **Licence:** [MIT](https://github.com/MervinPraison/PraisonAI/blob/main/LICENSE)
---
## Full feature reference
Everything below is the full reference that was on this page before. Nothing has been removed; the longest parts are folded, so click a heading's summary to open it.
### Desktop app
<div align="center">
<br>
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
</a>
<br>
<br>
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
<img src="https://img.shields.io/badge/Download_for_macOS-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Download for macOS" />
</a>
@@ -46,12 +164,17 @@ curl -fsSL https://praison.ai/install.sh | bash
<a href="https://github.com/MervinPraison/PraisonAI/releases/latest">
<img src="https://img.shields.io/badge/Download_for_Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download for Linux" />
</a>
<br>
</div>
<p align="center">
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
</p>
### Other ways to install
One-line installer:
```bash
curl -fsSL https://praison.ai/install.sh | bash
```
The full `praisonai` package, with the CLI, bots, dashboard and integrations:
```
██████╗ ██████╗ █████╗ ██╗███████╗ ██████╗ ███╗ ██╗ █████╗ ██╗
@@ -64,10 +187,20 @@ curl -fsSL https://praison.ai/install.sh | bash
pip install praisonai
```
<p align="center">
<img src=".github/images/dashboard.png" alt="PraisonAI Dashboard" width="800" />
</p>
<div align="center">
<a href="https://docs.praison.ai">
<a href="https://trendshift.io/repositories/9130" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9130" alt="MervinPraison%2FPraisonAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<div align="center">
<a href="https://praison.ai/docs">
<p align="center">
<img src="https://img.shields.io/badge/📚_Documentation-Visit_docs.praison.ai-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
<img src="https://img.shields.io/badge/📚_Documentation-Visit_praison.ai%2Fdocs-blue?style=for-the-badge&logo=bookstack&logoColor=white" alt="Documentation" />
</p>
</a>
</div>
@@ -114,6 +247,9 @@ Most frameworks hand you one or two layers and leave the rest as homework. Prais
Each layer wraps the one inside it. When an agent misbehaves, the layer tells you where to look.
<details>
<summary><strong>Show the five layers, examples and Managed Agents</strong></summary>
```
┌─────────────────────────────────────────────────────────────────┐
│ ⬡ MANAGED AGENTS — Where does it actually run? │
@@ -174,7 +310,7 @@ agent = Agent(
)
```
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://docs.praison.ai/docs/concepts/handoffs)
> **Isolate** is `handoffs=[specialist]` — a sub-agent inherits the last few messages and the intersection of your tools, not your whole transcript. [📖 Handoffs](https://praison.ai/docs/concepts/handoffs)
### Layer 3 · Harness — *Can it act, and be checked?*
@@ -217,7 +353,7 @@ print(result.completion_reason)
# surfaced here as completion_reason="error")
```
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://docs.praison.ai/docs/features/doom-loop-detection)
> **Doom-loop detection is on by default.** Repeated identical tool calls and A→B→A→B oscillation get caught — while a poller whose output keeps changing does not. [📖 Doom Loop Detection](https://praison.ai/docs/features/doom-loop-detection)
### Layer 5 · Graph — *Who runs when, and who checks whom?*
@@ -237,7 +373,7 @@ flow = AgentFlow(steps=[
flow.run("Ticket #123: login fails on Safari")
```
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://docs.praison.ai/docs/concepts/agentflow)
> The same graph is expressible in YAML with no Python at all. [📖 AgentFlow](https://praison.ai/docs/concepts/agentflow)
### ⬡ Outside the stack: Managed Agents — *Where does it actually run?*
@@ -332,6 +468,8 @@ Sandboxes shut themselves down when idle (`auto_shutdown`, `idle_timeout_s`), an
<sub>Stack framing adapted from [The Five-Layer Agent Stack](https://mer.vin/2026/07/five-layer-agent-stack-match-bug-to-right-layer/) and [Agent Harnesses vs Orbs](https://mer.vin/2026/08/agent-harnesses-vs-orbs-why-remote-sandboxes-beat-local-agent-loops/).</sub>
</details>
---
## 🌌 The PraisonAI Ecosystem
@@ -411,6 +549,11 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
</details>
<div align="center">
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
<img src="https://img.shields.io/badge/Highlighted_by_Elon_Musk-000000?style=for-the-badge&logo=x&logoColor=white" alt="Highlighted by Elon Musk" />
</a>
<br>
<br>
<a href="https://x.com/elonmusk/status/1893870468249141688" target="_blank">
<img src=".github/images/elon_musk_praisonai.png" alt="Highlighted by Elon Musk" width="600" />
</a>
@@ -426,13 +569,13 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
|--|---------|-----|
| 🔌 | **MCP Protocol** — stdio, HTTP, WebSocket, SSE | `tools=MCP("npx ...")` |
| 🧠 | **Planning Mode** — plan → execute → reason | `planning=True` |
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://docs.praison.ai/docs/agents/deep-research) |
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://docs.praison.ai/docs/code/external-agents) |
| 🔍 | **Deep Research** — multi-step autonomous research | [Docs](https://praison.ai/docs/agents/deep-research) |
| 🤖 | **External Agents** — orchestrate Claude Code, Gemini CLI, Codex | [Docs](https://praison.ai/docs/code/external-agents) |
| 🔄 | **Agent Handoffs** — seamless conversation passing | `handoffs=[other_agent]` |
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://docs.praison.ai/docs/concepts/guardrails) |
| 🛡️ | **Guardrails** — input/output validation | [Docs](https://praison.ai/docs/concepts/guardrails) |
| | **Web Search + Fetch** — native browsing | `web=True` |
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://docs.praison.ai/docs/concepts/reflection) |
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://docs.praison.ai/docs/concepts/agentflow) |
| 🪞 | **Self Reflection** — agent reviews its own output | [Docs](https://praison.ai/docs/concepts/reflection) |
| 🔀 | **Workflow Patterns** — route, parallel, loop, repeat | [Docs](https://praison.ai/docs/concepts/agentflow) |
| 🧠 | **Memory (zero deps)** — works out of the box | `memory=True` |
<details>
@@ -443,18 +586,18 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
| 💡 | **Prompt Caching** — reduce latency + cost | `caching=True` |
| 💾 | **Sessions + Auto-Save** — persistent state across restarts | `auto_save="my-project"` |
| 💭 | **Thinking Budgets** — control reasoning depth | `agent.thinking_budget = 1024` |
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://docs.praison.ai/docs/concepts/rag) |
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://docs.praison.ai/docs/features/model-router) |
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://docs.praison.ai/docs/features/checkpoints) |
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://docs.praison.ai/docs/features/a2a) |
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://docs.praison.ai/docs/features/context-compaction) |
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://docs.praison.ai/docs/features/telemetry) |
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://docs.praison.ai/docs/features/policy-engine) |
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://docs.praison.ai/docs/features/background-tasks) |
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://docs.praison.ai/docs/features/doom-loop-detection) |
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://docs.praison.ai/docs/features/graph-memory) |
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://docs.praison.ai/docs/features/sandbox) |
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://docs.praison.ai/docs/features/bot-gateway) |
| 📚 | **RAG + Quality-Based RAG** — auto quality scoring retrieval | [Docs](https://praison.ai/docs/concepts/rag) |
| 📊 | **Model Router** — auto-routes to cheapest capable model | [Docs](https://praison.ai/docs/features/model-router) |
| 🧊 | **Shadow Git Checkpoints** — auto-rollback on failure | [Docs](https://praison.ai/docs/features/checkpoints) |
| 📡 | **A2A Protocol** — agent-to-agent interop | [Docs](https://praison.ai/docs/features/a2a) |
| 📏 | **Context Compaction** — never hit token limits | [Docs](https://praison.ai/docs/features/context-compaction) |
| 📡 | **Telemetry** — OpenTelemetry traces, spans, metrics | [Docs](https://praison.ai/docs/features/telemetry) |
| 📜 | **Policy Engine** — declarative agent behavior control | [Docs](https://praison.ai/docs/features/policy-engine) |
| 🔄 | **Background Tasks** — fire-and-forget agents | [Docs](https://praison.ai/docs/features/background-tasks) |
| 🔁 | **Doom Loop Detection** — auto-recovery from stuck agents | [Docs](https://praison.ai/docs/features/doom-loop-detection) |
| 🕸️ | **Graph Memory** — Neo4j-style relationship tracking | [Docs](https://praison.ai/docs/features/graph-memory) |
| 🏖️ | **Sandbox Execution** — isolated code execution | [Docs](https://praison.ai/docs/features/sandbox) |
| 🖥️ | **Bot Gateway** — multi-agent routing across channels | [Docs](https://praison.ai/docs/features/bot-gateway) |
</details>
@@ -465,6 +608,9 @@ Powered by 100+ LLMs (OpenAI, Anthropic, Gemini & local models).
## 📘 Using Python Code
<details>
<summary><strong>Show Python examples: single agent, multi-agent, MCP, tools, databases, Claw, Langflow, UI</strong></summary>
> Unsure whether a result is a bug or model randomness? Check the [SDK Behavior Matrix](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/docs/sdk-behavior-matrix.md) for `stream`, memory prefetch, structured output, teams and AG-UI.
### 1. Single Agent
@@ -510,7 +656,7 @@ agent = Agent(
)
```
> 📖 [Full MCP docs](https://docs.praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
> 📖 [Full MCP docs](https://praison.ai/docs/mcp/transports) — stdio, HTTP, WebSocket, SSE transports
### 4. Custom Tools
@@ -562,7 +708,7 @@ agent.start("Search for AI news and calculate 15*4")
```
> ⚠️ **Security Note:** Never use `eval()`, `exec()`, or `subprocess` in tool functions that process LLM-generated or user-supplied input. Always validate and sanitize inputs to prevent code injection attacks.
> 📖 [Full tools docs](https://docs.praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
> 📖 [Full tools docs](https://praison.ai/docs/tools/tools) — BaseTool, tool packages, 100+ built-in tools
### 5. Persistence (Databases)
@@ -579,7 +725,7 @@ agent = Agent(
agent.chat("Hello!") # Auto-persists messages, runs, traces
```
> 📖 [Full persistence docs](https://docs.praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
> 📖 [Full persistence docs](https://praison.ai/docs/databases/overview) — PostgreSQL, MySQL, SQLite, MongoDB, Redis, and 20+ more
### 6. PraisonAI Claw 🦞 (Dashboard UI)
@@ -601,7 +747,7 @@ Copy `.env.example` to `.env` and configure the following variables:
Open **http://localhost:8082** — the dashboard comes with 13 built-in pages: Chat, Agents, Memory, Knowledge, Channels, Guardrails, Cron, and more. Add messaging channels directly from the UI.
> 📖 [Full Claw docs](https://docs.praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
> 📖 [Full Claw docs](https://praison.ai/docs/concepts/claw) — platform tokens, CLI options, Docker, and YAML agent mode
### 7. Langflow Integration 🔗 (Visual Flow Builder)
@@ -614,7 +760,7 @@ praisonai flow
Open **http://localhost:7861** — use the **Agent** and **Agent Team** components to create sequential or parallel workflows. Connect Chat Input → Agent Team → Chat Output for instant multi-agent pipelines.
> 📖 [Full Flow docs](https://docs.praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
> 📖 [Full Flow docs](https://praison.ai/docs/concepts/agentflow) — visual agent building, component reference, and deployment
### 8. PraisonAI UI 🤖 (Clean Chat)
@@ -625,10 +771,15 @@ pip install "praisonai[ui]"
praisonai ui
```
</details>
---
## 📄 Using YAML (No Code)
<details>
<summary><strong>Show YAML examples</strong></summary>
### Example 1: Two Agents Working Together
Create `agents.yaml`:
@@ -700,6 +851,8 @@ praisonai agents.yaml
> - Tools in `tools.py` are automatically discovered
> - The function's docstring helps the AI understand how to use it
</details>
---
## 🎯 CLI Quick Reference
@@ -718,23 +871,26 @@ praisonai agents.yaml
| **Development** | `commit`, `docs`, `checkpoint`, `hooks` |
| **Scheduling** | `schedule start`, `schedule list`, `schedule stop` |
> 📖 [Full CLI reference](https://docs.praison.ai/docs/cli/cli-reference)
> 📖 [Full CLI reference](https://praison.ai/docs/cli/cli-reference)
---
## ✨ Key Features
<details>
<summary><strong>Show all features by category</strong></summary>
<details open>
<summary><strong>🤖 Core Agents</strong></summary>
| Feature | Code | Docs |
|---------|:----:|:----:|
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://docs.praison.ai/docs/agents/single) |
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://docs.praison.ai/docs/concepts/agents) |
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://docs.praison.ai/docs/features/autoagents) |
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://docs.praison.ai/docs/concepts/reflection) |
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://docs.praison.ai/docs/features/reasoning) |
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://docs.praison.ai/docs/features/multimodal) |
| Single Agent | [Example](examples/python/agents/single-agent.py) | [📖](https://praison.ai/docs/agents/single) |
| Multi Agents | [Example](examples/python/general/mini_agents_example.py) | [📖](https://praison.ai/docs/concepts/agents) |
| Auto Agents | [Example](examples/python/general/auto_agents_example.py) | [📖](https://praison.ai/docs/features/autoagents) |
| Self Reflection AI Agents | [Example](examples/python/concepts/self-reflection-details.py) | [📖](https://praison.ai/docs/concepts/reflection) |
| Reasoning AI Agents | [Example](examples/python/concepts/reasoning-extraction.py) | [📖](https://praison.ai/docs/features/reasoning) |
| Multi Modal AI Agents | [Example](examples/python/general/multimodal.py) | [📖](https://praison.ai/docs/features/multimodal) |
</details>
@@ -743,16 +899,16 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://docs.praison.ai/docs/features/routing) |
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://docs.praison.ai/docs/concepts/agentflow) |
| Simple Workflow | [Example](examples/python/workflows/simple_workflow.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow with Agents | [Example](examples/python/workflows/workflow_with_agents.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Agentic Routing (`route()`) | [Example](examples/python/workflows/workflow_routing.py) | [📖](https://praison.ai/docs/features/routing) |
| Parallel Execution (`parallel()`) | [Example](examples/python/workflows/workflow_parallel.py) | [📖](https://praison.ai/docs/features/parallelisation) |
| Loop over List/CSV (`loop()`) | [Example](examples/python/workflows/workflow_loop_csv.py) | [📖](https://praison.ai/docs/features/repetitive) |
| Evaluator-Optimizer (`repeat()`) | [Example](examples/python/workflows/workflow_repeat.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
| Conditional Steps | [Example](examples/python/workflows/workflow_conditional.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow Branching | [Example](examples/python/workflows/workflow_branching.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow Early Stop | [Example](examples/python/workflows/workflow_early_stop.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
| Workflow Checkpoints | [Example](examples/python/workflows/workflow_checkpoints.py) | [📖](https://praison.ai/docs/concepts/agentflow) |
</details>
@@ -761,13 +917,13 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://docs.praison.ai/docs/features/codeagent) |
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://docs.praison.ai/docs/code/editing) |
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://docs.praison.ai/docs/code/external-agents) |
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://docs.praison.ai/docs/code/claude-code) |
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://docs.praison.ai/docs/code/gemini-cli) |
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://docs.praison.ai/docs/code/codex-cli) |
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://docs.praison.ai/docs/code/cursor-cli) |
| Code Interpreter Agents | [Example](examples/python/agents/code-agent.py) | [📖](https://praison.ai/docs/features/codeagent) |
| AI Code Editing Tools | [Example](examples/python/code/code_editing_example.py) | [📖](https://praison.ai/docs/code/editing) |
| External Agents (All) | [Example](examples/python/code/external_agents_example.py) | [📖](https://praison.ai/docs/code/external-agents) |
| Claude Code CLI | [Example](examples/python/code/claude_code_example.py) | [📖](https://praison.ai/docs/code/claude-code) |
| Gemini CLI | [Example](examples/python/code/gemini_cli_example.py) | [📖](https://praison.ai/docs/code/gemini-cli) |
| Codex CLI | [Example](examples/python/code/codex_cli_example.py) | [📖](https://praison.ai/docs/code/codex-cli) |
| Cursor CLI | [Example](examples/python/code/cursor_cli_example.py) | [📖](https://praison.ai/docs/code/cursor-cli) |
</details>
@@ -776,18 +932,18 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://docs.praison.ai/docs/concepts/memory) |
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://docs.praison.ai/docs/features/claude-memory-tool) |
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://docs.praison.ai/docs/concepts/knowledge) |
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://docs.praison.ai/docs/concepts/rag) |
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://docs.praison.ai/docs/features/chat-with-pdf) |
| Data Readers (PDF, DOCX, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/chunking-strategies) |
| Vector Store Selection | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/knowledge-backends) |
| Retrieval Strategies | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval-strategies) |
| Rerankers | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/smart-retrieval) |
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/incremental-indexing) |
| Query Engines (Sub-Question, etc.) | [CLI](https://docs.praison.ai/docs/cli/knowledge) | [📖](https://docs.praison.ai/docs/features/retrieval) |
| Memory (Short & Long Term) | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
| File-Based Memory | [Example](examples/python/general/memory_example.py) | [📖](https://praison.ai/docs/concepts/memory) |
| Claude Memory Tool | [Example](examples/python/memory/claude_memory_example.py) | [📖](https://praison.ai/docs/features/claude-memory-tool) |
| Add Custom Knowledge | [Example](examples/python/concepts/knowledge-agents.py) | [📖](https://praison.ai/docs/concepts/knowledge) |
| RAG Agents | [Example](examples/python/concepts/rag-agents.py) | [📖](https://praison.ai/docs/concepts/rag) |
| Chat with PDF Agents | [Example](examples/python/concepts/chat-with-pdf.py) | [📖](https://praison.ai/docs/features/chat-with-pdf) |
| Data Readers (PDF, DOCX, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/chunking-strategies) |
| Vector Store Selection | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/knowledge-backends) |
| Retrieval Strategies | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval-strategies) |
| Rerankers | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/smart-retrieval) |
| Index Types (Vector/Keyword/Hybrid) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/incremental-indexing) |
| Query Engines (Sub-Question, etc.) | [CLI](https://praison.ai/docs/cli/knowledge) | [📖](https://praison.ai/docs/features/retrieval) |
</details>
@@ -796,12 +952,12 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://docs.praison.ai/docs/agents/deep-research) |
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://docs.praison.ai/docs/agents/query-rewriter) |
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/agents/websearch) |
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://docs.praison.ai/docs/tools/tavily) |
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://docs.praison.ai/docs/tools/web-search) |
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
| Deep Research Agents | [Example](examples/python/agents/research-agent.py) | [📖](https://praison.ai/docs/agents/deep-research) |
| Query Rewriter Agent | [Example](examples/python/agents/query-rewriter-agent.py) | [📖](https://praison.ai/docs/agents/query-rewriter) |
| Native Web Search | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/agents/websearch) |
| Built-in Search Tools | [Example](examples/python/agents/websearch-agent.py) | [📖](https://praison.ai/docs/tools/tavily) |
| Unified Web Search | [Example](examples/python/web_search_example.py) | [📖](https://praison.ai/docs/tools/web-search) |
| Web Fetch (Anthropic) | [Example](examples/python/agents/web-fetch-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
</details>
@@ -810,12 +966,12 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://docs.praison.ai/docs/concepts/planning) |
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://docs.praison.ai/docs/features/promptchaining) |
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://docs.praison.ai/docs/concepts/evaluation) |
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://docs.praison.ai/docs/concepts/orchestration) |
| Planning Mode | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
| Planning Tools | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
| Planning Reasoning | [Example](examples/python/agents/planning-agent.py) | [📖](https://praison.ai/docs/concepts/planning) |
| Prompt Chaining | [Example](examples/python/general/prompt_chaining.py) | [📖](https://praison.ai/docs/features/promptchaining) |
| Evaluator Optimiser | [Example](examples/python/general/evaluator-optimiser.py) | [📖](https://praison.ai/docs/concepts/evaluation) |
| Orchestrator Workers | [Example](examples/python/general/orchestrator-workers.py) | [📖](https://praison.ai/docs/concepts/orchestration) |
</details>
@@ -824,15 +980,15 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://docs.praison.ai/docs/agents/data-analyst) |
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://docs.praison.ai/docs/agents/finance) |
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://docs.praison.ai/docs/agents/shopping) |
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://docs.praison.ai/docs/agents/recommendation) |
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://docs.praison.ai/docs/agents/wikipedia) |
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://docs.praison.ai/docs/agents/programming) |
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://docs.praison.ai/docs/features/mathagent) |
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://docs.praison.ai/docs/agents/markdown) |
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://docs.praison.ai/docs/agents/prompt-expander) |
| Data Analyst Agent | [Example](examples/python/agents/data-analyst-agent.py) | [📖](https://praison.ai/docs/agents/data-analyst) |
| Finance Agent | [Example](examples/python/agents/finance-agent.py) | [📖](https://praison.ai/docs/agents/finance) |
| Shopping Agent | [Example](examples/python/agents/shopping-agent.py) | [📖](https://praison.ai/docs/agents/shopping) |
| Recommendation Agent | [Example](examples/python/agents/recommendation-agent.py) | [📖](https://praison.ai/docs/agents/recommendation) |
| Wikipedia Agent | [Example](examples/python/agents/wikipedia-agent.py) | [📖](https://praison.ai/docs/agents/wikipedia) |
| Programming Agent | [Example](examples/python/agents/programming-agent.py) | [📖](https://praison.ai/docs/agents/programming) |
| Math Agents | [Example](examples/python/agents/math-agent.py) | [📖](https://praison.ai/docs/features/mathagent) |
| Markdown Agent | [Example](examples/python/agents/markdown-agent.py) | [📖](https://praison.ai/docs/agents/markdown) |
| Prompt Expander Agent | [Example](examples/python/agents/prompt-expander-agent.py) | [📖](https://praison.ai/docs/agents/prompt-expander) |
</details>
@@ -841,10 +997,10 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://docs.praison.ai/docs/features/image-generation) |
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://docs.praison.ai/docs/agents/image-to-text) |
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://docs.praison.ai/docs/agents/video) |
| Camera Integration | [Example](examples/python/camera/) | [📖](https://docs.praison.ai/docs/features/camera-integration) |
| Image Generation Agent | [Example](examples/python/image/image-agent.py) | [📖](https://praison.ai/docs/features/image-generation) |
| Image to Text Agent | [Example](examples/python/agents/image-to-text-agent.py) | [📖](https://praison.ai/docs/agents/image-to-text) |
| Video Agent | [Example](examples/python/agents/video-agent.py) | [📖](https://praison.ai/docs/agents/video) |
| Camera Integration | [Example](examples/python/camera/) | [📖](https://praison.ai/docs/features/camera-integration) |
</details>
@@ -853,12 +1009,12 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://docs.praison.ai/docs/mcp/transports) |
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://docs.praison.ai/docs/mcp/sse-transport) |
| MCP Config Management | [Docs](https://docs.praison.ai/docs/cli/mcp) | [📖](https://docs.praison.ai/docs/cli/mcp) |
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://docs.praison.ai/docs/features/langchain) |
| MCP Transports | [Example](examples/python/mcp/mcp-transports-overview.py) | [📖](https://praison.ai/docs/mcp/transports) |
| WebSocket MCP | [Example](examples/python/mcp/websocket-mcp.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
| MCP Security | [Example](examples/python/mcp/mcp-security.py) | [📖](https://praison.ai/docs/mcp/transports) |
| MCP Resumability | [Example](examples/python/mcp/mcp-resumability.py) | [📖](https://praison.ai/docs/mcp/sse-transport) |
| MCP Config Management | [Docs](https://praison.ai/docs/cli/mcp) | [📖](https://praison.ai/docs/cli/mcp) |
| LangChain Integrated Agents | [Example](examples/python/general/langchain_example.py) | [📖](https://praison.ai/docs/features/langchain) |
</details>
@@ -867,9 +1023,9 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://docs.praison.ai/docs/concepts/guardrails) |
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://docs.praison.ai/docs/concepts/approval) |
| Rules & Instructions | [Docs](https://docs.praison.ai/docs/features/rules) | [📖](https://docs.praison.ai/docs/features/rules) |
| Guardrails | [Example](examples/python/guardrails/comprehensive-guardrails-example.py) | [📖](https://praison.ai/docs/concepts/guardrails) |
| Human Approval | [Example](examples/python/general/human_approval_example.py) | [📖](https://praison.ai/docs/concepts/approval) |
| Rules & Instructions | [Docs](https://praison.ai/docs/features/rules) | [📖](https://praison.ai/docs/features/rules) |
</details>
@@ -878,16 +1034,16 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://docs.praison.ai/docs/features/async) |
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://docs.praison.ai/docs/features/parallelisation) |
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://docs.praison.ai/docs/features/repetitive) |
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://docs.praison.ai/docs/concepts/handoffs) |
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://docs.praison.ai/docs/features/stateful-agents) |
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://docs.praison.ai/docs/concepts/autonomy) |
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://docs.praison.ai/docs/features/structured) |
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://docs.praison.ai/docs/features/model-router) |
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://docs.praison.ai/docs/features/model-capabilities) |
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://docs.praison.ai/docs/features/fast-context) |
| Async & Parallel Processing | [Example](examples/python/general/async_example.py) | [📖](https://praison.ai/docs/features/async) |
| Parallelisation | [Example](examples/python/general/parallelisation.py) | [📖](https://praison.ai/docs/features/parallelisation) |
| Repetitive Agents | [Example](examples/python/concepts/repetitive-agents.py) | [📖](https://praison.ai/docs/features/repetitive) |
| Agent Handoffs | [Example](examples/python/handoff/handoff_basic.py) | [📖](https://praison.ai/docs/concepts/handoffs) |
| Stateful Agents | [Example](examples/python/stateful/workflow-state-example.py) | [📖](https://praison.ai/docs/features/stateful-agents) |
| Autonomous Workflow | [Example](examples/python/general/autonomous-agent.py) | [📖](https://praison.ai/docs/concepts/autonomy) |
| Structured Output Agents | [Example](examples/python/general/structured_agents_example.py) | [📖](https://praison.ai/docs/features/structured) |
| Model Router | [Example](examples/python/agents/router-agent-cost-optimization.py) | [📖](https://praison.ai/docs/features/model-router) |
| Prompt Caching | [Example](examples/python/agents/prompt-caching-agent.py) | [📖](https://praison.ai/docs/features/model-capabilities) |
| Fast Context | [Example](examples/context/00_agent_fast_context_basic.py) | [📖](https://praison.ai/docs/features/fast-context) |
</details>
@@ -896,21 +1052,21 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://docs.praison.ai/docs/tools/tools) |
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://docs.praison.ai/docs/developers/agents-playbook) |
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://docs.praison.ai/docs/models) |
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://docs.praison.ai/docs/concepts/hooks) |
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://docs.praison.ai/docs/features/middleware) |
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://docs.praison.ai/docs/features/configurable-model) |
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://docs.praison.ai/docs/features/rate-limiter) |
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://docs.praison.ai/docs/features/injected-state) |
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://docs.praison.ai/docs/features/checkpoints) |
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://docs.praison.ai/docs/features/background-tasks) |
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://docs.praison.ai/docs/features/policy-engine) |
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://docs.praison.ai/docs/features/thinking-budgets) |
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://docs.praison.ai/docs/features/output-styles) |
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://docs.praison.ai/docs/features/context-compaction) |
| 100+ Custom Tools | [Example](examples/python/general/tools_example.py) | [📖](https://praison.ai/docs/tools/tools) |
| YAML Configuration | [Example](examples/cookbooks/yaml/secondary_market_research_agents.yaml) | [📖](https://praison.ai/docs/developers/agents-playbook) |
| 100+ LLM Support | [Example](examples/python/providers/openai/openai_gpt4_example.py) | [📖](https://praison.ai/docs/models) |
| Callback Agents | [Example](examples/python/general/advanced-callback-systems.py) | [📖](https://praison.ai/docs/concepts/hooks) |
| Hooks | [Example](examples/python/hooks/hooks_example.py) | [📖](https://praison.ai/docs/concepts/hooks) |
| Middleware System | [Example](examples/middleware/basic_middleware.py) | [📖](https://praison.ai/docs/features/middleware) |
| Configurable Model | [Example](examples/middleware/configurable_model.py) | [📖](https://praison.ai/docs/features/configurable-model) |
| Rate Limiter | [Example](examples/middleware/rate_limiter.py) | [📖](https://praison.ai/docs/features/rate-limiter) |
| Injected Tool State | [Example](examples/middleware/injected_state.py) | [📖](https://praison.ai/docs/features/injected-state) |
| Shadow Git Checkpoints | [Example](examples/checkpoints/basic_checkpoint.py) | [📖](https://praison.ai/docs/features/checkpoints) |
| Background Tasks | [Example](examples/background/basic_background.py) | [📖](https://praison.ai/docs/features/background-tasks) |
| Policy Engine | [Example](examples/policy/basic_policy.py) | [📖](https://praison.ai/docs/features/policy-engine) |
| Thinking Budgets | [Example](examples/thinking/basic_thinking.py) | [📖](https://praison.ai/docs/features/thinking-budgets) |
| Output Styles | [Example](examples/output/basic_output.py) | [📖](https://praison.ai/docs/features/output-styles) |
| Context Compaction | [Example](examples/compaction/basic_compaction.py) | [📖](https://praison.ai/docs/features/context-compaction) |
</details>
@@ -919,14 +1075,14 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://docs.praison.ai/docs/concepts/session-management) |
| Auto-Save Sessions | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
| History in Context | [Docs](https://docs.praison.ai/docs/cli/session) | [📖](https://docs.praison.ai/docs/cli/session) |
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://docs.praison.ai/docs/features/telemetry) |
| Langfuse Tracing | [Docs](https://docs.praison.ai/docs/observability/langfuse) | [📖](https://docs.praison.ai/docs/observability/langfuse) |
| Project Docs (.praison/docs/) | [Docs](https://docs.praison.ai/docs/cli/docs) | [📖](https://docs.praison.ai/docs/cli/docs) |
| AI Commit Messages | [Docs](https://docs.praison.ai/docs/cli/commit) | [📖](https://docs.praison.ai/docs/cli/commit) |
| @Mentions in Prompts | [Docs](https://docs.praison.ai/docs/cli/mentions) | [📖](https://docs.praison.ai/docs/cli/mentions) |
| Sessions Management | [Example](examples/python/sessions/comprehensive-session-management.py) | [📖](https://praison.ai/docs/concepts/session-management) |
| Auto-Save Sessions | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
| History in Context | [Docs](https://praison.ai/docs/cli/session) | [📖](https://praison.ai/docs/cli/session) |
| Telemetry | [Example](examples/python/telemetry/production-telemetry-example.py) | [📖](https://praison.ai/docs/features/telemetry) |
| Langfuse Tracing | [Docs](https://praison.ai/docs/observability/langfuse) | [📖](https://praison.ai/docs/observability/langfuse) |
| Project Docs (.praison/docs/) | [Docs](https://praison.ai/docs/cli/docs) | [📖](https://praison.ai/docs/cli/docs) |
| AI Commit Messages | [Docs](https://praison.ai/docs/cli/commit) | [📖](https://praison.ai/docs/cli/commit) |
| @Mentions in Prompts | [Docs](https://praison.ai/docs/cli/mentions) | [📖](https://praison.ai/docs/cli/mentions) |
</details>
@@ -935,20 +1091,20 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://docs.praison.ai/docs/cli/slash-commands) |
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://docs.praison.ai/docs/cli/autonomy-modes) |
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://docs.praison.ai/docs/cli/cost-tracking) |
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://docs.praison.ai/docs/cli/repo-map) |
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://docs.praison.ai/docs/cli/interactive-tui) |
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://docs.praison.ai/docs/cli/git-integration) |
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://docs.praison.ai/docs/cli/sandbox-execution) |
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://docs.praison.ai/docs/cli/compare) |
| Profile/Benchmark | [Docs](https://docs.praison.ai/docs/cli/profile) | [📖](https://docs.praison.ai/docs/cli/profile) |
| Auto Mode | [Docs](https://docs.praison.ai/docs/cli/auto) | [📖](https://docs.praison.ai/docs/cli/auto) |
| Init | [Docs](https://docs.praison.ai/docs/cli/init) | [📖](https://docs.praison.ai/docs/cli/init) |
| File Input | [Docs](https://docs.praison.ai/docs/cli/file-input) | [📖](https://docs.praison.ai/docs/cli/file-input) |
| Final Agent | [Docs](https://docs.praison.ai/docs/cli/final-agent) | [📖](https://docs.praison.ai/docs/cli/final-agent) |
| Max Tokens | [Docs](https://docs.praison.ai/docs/cli/max-tokens) | [📖](https://docs.praison.ai/docs/cli/max-tokens) |
| Slash Commands | [Example](examples/python/cli/slash_commands_example.py) | [📖](https://praison.ai/docs/cli/slash-commands) |
| Autonomy Modes | [Example](examples/python/cli/autonomy_modes_example.py) | [📖](https://praison.ai/docs/cli/autonomy-modes) |
| Cost Tracking | [Example](examples/python/cli/cost_tracking_example.py) | [📖](https://praison.ai/docs/cli/cost-tracking) |
| Repository Map | [Example](examples/python/cli/repo_map_example.py) | [📖](https://praison.ai/docs/cli/repo-map) |
| Interactive TUI | [Example](examples/python/cli/interactive_tui_example.py) | [📖](https://praison.ai/docs/cli/interactive-tui) |
| Git Integration | [Example](examples/python/cli/git_integration_example.py) | [📖](https://praison.ai/docs/cli/git-integration) |
| Sandbox Execution | [Example](examples/python/cli/sandbox_execution_example.py) | [📖](https://praison.ai/docs/cli/sandbox-execution) |
| CLI Compare | [Example](examples/compare/cli_compare_basic.py) | [📖](https://praison.ai/docs/cli/compare) |
| Profile/Benchmark | [Docs](https://praison.ai/docs/cli/profile) | [📖](https://praison.ai/docs/cli/profile) |
| Auto Mode | [Docs](https://praison.ai/docs/cli/auto) | [📖](https://praison.ai/docs/cli/auto) |
| Init | [Docs](https://praison.ai/docs/cli/init) | [📖](https://praison.ai/docs/cli/init) |
| File Input | [Docs](https://praison.ai/docs/cli/file-input) | [📖](https://praison.ai/docs/cli/file-input) |
| Final Agent | [Docs](https://praison.ai/docs/cli/final-agent) | [📖](https://praison.ai/docs/cli/final-agent) |
| Max Tokens | [Docs](https://praison.ai/docs/cli/max-tokens) | [📖](https://praison.ai/docs/cli/max-tokens) |
</details>
@@ -957,10 +1113,10 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://docs.praison.ai/docs/cli/eval) |
| Accuracy Evaluation | [Example](examples/eval/accuracy_example.py) | [📖](https://praison.ai/docs/cli/eval) |
| Performance Evaluation | [Example](examples/eval/performance_example.py) | [📖](https://praison.ai/docs/cli/eval) |
| Reliability Evaluation | [Example](examples/eval/reliability_example.py) | [📖](https://praison.ai/docs/cli/eval) |
| Criteria Evaluation | [Example](examples/eval/criteria_example.py) | [📖](https://praison.ai/docs/cli/eval) |
</details>
@@ -969,8 +1125,8 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://docs.praison.ai/docs/concepts/skills) |
| Skills Management | [Example](examples/skills/basic_skill_usage.py) | [📖](https://praison.ai/docs/concepts/skills) |
| Custom Skills | [Example](examples/skills/custom_skill_example.py) | [📖](https://praison.ai/docs/concepts/skills) |
</details>
@@ -979,7 +1135,9 @@ praisonai agents.yaml
| Feature | Code | Docs |
|---------|:----:|:----:|
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://docs.praison.ai/docs/cli/scheduler) |
| Agent Scheduler | [Example](examples/python/scheduled_agents/news_checker_live.py) | [📖](https://praison.ai/docs/cli/scheduler) |
</details>
</details>
@@ -1008,7 +1166,7 @@ PraisonAI keeps framework overhead microsecond-class: creating an `Agent` (one t
|--------------------|-----------|----------|
| Avg Agent instantiation (1 tool, 1000 iterations) | **46 μs** | 2026-10-15 · praisonaiagents 1.7.12 |
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://docs.praison.ai/docs/developers/benchmarks)
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://praison.ai/docs/developers/benchmarks)
---
@@ -1018,7 +1176,7 @@ Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md]
## ⭐ Star History
[![Star History Chart](https://api.star-history.com/svg?repos=MervinPraison/PraisonAI&type=Date)](https://docs.praison.ai)
[![Star History Chart](https://api.star-history.com/svg?repos=MervinPraison/PraisonAI&type=Date)](https://praison.ai/docs)
---
@@ -1089,6 +1247,9 @@ We welcome contributions! Fork the repo, create a branch, and submit a PR → [C
## ❓ FAQ & Troubleshooting
<details>
<summary><strong>Show FAQ and troubleshooting</strong></summary>
<details>
<summary><strong>ModuleNotFoundError: No module named 'praisonaiagents'</strong></summary>
@@ -1107,7 +1268,7 @@ Ensure your API key is set:
export OPENAI_API_KEY=your_key_here
```
For other providers, see [Models docs](https://docs.praison.ai/docs/models).
For other providers, see [Models docs](https://praison.ai/docs/models).
</details>
@@ -1148,7 +1309,7 @@ agent = Agent(instructions="...", llm="ollama/llama3.2", base_url="http://192.16
The same shape works for other local runtimes — `lm_studio/`, `vllm/` and
`hosted_vllm/` with their server's `base_url`.
See [Models docs](https://docs.praison.ai/docs/models) for more details.
See [Models docs](https://praison.ai/docs/models) for more details.
</details>
@@ -1168,7 +1329,7 @@ agent = Agent(
)
```
See [Persistence docs](https://docs.praison.ai/docs/databases/overview) for supported databases.
See [Persistence docs](https://praison.ai/docs/databases/overview) for supported databases.
</details>
@@ -1185,7 +1346,7 @@ agent = Agent(
)
```
See [Memory docs](https://docs.praison.ai/docs/concepts/memory) for more options.
See [Memory docs](https://praison.ai/docs/concepts/memory) for more options.
</details>
@@ -1201,7 +1362,7 @@ agents = Agents(agents=[agent1, agent2])
agents.start()
```
See [Agents docs](https://docs.praison.ai/docs/concepts/agents) for more examples.
See [Agents docs](https://praison.ai/docs/concepts/agents) for more examples.
</details>
@@ -1216,13 +1377,15 @@ agent = Agent(
)
```
See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport options.
See [MCP docs](https://praison.ai/docs/mcp/transports) for all transport options.
</details>
</details>
### Getting Help
- 📚 [Full Documentation](https://docs.praison.ai)
- 📚 [Full Documentation](https://praison.ai/docs)
- 🐛 [Report Issues](https://github.com/MervinPraison/PraisonAI/issues)
- 💬 [Discussions](https://github.com/MervinPraison/PraisonAI/discussions)
@@ -1231,7 +1394,7 @@ See [MCP docs](https://docs.praison.ai/docs/mcp/transports) for all transport op
<div align="center">
<p><strong>Made with ❤️ by the PraisonAI Team</strong></p>
<p>
<a href="https://docs.praison.ai">📚 Documentation</a> •
<a href="https://praison.ai/docs">📚 Documentation</a> •
<a href="https://github.com/MervinPraison/PraisonAI">GitHub</a> •
<a href="https://youtube.com/@MervinPraison">▶️ YouTube</a> •
<a href="https://x.com/MervinPraison">𝕏 X</a> •
@@ -10,6 +10,7 @@ from typing import Optional, Dict, Any
from ..diff.diff_strategy import apply_search_replace_diff
from ..utils.file_utils import (
atomic_write,
file_exists,
is_path_within_directory,
)
@@ -142,8 +143,7 @@ def apply_diff(
f.write(original_content)
# Write the modified content
with open(abs_path, 'w', encoding=encoding) as f:
f.write(result.content)
atomic_write(abs_path, result.content, encoding=encoding)
return {
'success': True,
@@ -10,6 +10,7 @@ import re
from typing import Optional, List, Dict, Any
from ..utils.file_utils import (
atomic_write,
file_exists,
is_path_within_directory,
)
@@ -191,8 +192,7 @@ def search_replace(
f.write(original_content)
# Write the modified content
with open(abs_path, 'w', encoding=encoding) as f:
f.write(content)
atomic_write(abs_path, content, encoding=encoding)
return {
'success': True,
@@ -8,6 +8,7 @@ import os
from typing import Optional, Dict, Any
from ..utils.file_utils import (
atomic_write,
file_exists,
create_directories_for_file,
is_path_within_directory,
@@ -131,8 +132,7 @@ def write_file(
pass # If we can't read, just use the content as-is
# Write the file
with open(abs_path, 'w', encoding=encoding) as f:
f.write(processed_content)
atomic_write(abs_path, processed_content, encoding=encoding)
return {
'success': True,
Loaded 100 of 111 files, more files were not shown because too many files have changed in this diff. Show more